docs(runbooks): add slow_page_load and slow_form_fill runbooks with o…#41
docs(runbooks): add slow_page_load and slow_form_fill runbooks with o…#41subodh2711 merged 3 commits intodevfrom
Conversation
…ptel-query debugging - Add new runbooks for slow_page_load (LCP) and slow_form_fill violation types - Add "Debugging with optel-query" section to all four runbooks with violation-specific CLI commands, output annotations, natural language prompts, and a debugging workflow summary - Update index.md with new performance violation entries and an optel-query overview section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
There was a problem hiding this comment.
Code Review
This pull request introduces new runbooks for 'Slow Form Fill' and 'Slow Page Load (LCP)' and updates existing documentation for 'Session Error Count' and 'TooManyCallsException' to include debugging instructions for the optel-query tool. The reviewer feedback focuses on improving documentation quality by replacing hardcoded future dates with placeholders, fixing markdown table formatting, and using placeholders for 'Last Updated' fields.
| node tools/rum-query/cli.js example.com 2026-01-20 2026-01-20 \ | ||
| --query '{"url":["https://example.com/checkout"],"checkpoint":["error"]}' \ | ||
| --facet-values error | ||
|
|
||
| # Example: get LCP metrics for a URL | ||
| node tools/rum-query/cli.js example.com 2026-01-20 2026-01-20 \ | ||
| --query '{"url":["https://example.com/apply"],"checkpoint":["lcp"]}' \ | ||
| --series lcp,cls,inp | ||
| ``` | ||
| "Show me all errors on /checkout for 2026-01-20" | ||
|
|
||
| "Get the errors which are coming on 2026-01-20 but not occurring on 2026-01-13" | ||
| #### 2. Natural Language (via AI skill in Claude Code or Cursor) | ||
|
|
||
| "Which browsers and devices are experiencing errors on /checkout?" | ||
| Ask questions in plain English — the skill builds and runs the CLI command for you: | ||
|
|
||
| "What are the most common errors on example.com today?" | ||
| ``` | ||
| "Show me all errors on /checkout for 2026-01-20" | ||
| "Compare LCP on /apply between today and last week" | ||
| "Which forms had the most validation errors today?" | ||
| "Are mobile users submitting fewer forms than desktop users?" |
There was a problem hiding this comment.
| - `"Show me all errors on https://example.com/checkout on 2026-01-20"` | ||
| - `"Which errors appeared on 2026-01-20 but were not present on 2026-01-13 for https://example.com/checkout?"` | ||
| - `"What is the most common error on https://example.com/checkout today?"` | ||
| - `"Compare error counts between mobile and desktop on https://example.com/checkout for 2026-01-20"` | ||
| - `"Which pages on example.com have the most errors today?"` | ||
| - `"Show me errors from payment.js on https://example.com/checkout on 2026-01-20"` |
There was a problem hiding this comment.
The example prompts use future dates (2026-01-20, 2026-01-13), which is inconsistent with the CLI examples in this runbook that use placeholders like <alert-date>. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.
| - `"Show me all errors on https://example.com/checkout on 2026-01-20"` | |
| - `"Which errors appeared on 2026-01-20 but were not present on 2026-01-13 for https://example.com/checkout?"` | |
| - `"What is the most common error on https://example.com/checkout today?"` | |
| - `"Compare error counts between mobile and desktop on https://example.com/checkout for 2026-01-20"` | |
| - `"Which pages on example.com have the most errors today?"` | |
| - `"Show me errors from payment.js on https://example.com/checkout on 2026-01-20"` | |
| - `"Show me all errors on https://example.com/checkout on <date>"` | |
| - `"Which errors appeared on <date> but were not present on <baseline-date> for https://example.com/checkout?"` | |
| - `"What is the most common error on https://example.com/checkout today?"` | |
| - `"Compare error counts between mobile and desktop on https://example.com/checkout for <date>"` | |
| - `"Which pages on example.com have the most errors today?"` | |
| - `"Show me errors from payment.js on https://example.com/checkout on <date>"` |
| ### Scenario: Loan application form taking 2× longer to complete | ||
|
|
||
| #### Step 1: Initial Assessment | ||
| **Query:** `"Show me form fill duration for https://example.com/apply/loan on 2026-03-20"` |
There was a problem hiding this comment.
| - `"How many users submitted the form on https://example.com/apply/loan on 2026-03-20?"` | ||
| - `"Compare form submissions on https://example.com/apply/loan between 2026-03-20 and 2026-03-13"` | ||
| - `"Which form fields have the most validation errors on https://example.com/apply/loan today?"` | ||
| - `"Show me which fields users are filling on https://example.com/apply/loan on 2026-03-20"` | ||
| - `"What is the form block load time for https://example.com/apply/loan today?"` | ||
| - `"Are mobile users submitting fewer forms than desktop users on https://example.com/apply/loan today?"` |
There was a problem hiding this comment.
The example prompts use future dates (2026-03-20, 2026-03-13). This is inconsistent with the CLI examples in this runbook that use placeholders. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.
| - `"How many users submitted the form on https://example.com/apply/loan on 2026-03-20?"` | |
| - `"Compare form submissions on https://example.com/apply/loan between 2026-03-20 and 2026-03-13"` | |
| - `"Which form fields have the most validation errors on https://example.com/apply/loan today?"` | |
| - `"Show me which fields users are filling on https://example.com/apply/loan on 2026-03-20"` | |
| - `"What is the form block load time for https://example.com/apply/loan today?"` | |
| - `"Are mobile users submitting fewer forms than desktop users on https://example.com/apply/loan today?"` | |
| - `"How many users submitted the form on https://example.com/apply/loan on <date>?"` | |
| - `"Compare form submissions on https://example.com/apply/loan between <date> and <baseline-date>"` | |
| - `"Which form fields have the most validation errors on https://example.com/apply/loan today?"` | |
| - `"Show me which fields users are filling on https://example.com/apply/loan on <date>"` | |
| - `"What is the form block load time for https://example.com/apply/loan today?"` | |
| - `"Are mobile users submitting fewer forms than desktop users on https://example.com/apply/loan today?"` |
|
|
||
| **Query parameters (all optional):** | ||
| | Parameter | Description | | ||
| |----------------|--------------------------------------------------| |
There was a problem hiding this comment.
The markdown for this table's header separator is shorter than the header text. While many parsers handle this, it's best practice to make the separator at least as long as the header text for consistent rendering across different markdown viewers. It also improves readability of the raw markdown.
| |----------------|--------------------------------------------------| | |
| |-----------------|--------------------------------------------------| |
| ### Scenario: LCP spike on a high-traffic form landing page | ||
|
|
||
| #### Step 1: Initial Assessment | ||
| **Query:** `"Show me LCP performance for https://example.com/apply on 2026-03-15"` |
There was a problem hiding this comment.
| - `"What is the LCP for https://example.com/apply on 2026-03-15?"` | ||
| - `"Compare LCP on https://example.com/apply between 2026-03-15 and 2026-03-08"` | ||
| - `"Which pages on example.com have the worst LCP today?"` | ||
| - `"Show me LCP broken down by device type for https://example.com/apply on 2026-03-15"` | ||
| - `"Are there any missing resources on https://example.com/apply today?"` |
There was a problem hiding this comment.
The example prompts use future dates (2026-03-15, 2026-03-08). This is inconsistent with the CLI examples in this runbook that use placeholders. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.
| - `"What is the LCP for https://example.com/apply on 2026-03-15?"` | |
| - `"Compare LCP on https://example.com/apply between 2026-03-15 and 2026-03-08"` | |
| - `"Which pages on example.com have the worst LCP today?"` | |
| - `"Show me LCP broken down by device type for https://example.com/apply on 2026-03-15"` | |
| - `"Are there any missing resources on https://example.com/apply today?"` | |
| - `"What is the LCP for https://example.com/apply on <date>?"` | |
| - `"Compare LCP on https://example.com/apply between <date> and <baseline-date>"` | |
| - `"Which pages on example.com have the worst LCP today?"` | |
| - `"Show me LCP broken down by device type for https://example.com/apply on <date>"` | |
| - `"Are there any missing resources on https://example.com/apply today?"` |
|
|
||
| **Query parameters (all optional):** | ||
| | Parameter | Description | | ||
| |----------------|--------------------------------------------------| |
There was a problem hiding this comment.
The markdown for this table's header separator is shorter than the header text. While many parsers handle this, it's best practice to make the separator at least as long as the header text for consistent rendering across different markdown viewers. It also improves readability of the raw markdown.
| |----------------|--------------------------------------------------| | |
| |-----------------|--------------------------------------------------| |
|
|
||
| --- | ||
|
|
||
| **Last Updated:** 2026-03-30 |
| - `"Show me all TooManyCallsException errors on https://example.com/apply/credit-card on 2026-01-21"` | ||
| - `"What percentage of page views had errors on https://example.com/apply/credit-card on 2026-01-21?"` | ||
| - `"Which forms on example.com had the most errors on 2026-01-21?"` | ||
| - `"Compare errors on https://example.com/apply/credit-card between 2026-01-21 and 2026-01-14"` | ||
| - `"Did form submissions drop on https://example.com/apply/credit-card on 2026-01-21 compared to last week?"` |
There was a problem hiding this comment.
The example prompts use future dates (2026-01-21, 2026-01-14). This is inconsistent with the CLI examples in this runbook that use placeholders. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.
| - `"Show me all TooManyCallsException errors on https://example.com/apply/credit-card on 2026-01-21"` | |
| - `"What percentage of page views had errors on https://example.com/apply/credit-card on 2026-01-21?"` | |
| - `"Which forms on example.com had the most errors on 2026-01-21?"` | |
| - `"Compare errors on https://example.com/apply/credit-card between 2026-01-21 and 2026-01-14"` | |
| - `"Did form submissions drop on https://example.com/apply/credit-card on 2026-01-21 compared to last week?"` | |
| - `"Show me all TooManyCallsException errors on https://example.com/apply/credit-card on <date>"` | |
| - `"What percentage of page views had errors on https://example.com/apply/credit-card on <date>?"` | |
| - `"Which forms on example.com had the most errors on <date>?"` | |
| - `"Compare errors on https://example.com/apply/credit-card between <date> and <baseline-date>"` | |
| - `"Did form submissions drop on https://example.com/apply/credit-card on <date> compared to last week?"` |
…ptel-query debugging Add runbooks for the four previously undocumented violation types: - low_hourly_pageviews: traffic drop during IST business hours - high_api_errors: missing JSON resources per session exceeding baseline - slow_api_response: JSON API response time exceeding baseline - high_external_redirect_error_rate: elevated JS errors for users from a specific external referrer Each runbook includes detection thresholds, common causes, resolution steps, a worked example, optel-query CLI commands specific to that violation type, natural language query examples, and Alert API resolution steps. Update index.md to list all runbooks under organised violation-type categories. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…code Corrected minimum sample size thresholds and z-score/drop conditions across all seven runbooks to match the values in optel-alerts/src/optel/loader.js: - slow_page_load: min sessions 10 → >20 (lcpCount > 20) - slow_form_fill: "at least 5" → "more than 5" (submissionCount > 5) - high_api_errors: sessionCount > 10 → > 20 - low_hourly_pageviews: removed incorrect "no z-score" claim; alert requires z-score < -2 AND >20% drop - session_error_count: min page views 10 → >20 - slow_api_response: requestCount > 10 → > 20 - high_external_redirect_error_rate: sessionCount >= 10 → >= 20 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ptel-query debugging
Description
PR Review Checklist
npm run lint); no new violations introduced.Preview URLs
Before (baseline): https://dev--forms-dashboard--adobe-rnd.aem.live/index.html
After (this branch): https://docs-runbooks-slow-violations--forms-dashboard--adobe-rnd.aem.live/index.html