Skip to content

docs(runbooks): add slow_page_load and slow_form_fill runbooks with o…#41

Merged
subodh2711 merged 3 commits intodevfrom
docs/runbooks-slow-violations
Apr 4, 2026
Merged

docs(runbooks): add slow_page_load and slow_form_fill runbooks with o…#41
subodh2711 merged 3 commits intodevfrom
docs/runbooks-slow-violations

Conversation

@subodh2711
Copy link
Copy Markdown
Collaborator

@subodh2711 subodh2711 commented Mar 30, 2026

…ptel-query debugging

  • Add new runbooks for slow_page_load (LCP) and slow_form_fill violation types
  • Add "Debugging with optel-query" section to all four runbooks with violation-specific CLI commands, output annotations, natural language prompts, and a debugging workflow summary
  • Update index.md with new performance violation entries and an optel-query overview section

Description

PR Review Checklist

  • Change is described and scoped (issue link or short description above).
  • Tests/checks: Unit and/or E2E tests were run (or N/A with reason).
  • Lint passes (npm run lint); no new violations introduced.
  • Documentation / agent context (e.g. CLAUDE.md, .specify/) updated if this change affects them (or N/A).
  • If implementing a spec: spec/plan were followed and tasks.md (or equivalent) reflected (or N/A).

Preview URLs

Before (baseline): https://dev--forms-dashboard--adobe-rnd.aem.live/index.html
After (this branch): https://docs-runbooks-slow-violations--forms-dashboard--adobe-rnd.aem.live/index.html

…ptel-query debugging

- Add new runbooks for slow_page_load (LCP) and slow_form_fill violation types
- Add "Debugging with optel-query" section to all four runbooks with violation-specific CLI commands, output annotations, natural language prompts, and a debugging workflow summary
- Update index.md with new performance violation entries and an optel-query overview section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@aem-code-sync
Copy link
Copy Markdown

aem-code-sync Bot commented Mar 30, 2026

Hello, I'm the AEM Code Sync Bot and I will run some actions to deploy your branch and validate page speed.
In case there are problems, just click a checkbox below to rerun the respective action.

  • Re-run PSI checks
  • Re-sync branch
Commits

@aem-code-sync
Copy link
Copy Markdown

aem-code-sync Bot commented Mar 30, 2026

Page Scores Audits Google
📱 /index.html PERFORMANCE A11Y SEO BEST PRACTICES SI FCP LCP TBT CLS PSI
🖥️ /index.html PERFORMANCE A11Y SEO BEST PRACTICES SI FCP LCP TBT CLS PSI

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces new runbooks for 'Slow Form Fill' and 'Slow Page Load (LCP)' and updates existing documentation for 'Session Error Count' and 'TooManyCallsException' to include debugging instructions for the optel-query tool. The reviewer feedback focuses on improving documentation quality by replacing hardcoded future dates with placeholders, fixing markdown table formatting, and using placeholders for 'Last Updated' fields.

Comment thread docs/runbooks/index.md
Comment on lines +50 to +68
node tools/rum-query/cli.js example.com 2026-01-20 2026-01-20 \
--query '{"url":["https://example.com/checkout"],"checkpoint":["error"]}' \
--facet-values error

# Example: get LCP metrics for a URL
node tools/rum-query/cli.js example.com 2026-01-20 2026-01-20 \
--query '{"url":["https://example.com/apply"],"checkpoint":["lcp"]}' \
--series lcp,cls,inp
```
"Show me all errors on /checkout for 2026-01-20"

"Get the errors which are coming on 2026-01-20 but not occurring on 2026-01-13"
#### 2. Natural Language (via AI skill in Claude Code or Cursor)

"Which browsers and devices are experiencing errors on /checkout?"
Ask questions in plain English — the skill builds and runs the CLI command for you:

"What are the most common errors on example.com today?"
```
"Show me all errors on /checkout for 2026-01-20"
"Compare LCP on /apply between today and last week"
"Which forms had the most validation errors today?"
"Are mobile users submitting fewer forms than desktop users?"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example commands and queries use a future date 2026-01-20. This could be confusing for users. For clarity and consistency with the other runbooks, it would be better to use placeholders like <date> or rely on relative terms like "today", which is already used in other examples here.

Comment on lines +513 to +518
- `"Show me all errors on https://example.com/checkout on 2026-01-20"`
- `"Which errors appeared on 2026-01-20 but were not present on 2026-01-13 for https://example.com/checkout?"`
- `"What is the most common error on https://example.com/checkout today?"`
- `"Compare error counts between mobile and desktop on https://example.com/checkout for 2026-01-20"`
- `"Which pages on example.com have the most errors today?"`
- `"Show me errors from payment.js on https://example.com/checkout on 2026-01-20"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example prompts use future dates (2026-01-20, 2026-01-13), which is inconsistent with the CLI examples in this runbook that use placeholders like <alert-date>. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.

Suggested change
- `"Show me all errors on https://example.com/checkout on 2026-01-20"`
- `"Which errors appeared on 2026-01-20 but were not present on 2026-01-13 for https://example.com/checkout?"`
- `"What is the most common error on https://example.com/checkout today?"`
- `"Compare error counts between mobile and desktop on https://example.com/checkout for 2026-01-20"`
- `"Which pages on example.com have the most errors today?"`
- `"Show me errors from payment.js on https://example.com/checkout on 2026-01-20"`
- `"Show me all errors on https://example.com/checkout on <date>"`
- `"Which errors appeared on <date> but were not present on <baseline-date> for https://example.com/checkout?"`
- `"What is the most common error on https://example.com/checkout today?"`
- `"Compare error counts between mobile and desktop on https://example.com/checkout for <date>"`
- `"Which pages on example.com have the most errors today?"`
- `"Show me errors from payment.js on https://example.com/checkout on <date>"`

### Scenario: Loan application form taking 2× longer to complete

#### Step 1: Initial Assessment
**Query:** `"Show me form fill duration for https://example.com/apply/loan on 2026-03-20"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This example workflow, and others throughout the document, use a future date (2026-03-20). This can be confusing for the reader. It would be clearer to use placeholders like [date] or [current-date] to indicate that these values should be replaced by the user.

Comment on lines +463 to +468
- `"How many users submitted the form on https://example.com/apply/loan on 2026-03-20?"`
- `"Compare form submissions on https://example.com/apply/loan between 2026-03-20 and 2026-03-13"`
- `"Which form fields have the most validation errors on https://example.com/apply/loan today?"`
- `"Show me which fields users are filling on https://example.com/apply/loan on 2026-03-20"`
- `"What is the form block load time for https://example.com/apply/loan today?"`
- `"Are mobile users submitting fewer forms than desktop users on https://example.com/apply/loan today?"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example prompts use future dates (2026-03-20, 2026-03-13). This is inconsistent with the CLI examples in this runbook that use placeholders. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.

Suggested change
- `"How many users submitted the form on https://example.com/apply/loan on 2026-03-20?"`
- `"Compare form submissions on https://example.com/apply/loan between 2026-03-20 and 2026-03-13"`
- `"Which form fields have the most validation errors on https://example.com/apply/loan today?"`
- `"Show me which fields users are filling on https://example.com/apply/loan on 2026-03-20"`
- `"What is the form block load time for https://example.com/apply/loan today?"`
- `"Are mobile users submitting fewer forms than desktop users on https://example.com/apply/loan today?"`
- `"How many users submitted the form on https://example.com/apply/loan on <date>?"`
- `"Compare form submissions on https://example.com/apply/loan between <date> and <baseline-date>"`
- `"Which form fields have the most validation errors on https://example.com/apply/loan today?"`
- `"Show me which fields users are filling on https://example.com/apply/loan on <date>"`
- `"What is the form block load time for https://example.com/apply/loan today?"`
- `"Are mobile users submitting fewer forms than desktop users on https://example.com/apply/loan today?"`


**Query parameters (all optional):**
| Parameter | Description |
|----------------|--------------------------------------------------|
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The markdown for this table's header separator is shorter than the header text. While many parsers handle this, it's best practice to make the separator at least as long as the header text for consistent rendering across different markdown viewers. It also improves readability of the raw markdown.

Suggested change
|----------------|--------------------------------------------------|
|-----------------|--------------------------------------------------|

### Scenario: LCP spike on a high-traffic form landing page

#### Step 1: Initial Assessment
**Query:** `"Show me LCP performance for https://example.com/apply on 2026-03-15"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This example workflow, and others throughout the document, use a future date (2026-03-15). This can be confusing for the reader. It would be clearer to use placeholders like [date] or [current-date] to indicate that these values should be replaced by the user.

Comment on lines +447 to +451
- `"What is the LCP for https://example.com/apply on 2026-03-15?"`
- `"Compare LCP on https://example.com/apply between 2026-03-15 and 2026-03-08"`
- `"Which pages on example.com have the worst LCP today?"`
- `"Show me LCP broken down by device type for https://example.com/apply on 2026-03-15"`
- `"Are there any missing resources on https://example.com/apply today?"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example prompts use future dates (2026-03-15, 2026-03-08). This is inconsistent with the CLI examples in this runbook that use placeholders. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.

Suggested change
- `"What is the LCP for https://example.com/apply on 2026-03-15?"`
- `"Compare LCP on https://example.com/apply between 2026-03-15 and 2026-03-08"`
- `"Which pages on example.com have the worst LCP today?"`
- `"Show me LCP broken down by device type for https://example.com/apply on 2026-03-15"`
- `"Are there any missing resources on https://example.com/apply today?"`
- `"What is the LCP for https://example.com/apply on <date>?"`
- `"Compare LCP on https://example.com/apply between <date> and <baseline-date>"`
- `"Which pages on example.com have the worst LCP today?"`
- `"Show me LCP broken down by device type for https://example.com/apply on <date>"`
- `"Are there any missing resources on https://example.com/apply today?"`


**Query parameters (all optional):**
| Parameter | Description |
|----------------|--------------------------------------------------|
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The markdown for this table's header separator is shorter than the header text. While many parsers handle this, it's best practice to make the separator at least as long as the header text for consistent rendering across different markdown viewers. It also improves readability of the raw markdown.

Suggested change
|----------------|--------------------------------------------------|
|-----------------|--------------------------------------------------|


---

**Last Updated:** 2026-03-30
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The Last Updated date is set to a future date (2026-03-30). This should be the date the document was last modified. Consider using a placeholder like YYYY-MM-DD.

Suggested change
**Last Updated:** 2026-03-30
**Last Updated:** YYYY-MM-DD

Comment on lines +526 to +530
- `"Show me all TooManyCallsException errors on https://example.com/apply/credit-card on 2026-01-21"`
- `"What percentage of page views had errors on https://example.com/apply/credit-card on 2026-01-21?"`
- `"Which forms on example.com had the most errors on 2026-01-21?"`
- `"Compare errors on https://example.com/apply/credit-card between 2026-01-21 and 2026-01-14"`
- `"Did form submissions drop on https://example.com/apply/credit-card on 2026-01-21 compared to last week?"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example prompts use future dates (2026-01-21, 2026-01-14). This is inconsistent with the CLI examples in this runbook that use placeholders. Using placeholders like <date> or <baseline-date> would improve clarity and consistency.

Suggested change
- `"Show me all TooManyCallsException errors on https://example.com/apply/credit-card on 2026-01-21"`
- `"What percentage of page views had errors on https://example.com/apply/credit-card on 2026-01-21?"`
- `"Which forms on example.com had the most errors on 2026-01-21?"`
- `"Compare errors on https://example.com/apply/credit-card between 2026-01-21 and 2026-01-14"`
- `"Did form submissions drop on https://example.com/apply/credit-card on 2026-01-21 compared to last week?"`
- `"Show me all TooManyCallsException errors on https://example.com/apply/credit-card on <date>"`
- `"What percentage of page views had errors on https://example.com/apply/credit-card on <date>?"`
- `"Which forms on example.com had the most errors on <date>?"`
- `"Compare errors on https://example.com/apply/credit-card between <date> and <baseline-date>"`
- `"Did form submissions drop on https://example.com/apply/credit-card on <date> compared to last week?"`

…ptel-query debugging

Add runbooks for the four previously undocumented violation types:
- low_hourly_pageviews: traffic drop during IST business hours
- high_api_errors: missing JSON resources per session exceeding baseline
- slow_api_response: JSON API response time exceeding baseline
- high_external_redirect_error_rate: elevated JS errors for users from a specific external referrer

Each runbook includes detection thresholds, common causes, resolution steps, a
worked example, optel-query CLI commands specific to that violation type, natural
language query examples, and Alert API resolution steps.

Update index.md to list all runbooks under organised violation-type categories.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@aem-code-sync aem-code-sync Bot temporarily deployed to docs/runbooks-slow-violations March 30, 2026 09:37 Inactive
…code

Corrected minimum sample size thresholds and z-score/drop conditions across
all seven runbooks to match the values in optel-alerts/src/optel/loader.js:
- slow_page_load: min sessions 10 → >20 (lcpCount > 20)
- slow_form_fill: "at least 5" → "more than 5" (submissionCount > 5)
- high_api_errors: sessionCount > 10 → > 20
- low_hourly_pageviews: removed incorrect "no z-score" claim; alert requires
  z-score < -2 AND >20% drop
- session_error_count: min page views 10 → >20
- slow_api_response: requestCount > 10 → > 20
- high_external_redirect_error_rate: sessionCount >= 10 → >= 20

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@subodh2711 subodh2711 merged commit 1e81940 into dev Apr 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants