Skip to content

feat(redis_admin): parse repo_id/commit_sha aliases + batch-resolve ComputeComparison repoid/commitid#916

Open
thomasrockhu-codecov wants to merge 3 commits intoredis-admin/unacked-queue-drilldownfrom
redis-admin/parse-breadcrumb-and-comparison
Open

feat(redis_admin): parse repo_id/commit_sha aliases + batch-resolve ComputeComparison repoid/commitid#916
thomasrockhu-codecov wants to merge 3 commits intoredis-admin/unacked-queue-drilldownfrom
redis-admin/parse-breadcrumb-and-comparison

Conversation

@thomasrockhu-codecov
Copy link
Copy Markdown
Contributor

@thomasrockhu-codecov thomasrockhu-codecov commented May 5, 2026

Summary

The Redis-admin celery_broker changelist already extracts (repoid, commitid) from envelope kwargs for most worker tasks, but two task families were silently invisible to the columns and the ?repoid= / ?commitid= filters:

  • UploadBreadcrumbTask (and transplant_report / process_flakes / detect_flakes) carry their repo + commit as repo_id / commit_sha. Sample payload from prod:

    {
      "commit_sha": "af73fb73310312d4d2633449e50e6e141191c28c",
      "repo_id": 18802842,
      "breadcrumb_data": { "...": "..." },
      "upload_ids": [],
      "sentry_trace_id": "..."
    }
  • ComputeComparisonTask carries only comparison_id; the (repoid, commitid) lives in the compare_commitcomparison row that id points at.

Changes

  • families.py: parse_celery_envelope accepts repo_id / commit_sha as aliases for repoid / commitid (canonical wins when both are present). comparison_id was already added to CeleryEnvelopeMeta by feat(redis_admin): unacked queue per-message drill-down + frequency chart + clear-by-filter #911's c0abd58.
  • queryset.py: New _resolve_comparison_repo_commits batch resolver. CeleryBrokerQueueQuerySet._build_row stashes comparison_id; _materialise calls _hydrate_comparison_rows once per page before the request-cache snapshot so a downstream _clone() (e.g. ChangeList's filter clone) reads already-resolved values and the repoid / commitid pushdown narrows correctly. _stream_frequency_aggregate uses a (task, comparison_id) side-counter that _merge_comparison_counter_into_main folds into the main (task, repoid, commitid) counter via one batched lookup so the chart matches the changelist. Resolver failures collapse to (task, None, None) so counts aren't dropped.
  • tests/test_celery_broker_queue.py: 10 new tests covering parser aliases (canonical-wins precedence, string-int comparison_id), one ORM call per page, resolver skipped when no comparison_ids, fallback when the resolver returns {}, and chart bucket resolution.

The unacked-queue counterpart of this work (analogous helpers + tests) intentionally lives in #911 alongside the unacked drilldown surface, so this PR is scoped strictly to celery_broker.

Stacked on

Test plan


Note

Medium Risk
Medium risk because it introduces new Django ORM reads during admin materialization/chart aggregation and changes how celery envelope fields are interpreted, which could affect filtering/performance if misbehaving.

Overview
The celery-broker redis-admin now recognizes repo_id/commit_sha kwargs as aliases for repoid/commitid, ensuring those task messages populate the structured columns and support existing filters.

For ComputeComparison messages that only include comparison_id, the queryset now batch-resolves (repoid, commitid) from CommitComparison during materialization and frequency aggregation (with safe fallbacks on import/DB errors), and the test suite adds coverage for alias precedence, hydration behavior, and single-query batching.

Reviewed by Cursor Bugbot for commit 2d1c900. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7ec78c8. Configure here.

Comment thread apps/codecov-api/redis_admin/queryset.py
@sentry
Copy link
Copy Markdown
Contributor

sentry Bot commented May 5, 2026

Codecov Report

❌ Patch coverage is 96.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.60%. Comparing base (763ca71) to head (2d1c900).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/codecov-api/redis_admin/queryset.py 96.42% 2 Missing ⚠️
Additional details and impacted files
@@                           Coverage Diff                           @@
##           redis-admin/unacked-queue-drilldown     #916      +/-   ##
=======================================================================
- Coverage                                91.60%   91.60%   -0.01%     
=======================================================================
  Files                                     1316     1316              
  Lines                                    51774    51830      +56     
  Branches                                  1625     1625              
=======================================================================
+ Hits                                     47428    47479      +51     
- Misses                                    4040     4045       +5     
  Partials                                   306      306              
Flag Coverage Δ
apiunit 94.06% <96.66%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-notifications
Copy link
Copy Markdown

codecov-notifications Bot commented May 5, 2026

Codecov Report

❌ Patch coverage is 96.66667% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/codecov-api/redis_admin/queryset.py 96.42% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ery envelopes

`UploadBreadcrumbTask` and a few sibling handlers (`transplant_report`,
`process_flakes`, `detect_flakes`) carry their repo + commit as
`repo_id` / `commit_sha` instead of the codecov-historic
`repoid` / `commitid`. The changelist columns and `?repoid=` /
`?commitid=` filters silently ignored those messages because
`parse_celery_envelope` only checked the canonical names. Accept
both shapes; canonical wins when both are present so a task
that sets both stays consistent with every other call site.

Sample envelope (UploadBreadcrumbTask):

    {
      "commit_sha": "af73fb73310312d4d2633449e50e6e141191c28c",
      "repo_id": 18802842,
      "breadcrumb_data": { "...": "..." },
      ...
    }
… celery_broker queryset

\`ComputeComparisonTask\` envelopes only carry \`comparison_id\` in
kwargs, so the changelist columns (and the \`?repoid=\` /
\`?commitid=\` pushdown) had nothing to render or filter on. Stash
\`comparison_id\` on the materialised row, then run a single batched
\`CommitComparison.objects.filter(id__in=...)\` lookup at materialise
time so every row in the page exposes its resolved
\`(repoid, commitid)\` before \`_matches_filters\` runs.

Same pattern in \`_stream_frequency_aggregate\`: messages whose
envelope only carries \`comparison_id\` accumulate into a side
\`(task, comparison_id)\` Counter, then merge into the main
\`(task, repoid, commitid)\` counter via one lookup so the chart
shows resolved buckets instead of collapsing every message into
the all-None row. Resolution failures (missing app, DB hiccup,
deleted comparison_id) collapse into \`(task, None, None)\` so
counts aren't dropped.
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the redis-admin/parse-breadcrumb-and-comparison branch from 7ec78c8 to ed75b64 Compare May 6, 2026 16:21
@thomasrockhu-codecov thomasrockhu-codecov changed the title feat(redis_admin): batch-resolve ComputeComparison + parse UploadBreadcrumb repoid/commitid in celery_broker admin feat(redis_admin): parse repo_id/commit_sha aliases + batch-resolve ComputeComparison repoid/commitid May 6, 2026
…on hydration

Pins the celery_broker contract added in this stack:

* Parser: \`repo_id\` / \`commit_sha\` aliases work on UploadBreadcrumb-
  shaped envelopes; canonical \`repoid\` / \`commitid\` win when both
  are present; \`comparison_id\` extraction handles int + string-int.
* \`_materialise\` calls the resolver once per page, applies it
  before \`_matches_filters\` (so a \`?repoid=\` filter narrows
  ComputeComparisonTask rows correctly), skips the resolver
  entirely when no \`comparison_id\`s are present, and falls back
  cleanly when the resolver returns \`{}\`.
* \`_stream_frequency_aggregate\` resolves comparison buckets into
  the main \`(task, repoid, commitid)\` counter so the chart matches
  the changelist.
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the redis-admin/parse-breadcrumb-and-comparison branch from ed75b64 to 2d1c900 Compare May 6, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant