feat(redis_admin): parse repo_id/commit_sha aliases + batch-resolve ComputeComparison repoid/commitid#916
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7ec78c8. Configure here.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## redis-admin/unacked-queue-drilldown #916 +/- ##
=======================================================================
- Coverage 91.60% 91.60% -0.01%
=======================================================================
Files 1316 1316
Lines 51774 51830 +56
Branches 1625 1625
=======================================================================
+ Hits 47428 47479 +51
- Misses 4040 4045 +5
Partials 306 306
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…ery envelopes
`UploadBreadcrumbTask` and a few sibling handlers (`transplant_report`,
`process_flakes`, `detect_flakes`) carry their repo + commit as
`repo_id` / `commit_sha` instead of the codecov-historic
`repoid` / `commitid`. The changelist columns and `?repoid=` /
`?commitid=` filters silently ignored those messages because
`parse_celery_envelope` only checked the canonical names. Accept
both shapes; canonical wins when both are present so a task
that sets both stays consistent with every other call site.
Sample envelope (UploadBreadcrumbTask):
{
"commit_sha": "af73fb73310312d4d2633449e50e6e141191c28c",
"repo_id": 18802842,
"breadcrumb_data": { "...": "..." },
...
}
… celery_broker queryset \`ComputeComparisonTask\` envelopes only carry \`comparison_id\` in kwargs, so the changelist columns (and the \`?repoid=\` / \`?commitid=\` pushdown) had nothing to render or filter on. Stash \`comparison_id\` on the materialised row, then run a single batched \`CommitComparison.objects.filter(id__in=...)\` lookup at materialise time so every row in the page exposes its resolved \`(repoid, commitid)\` before \`_matches_filters\` runs. Same pattern in \`_stream_frequency_aggregate\`: messages whose envelope only carries \`comparison_id\` accumulate into a side \`(task, comparison_id)\` Counter, then merge into the main \`(task, repoid, commitid)\` counter via one lookup so the chart shows resolved buckets instead of collapsing every message into the all-None row. Resolution failures (missing app, DB hiccup, deleted comparison_id) collapse into \`(task, None, None)\` so counts aren't dropped.
7ec78c8 to
ed75b64
Compare
…on hydration
Pins the celery_broker contract added in this stack:
* Parser: \`repo_id\` / \`commit_sha\` aliases work on UploadBreadcrumb-
shaped envelopes; canonical \`repoid\` / \`commitid\` win when both
are present; \`comparison_id\` extraction handles int + string-int.
* \`_materialise\` calls the resolver once per page, applies it
before \`_matches_filters\` (so a \`?repoid=\` filter narrows
ComputeComparisonTask rows correctly), skips the resolver
entirely when no \`comparison_id\`s are present, and falls back
cleanly when the resolver returns \`{}\`.
* \`_stream_frequency_aggregate\` resolves comparison buckets into
the main \`(task, repoid, commitid)\` counter so the chart matches
the changelist.
ed75b64 to
2d1c900
Compare

Summary
The Redis-admin celery_broker changelist already extracts
(repoid, commitid)from envelope kwargs for most worker tasks, but two task families were silently invisible to the columns and the?repoid=/?commitid=filters:UploadBreadcrumbTask(andtransplant_report/process_flakes/detect_flakes) carry their repo + commit asrepo_id/commit_sha. Sample payload from prod:{ "commit_sha": "af73fb73310312d4d2633449e50e6e141191c28c", "repo_id": 18802842, "breadcrumb_data": { "...": "..." }, "upload_ids": [], "sentry_trace_id": "..." }ComputeComparisonTaskcarries onlycomparison_id; the(repoid, commitid)lives in thecompare_commitcomparisonrow that id points at.Changes
families.py:parse_celery_envelopeacceptsrepo_id/commit_shaas aliases forrepoid/commitid(canonical wins when both are present).comparison_idwas already added toCeleryEnvelopeMetaby feat(redis_admin): unacked queue per-message drill-down + frequency chart + clear-by-filter #911'sc0abd58.queryset.py: New_resolve_comparison_repo_commitsbatch resolver.CeleryBrokerQueueQuerySet._build_rowstashescomparison_id;_materialisecalls_hydrate_comparison_rowsonce per page before the request-cache snapshot so a downstream_clone()(e.g. ChangeList's filter clone) reads already-resolved values and therepoid/commitidpushdown narrows correctly._stream_frequency_aggregateuses a(task, comparison_id)side-counter that_merge_comparison_counter_into_mainfolds into the main(task, repoid, commitid)counter via one batched lookup so the chart matches the changelist. Resolver failures collapse to(task, None, None)so counts aren't dropped.tests/test_celery_broker_queue.py: 10 new tests covering parser aliases (canonical-wins precedence, string-int comparison_id), one ORM call per page, resolver skipped when nocomparison_ids, fallback when the resolver returns{}, and chart bucket resolution.The unacked-queue counterpart of this work (analogous helpers + tests) intentionally lives in #911 alongside the unacked drilldown surface, so this PR is scoped strictly to celery_broker.
Stacked on
redis-admin/unacked-queue-drilldown) — that branch added thecomparison_idfield onCeleryEnvelopeMeta. Once feat(redis_admin): unacked queue per-message drill-down + frequency chart + clear-by-filter #911 lands, this PR rebases naturally onto main.Test plan
Note
Medium Risk
Medium risk because it introduces new Django ORM reads during admin materialization/chart aggregation and changes how celery envelope fields are interpreted, which could affect filtering/performance if misbehaving.
Overview
The celery-broker redis-admin now recognizes
repo_id/commit_shakwargs as aliases forrepoid/commitid, ensuring those task messages populate the structured columns and support existing filters.For
ComputeComparisonmessages that only includecomparison_id, the queryset now batch-resolves(repoid, commitid)fromCommitComparisonduring materialization and frequency aggregation (with safe fallbacks on import/DB errors), and the test suite adds coverage for alias precedence, hydration behavior, and single-query batching.Reviewed by Cursor Bugbot for commit 2d1c900. Bugbot is set up for automated code reviews on this repo. Configure here.