[Grafana][PyTorch DevX] Fix viable/strict lag panel

[jathu]

Why the previous panel was wrong

The previous panel never actually compared viable/strict to main. It looked at push events on viable/strict and computed the gap between the push event time and the oldest commit it included. Specifically "how long the oldest commit included in each push had been sitting around before the push event fired". It has no relationship to how far behind main viable/strict is.

What the new query does

The new query measures the intended metric: each time viable/strict is updated, how many minutes behind main is its new head commit.

How it works:

1. viable CTE: every push event to refs/heads/viable/strict with its head commit's timestamp.
2. main CTE: every push event to refs/heads/main with its head commit's timestamp.
3. ASOF LEFT JOIN main m ON v.pushed_ts >= m.pushed_ts pairs each viable/strict push with the most recent main push at-or-before it — i.e., the state of main at the moment viable/strict was bumped.
4. lag_minutes = (m.commit_ts − v.commit_ts) / 60 — minutes between the two head commits' authoring times.

Test Plan: dashboard

Pushed to my test folder: https://pytorchci.grafana.net/dashboards/f/flrhzs

$ cd grafana
$ mise run push --folder flrhzs

Test Plan: sanity check against pytorch repo

Cross-checked against pytorch/pytorch for the last 12 months (3,489 rows, 5,088 distinct SHAs):

• 5,087/5,088 SHAs present with head_commit.timestamp matching git's %ct exactly. The missing one was a force-pushed-and-reverted bad commit — real anomaly, not a query bug.
• 30 sampled rows: v_commit is a git ancestor of the matched m_commit in all 30 (ASOF picks the right row).
• 10 sampled rows spanning min/median/max lag: dashboard lag_minutes matches (repo_m_ct − repo_v_ct)/60 to the second.

[georgehong]

Are the changes currently reflected in the cited test dashboards and would it be possible to see a before and after of the charts?

I'm wondering if it's reasonable to see a sawtooth pattern here, and I also remember we had a somewhat prolonged viable/strict divergence on 3/20 for more than 4 days.

[huydhn]

@claude check if this is a similar query is used on test-infra https://github.com/pytorch/test-infra/blob/main/torchci/clickhouse_queries/strict_lag_sec/query.sql

[huydhn]

Oh darn, we don't have claude bot here yet I think