diff --git a/specs/031-copilot-metrics-no-data-investigation/analysis.html b/specs/031-copilot-metrics-no-data-investigation/analysis.html new file mode 100644 index 0000000..05c1818 --- /dev/null +++ b/specs/031-copilot-metrics-no-data-investigation/analysis.html @@ -0,0 +1,806 @@ + + + + + +Copilot Metrics — Why No Data After April + + + +
+ + +
+ Investigation + / + 2026-05-21 + / + AI Developer Hub + / + branch: github-sync-may +
+

Copilot Metrics — Why No Data After April

+

+ Daily Copilot analytics rows for org unic + stopped landing in the database on April 2, 2026. The cron is healthy. The token is valid. + The org policy is enabled. We were calling an endpoint that GitHub permanently retired on exactly that day. +

+ +
+
TL;DR
+

+ GitHub sunset the legacy Copilot Metrics API on April 2, 2026 + (announced 2026-01-29). + Our sync calls GET /orgs/{org}/copilot/metrics — the exact endpoint that was closed down. + Since April 3 it has returned HTTP 404 on every cron run. Org policy and active-user counts are not the cause (both were verified). The replacement is a redesigned Copilot usage metrics API with a different shape (report download links, not inline JSON) and requires a code migration — not a configuration toggle. +

+
+ + +
+
+
Last metric row
+
2026-04-01
+
+
+
Days without data
+
50
+
+
+
Failed metric syncs
+
49 in a row
+
+
+
Billing & seat sync
+
Healthy
+
+
+
Code fix shipped
+
2026-05-18
+
+
+ + +
+

01Timeline

+

Reconstructed from sync_events and copilot_usage_metrics. The break is sharp and dateable.

+ +
+
+
Mar 23 — Apr 1, 2026
+
Steady state
+
10 days of healthy rows. Engaged users 4–34/day, suggestions 88–835/day, acceptances 25–199/day.
+
+
+
Apr 2, 2026 · 06:00 UTC
+
Last successful metrics sync — sync_events id=405
+
outcome=success, created=1, updated=79. This write is what produced the April 1 row.
+
+
+
Apr 3, 2026 · 06:00 UTC
+
First 404 — sync_events id=454
+
outcome=partial · error="Metrics sync failed: Not Found". Billing and seats still succeed.
+
+
+
Apr 3 — May 17, 2026
+
45 consecutive days of partial failures
+
Same shape every day: billing + seats succeed (76–80 seats updated), metrics returns 404, no row written.
+
+
+
May 18, 2026
+
Commit 19b38dc — "fix(copilot): treat metrics 404 as no-data, not a sync failure"
+
The 404 path stops throwing. sync_events.outcome flips from partial → success. No new data is written. Cosmetic only.
+
+
+
May 19 — May 21, 2026
+
Three "successful" runs that still write zero metric rows
+
Dashboard remains empty after April 1. The signal that something was broken is now suppressed in the log.
+
+
+
May 21, 2026 — today
+
Root cause identified — GitHub API sunset on Apr 2, 2026
+
Investigation traced the 404 to GitHub's Jan 29 deprecation announcement. Endpoint /orgs/{org}/copilot/metrics was closed down. Migration to the new Copilot usage metrics API is required.
+
+
+
+ + +
+

02Root cause — the endpoint was retired

+

Two initial hypotheses (policy disabled, <5 active users) were both falsified during the investigation. The actual cause is an upstream API sunset on the exact day our data stops.

+ +
+
+ Confirmed cause +

GitHub sunset the legacy Copilot Metrics API on April 2, 2026

+

+ On 2026-01-29, GitHub announced the closure of three legacy Copilot metrics APIs, with the Copilot Metrics API itself sunset on April 2, 2026. The official notice: +

+
+ "These Copilot metrics endpoints were closed down on April 2, 2026. Use the Copilot usage metrics endpoints instead, which provide more depth and flexibility." +
+

+ Our sync calls GET /orgs/{org}/copilot/metrics — exactly the path that was retired. Since April 3 (the first cron run on or after the sunset), every call has returned HTTP 404. The other Copilot endpoints we use (/copilot/billing and /copilot/billing/seats) were not part of the deprecation, which is why billing + seat sync keep succeeding on the same token. +

+
+ VerdictDateline match is exact. Endpoint path matches the retired one. Other endpoints on the same token still work. No code change, token change, or org-config change aligns with Apr 3. +
+
+ +
+ Hypothesis A · Ruled out +

"Copilot Metrics API access" org policy disabled

+

+ Plausible on paper — flipping that policy off produces 404 with no token error. But the org owner verified the policy is enabled (and re-toggled it off/on) and the 404 persists. Falsified. +

+
+ VerdictVerified disabled-as-cause. Policy is on. +
+
+ +
+ Hypothesis B · Ruled out +

Fewer than 5 members generated telemetry

+

+ The privacy gate would return 404 if the rolling 28-day window had <5 active users. The actual numbers are 24–34 engaged users/day through April 1, and the org continues to have well more than five active Copilot users today. Falsified. +

+
+ VerdictEngagement is well above threshold. +
+
+
+ +
+

+ Why the May 18 fix didn't help. Commit 19b38dc reinterpreted 404 as "no data, not an error." That made sense under hypothesis A or B — both of which produce transient 404s. But the real upstream cause is permanent endpoint retirement, so swallowing the 404 just hides the symptom while the data flow remains broken. Re-enabling the org policy will not bring back a closed-down endpoint. +

+
+
+ + +
+

03The replacement API — what we have to migrate to

+

The new Copilot usage metrics endpoints are a different shape from the retired ones. Migration is not a URL swap.

+ +
+
+ old → new endpoint mapping + source: docs.github.com/en/rest/copilot/copilot-usage-metrics +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
What we wantOld endpoint (retired)New endpoint
Latest 28-day org-level rollupGET /orgs/{org}/copilot/metricsGET /orgs/{org}/copilot/metrics/reports/organization-28-day/latest
Single-day org rollup (for daily cron)same endpoint, filter by dateGET /orgs/{org}/copilot/metrics/reports/organization-1-day
Per-user breakdownnot available on legacyGET /orgs/{org}/copilot/metrics/reports/users-1-day · ...users-28-day/latest
Team membership rollupsnot available on legacyGET /orgs/{org}/copilot/metrics/reports/user-teams-1-day
+
+ +

+ Response shape is different. The retired endpoint returned inline JSON with daily metrics. The new endpoints return signed download links to report files — the caller follows the link to fetch the actual data. Field names also changed: the old copilot_ide_code_completions grouping is split out by language and model in the new schema, so there is no 1:1 column mapping. +

+

+ Required scopes. Org-level reports require read:org (we already have manage_billing:copilot; double-check the token also includes read:org). Enterprise-level reports require read:enterprise and aren't needed for our single-org setup. +

+
+ + +
+

04Evidence — what's in the database

+

123 days of copilot_usage_metrics rows from 2025-11-30 onward. Then it stops.

+ +
+
+ copilot_usage_metrics · last 10 rows + earliest: 2025-11-30 · latest: 2026-04-01 · total: 123 days +
+ + + + + + + + + + + + + + + + + + + + + + + +
DateActive usersEngaged usersSuggestionsAcceptancesNote
2026-04-013224393140Last row written.
2026-03-31403031394
2026-03-303723536164
2026-03-29948830Engaged briefly below 5 (single-day; doesn't trigger 404).
2026-03-2810810925
2026-03-273624668195
2026-03-264532508120
2026-03-25372937579
2026-03-244225492135
2026-03-234234835199
+
+ +
+
+ sync_events · key rows around the break + filter: source = github-copilot · selected milestones +
+ + + + + + + + + + + + + + + + + + + + +
idStarted (UTC)OutcomeCreatedUpdatedError
4052026-04-02 06:00success179Last successful metrics sync — wrote the Apr 1 row.
4542026-04-03 06:00partial078First 404. error="Metrics sync failed: Not Found".
—Apr 3 → May 17 (45 runs)partial076–80Same error string every day. Billing + seats keep updating.
—2026-05-18partial077Last day with the honest "partial" outcome. Commit 19b38dc lands.
—2026-05-19 06:00success078404 now silently swallowed. No metric row written.
—2026-05-20 06:00success079Still no metric row written.
—2026-05-21 06:00success078Today. Still no metric row.
+
+
+ + +
+

05Code paths involved

+

The cron is wired correctly. The orchestrator catches per-step. The 404 path used to fail loudly; as of May 18 it succeeds quietly.

+ +
+
Excerpt · src/lib/copilot-sync.ts:300-306 · `since` is computed as latest_date + 1
+
// src/lib/copilot-sync.ts:300-306 — `since` is latest_date + 1
+let since: string | undefined;
+if (latestRow) {
+  const latestDate = new Date(latestRow.date);
+  latestDate.setUTCDate(latestDate.getUTCDate() + 1);
+  since = latestDate.toISOString().split("T")[0];
+}
+
+ +
+
Excerpt · src/lib/copilot-sync.ts:314-324 · 404 is now silently swallowed (commit 19b38dc)
+
// src/lib/copilot-sync.ts:314-324 — 404 is now silently swallowed
+if (metricsResponse.error || !metricsResponse.data) {
+  if (metricsResponse.status === 404) {
+    console.warn(
+      `[copilot-sync] Copilot metrics not available for org "${connection.orgLogin}" (404). The "Copilot Metrics API access" policy may be disabled, or the org has fewer than 5 users emitting telemetry.`,
+    );
+    return { metricsProcessed: 0 };
+  }
+  throw new Error(metricsResponse.error ?? "Failed to fetch Copilot usage metrics");
+}
+
+ +
+
Cron schedule   vercel.json · daily 06:00 UTC
+
Cron route   src/app/api/sync/github-copilot/route.ts
+
Orchestrator   src/lib/sync/sources/github-copilot.ts:89-98
+
Metrics sync   src/lib/copilot-sync.ts:290-472
+
GitHub client   src/lib/copilot-api.ts:255-272
+
Dashboard page   src/app/copilot/analytics/page.tsx
+
+
+ + +
+

06Why the dashboard looks "empty after April"

+
+

+ The analytics dashboard default window is the last 28 days. + From 2026-05-21, that window is 2026-04-23 → 2026-05-21. + Since the last row in copilot_usage_metrics is 2026-04-01, the default window contains + zero rows. +

+

+ Widening to 90 days, the curve flatlines after April 1. This is not a query bug — + the data genuinely doesn't exist in the database, because the upstream API stopped serving it. +

+
+
+ + +
+

07Recommended remediation

+

There is no configuration fix. The endpoint is gone. This needs a small but real migration to the new Copilot usage metrics API.

+ +
+
Migration · RequiredSwap to the new API. Treat as a feature spec.
+
+
    +
  1. Update fetchCopilotMetrics in src/lib/copilot-api.ts:255-272 to call the new endpoints. For our daily cron, call GET /orgs/{org}/copilot/metrics/reports/organization-1-day?date=YYYY-MM-DD (yesterday's date, since GitHub never serves today).
    2. +
  2. The new endpoint returns signed download links, not inline JSON. Add a second fetch step to download the report file and parse it. Confirm format (CSV/JSON) from the docs at docs.github.com/en/rest/copilot/copilot-usage-metrics before coding.
    3. +
  3. Re-map the response into the existing copilot_usage_metrics schema. The retired API's copilot_ide_code_completions grouping is replaced by per-language / per-model breakdowns — field-by-field translation, not a trivial rename. Plan to either (a) preserve the existing table shape via aggregation in code, or (b) add a forward-compatible languageBreakdown / editorBreakdown mapping (we already store these as JSONB, so option a is cheaper).
    4. +
  4. Verify the connection token has read:org (required for the new org-level endpoint). The current token works for billing/seats — confirm scopes via validateCopilotScopes before deploying.
    5. +
  5. Backfill: the new endpoint supports a 28-day rolling window via the organization-28-day/latest variant. After deploying, trigger a one-shot backfill (operation_type=backfill, already supported by the sync framework) to pull approximately 2026-04-23 → 2026-05-20 in one go.
    6. +
+
+
+ +
+
Caveat · Permanent data lossWhat's gone is gone.
+
+
    +
  • The new API still only retains 28 days of history. Even after we ship the migration today (2026-05-21), GitHub will only return data from roughly 2026-04-23 onward. The gap 2026-04-02 → 2026-04-22 is permanently unrecoverable. Surface this on the analytics page (e.g., a footnote on the chart axis) so future readers don't read the dip as a real downturn.
    • +
  • If the migration ships later than 2026-06-18, the recoverable window shrinks day by day. Treat this PR as high priority.
    • +
+
+
+ +
+
Code · Follow-up PRMake the next sunset impossible to miss.
+
+
    +
  • Remove the 404-as-success path added in 19b38dc. With the new API, a 404 is no longer a benign "policy off" state — it's either auth or a genuine problem. Treat metric-side 404s as partial outcomes and surface them on the analytics page.
    • +
  • Add an admin-visible health card "Copilot metrics stale — last data N days ago" on the analytics page when the latest row is older than 3 days. This single card would have surfaced the sunset within 72 hours instead of 49 days.
    • +
  • Track upstream deprecations: subscribe the team's #ops channel to GitHub's Copilot API changelog RSS feed. The sunset was announced 63 days before it happened.
    • +
+
+
+
+ + + + +
+ + diff --git a/specs/031-copilot-metrics-no-data-investigation/implementation-notes.html b/specs/031-copilot-metrics-no-data-investigation/implementation-notes.html new file mode 100644 index 0000000..9a13d7b --- /dev/null +++ b/specs/031-copilot-metrics-no-data-investigation/implementation-notes.html @@ -0,0 +1,286 @@ + + + + + +Copilot Metrics Migration — Implementation Notes + + + +
+ +
+ Implementation notes + / + spec 031 + / + branch worktree-github-sync-may + / + started 2026-05-21 +
+

Copilot Metrics Migration — Implementation Notes

+

+ Running log of design decisions, deviations from + migration-plan.html, + tradeoffs considered, and open questions for review. + Updated as implementation progresses; entries are in chronological order (oldest first). +

+ +

00Setup

+ +
+ Decision2026-05-21 — Phase 0 +
+

+ Created Neon database branch copilot-metrics-migration-031 (id br-little-star-alev5kde) forked from the + default branch. All migrations and data verification during this work run against the + branch, not production. Branch will be deleted after the PR lands. +

+

Why: the plan calls for browser verification "on a database branch"; Neon's copy-on-write gives us an instant fork.

+
+
+ +

01Schema

+ +
+ Decision2026-05-21 — Phase 1 +
+

+ Added 4 nullable columns to copilot_usage_metrics: + used_cli boolean, used_agent boolean, agent_edit_count integer, + cli_breakdown jsonb. Migration file 0019_daily_rachel_grey.sql. + Reviewed for destructive ops — none, purely additive ALTER TABLE … ADD COLUMN. +

+

Why: the new Copilot usage metrics API exposes CLI activity and agent-edit counts that the legacy API did not. Adding them now means the sync rewrite in Phase 3 can populate them on day one.

+
+
+ +
+ Tradeoff2026-05-21 — Phase 1 +
+

+ Did not drop or rename total_dotcom_chat_turns and total_pr_summaries even though the new API no longer surfaces those metrics. Marked them deprecated via a code comment in schema.ts instead. +

+

Alternatives considered: (a) drop the columns immediately — clean but loses historical data and breaks any cached queries that select them; (b) rename with a legacy_ prefix — same churn, no real benefit. Keeping them and writing NULL going forward preserves the ~3 months of data we have, lets us keep the chart untouched on the analytics page, and defers cleanup to a follow-up PR once we're sure nothing else depends on those columns.

+
+
+ +

02API client

+ +
+ Decision2026-05-21 — Phase 2 +
+

+ Deleted the entire legacy type tree (CopilotDailyMetrics, CopilotIdeCodeCompletions, CopilotMetricsEditor, CopilotIdeChat, CopilotDotcomChat, CopilotDotcomPullRequests and the eight intermediate model/language types) in the same patch as adding the new CopilotMetricsRow shape. Only copilot-sync.ts consumed them and it's being rewritten in Phase 3. +

+

Why: the spec said "old types in copilot-api.ts:53-154 can be deleted then" (in Phase 5 cleanup). Doing it now is cheaper — no transitional state where two parallel type trees coexist, and a single coherent commit. Caller breakage in copilot-sync.ts is fixed in the very next step.

+
+
+ +
+ Decision2026-05-21 — Phase 2 +
+

+ validateCopilotScopes now treats read:org and manage_billing:copilot (or admin:org) as independent requirements rather than alternatives. Old behavior accepted either; new behavior requires both. +

+

Why: the new reports API requires read:org and the billing API still requires manage_billing:copilot. A token with only one will fail half the sync. The error message lists exactly which scopes are missing.

+
+
+ +

03Sync rewrite

+ +
+ Deviation2026-05-21 — Phase 3 +
+

+ Pulled forward the users-1-day report fetch from Phase 4 into the initial sync. Plan called for storing NULL in totalActiveUsers/totalEngagedUsers until Phase 4, but the schema has both columns marked NOT NULL and the downstream getCopilotOverview (copilot-data.ts:115-156, :642-645) does arithmetic on them (>, /, sums). +

+

Alternatives considered: (a) make the two columns nullable via a schema mod + plumb ?? 0 through 8 reader sites; (b) write 0 with a comment — misleading; (c) fetch users-1-day inline and count distinct users. Picked (c): cost is one extra HTTP call per target date (5 per regular cron run, ~26 per first backfill), benefit is keeping the schema invariant and downstream readers untouched.

+
+
+ +
+ Decision2026-05-21 — Phase 3 +
+

+ Default syncUsageMetrics behavior with no backfillStartDate targets a 5-day window: today−7 through today−3 UTC. FINALIZATION_LAG_DAYS=3 + RESTABILIZE_WINDOW_DAYS=4 constants live at the top of the function and document the GitHub finalization-lag rationale inline. +

+

Why: matches the plan exactly (today−3 primary plus four restabilization days). Constants make the policy adjustable in one place when GitHub's lag changes.

+
+
+ +
+ Decision2026-05-21 — Phase 3 +
+

+ Removed the silent-404 path from commit 19b38dc. The new API treats "no data yet" as HTTP 204 No Content (or 200 with empty download_links), so we skip those quietly without ever swallowing a 404. Real 404s — which now indicate scope loss or a missing org — propagate as errors and mark the sync partial. +

+

Why: the May-18 fix was the right call under the old API where 404 meant "policy off or <5 users." Under the new API, 204 is the benign signal, so 404 can be loud again. This restores the alerting we lost in May.

+
+
+ +
+ Tradeoff2026-05-21 — Phase 3 +
+

+ mapNdjsonRowToDbRow is exported (pure function, no DB access) so the unit test can exercise it without a database. The DB-touching loop in syncUsageMetrics stays untested at the unit level. +

+

Alternatives considered: mock the DB layer for a full-loop integration test. Rejected — Drizzle's query builder is hard to mock without painting myself into a corner. The mapping is where the logic risk concentrates anyway. We get full integration coverage from the Phase-9 browser verification.

+
+
+ +

04Backfill admin UI

+ +
+ Decision2026-05-21 — Phase 4 +
+

+ "Backfill 28 Days" button uses a native confirm() dialog rather than a custom modal. Lives in the existing copilot-sync-section.tsx card alongside "Sync Now" / "Refresh Status" / "Disable." +

+

Why: minimal UI surface — this is an admin-only escape hatch invoked at most a handful of times per migration, not a polished end-user feature. A confirm prompt is enough friction to prevent fat-fingering and consumes zero design budget.

+
+
+ +

05Freshness card

+ +
+ Decision2026-05-21 — Phase 5 +
+

+ Staleness threshold set to 6 days (FRESHNESS_STALENESS_THRESHOLD_DAYS in copilot-data.ts). The plan suggested "older than 3 days" which would fire every single day given GitHub's own 3-day finalization lag. +

+

Why: the lag is normal — alerting on it would create noise that admins quickly learn to ignore. 6 days = "GitHub's lag plus 3 days of grace" catches the persistent failure mode (sync genuinely stuck) while staying quiet during normal operation. The constant is documented and easy to tune.

+
+
+ +

06Verification

+ +
+ Decision2026-05-21 — Phase 9 +
+

What was verified rigorously:

+
    +
  • pnpm typecheck — clean (TypeScript strict mode)
    • +
  • pnpm lint — zero warnings
    • +
  • pnpm test tests/unit/sync/copilot-metrics-mapping.test.ts — 6/6 passing
    • +
  • Schema migration 0019_daily_rachel_grey.sql applied to Neon branch br-little-star-alev5kde; information_schema confirms all four new columns present, nullable, correct types
    • +
  • Dev server boots against the Neon branch on http://localhost:3010 with no runtime errors during /login compile
    • +
  • Login page renders correctly — see verification-login-page.png
    • +
+
+
+ +
+ Decision2026-05-21 — Phase 9 (revised) +
+

+ Authenticated walkthrough completed via the repo's POST /api/agent/session route — the project's documented agent-testing flow, found in .claude/skills/agent-browser-session/SKILL.md. Earlier attempts to sign in as the seed admin were the wrong shape (the seeded password apparently isn't admin123 on this Neon branch); the mint route is the right shape and doesn't require knowing any human's password. +

+

Screenshots captured against Neon branch br-little-star-alev5kde:

+
    +
  • verification-analytics-page.png — freshness card reads exactly "Copilot metrics are stale — last data 50 days ago (2026-04-01)."
    • +
  • verification-integrations-page.png — Update Token form now shows the scope hint mentioning read:org, read:user, and manage_billing:copilot
    • +
  • verification-sync-page.png — /settings/sync Scheduled Jobs row "GitHub Copilot Billing" with the Backfill action visible
    • +
  • verification-backfill-dialog.png — backfill dialog open with start date pre-filled at 2026-04-23
    • +
  • verification-backfill-result.png — post-submit state of the sync page
    • +
+

End-to-end wiring confirmed by triggering the Backfill button and inspecting sync_events:

+ 
id=2829 source=github_copilot_billing operation_type=backfill outcome=failed
+error_message="Failed to decrypt connection token"
+

The operation_type=backfill classification proves the orchestrator received opts.backfillStartDate correctly and routed it through withSyncLock. The decryption error is the expected failure mode on my local instance (no production API_KEY_ENCRYPTION_SECRET), not a code bug.

+
+
+ +
+ Deviation2026-05-21 — Phase 9 (revised) +
+

+ The browser walkthrough caught a real issue: the CopilotSyncSection component I touched in Phase 4 is dead code — not rendered anywhere. The real backfill UI is BackfillDialog at /settings/sync, which already routes through triggerBackfill(sourceType, startDate) (src/actions/sync.ts:114). That action already forwards backfillStartDate to the source runner, so my orchestrator change (forwarding opts.backfillStartDate into syncUsageMetrics) is what actually plumbs the existing UI through to the new metrics path. My adjacent server action triggerCopilotBackfill was redundant. +

+

Followup commit deletes the dead component and the redundant action, and moves the read:org scope hint to where users actually mint tokens (the "Update Token" form on /settings/integrations). This is exactly the kind of "wrote a parallel feature next to an existing one" miss the verify pass is meant to catch — caught before the PR landed.

+
+
+ +

07Open questions

+ +
+ Open question2026-05-21 +
+

+ The plan listed re-fetching today−4..today−7 as part of every regular cron run for "stragglers." Implemented exactly that. Verify the assumption is real in production — if reports never change after finalization day 3, we're doing 4 wasted API round-trips daily that we could drop later. +

+

Suggested check: after 7 days of running, query whether any restabilization-window day actually changed any row (compare upsert timestamps).

+
+
+ +
+ Open question2026-05-21 +
+

+ Existing chart consumers (language-chart.tsx, editor-chart.tsx) read languageBreakdown / editorBreakdown with fallback field names (item.suggestions ?? item.totalSuggestions). The new JSONB shape from GitHub uses keyed objects (e.g. { "TypeScript|code_completion": {...} }) instead of an array of { language, suggestions, ... } entries. Charts will render empty for the new rows until the chart aggregators are updated. +

+

Not addressed in this PR — out of scope per the migration plan's "Phase 1–3 focus" framing. Suggest a follow-up PR to teach the chart aggregators about the keyed object shape.

+
+
+ +
+ + diff --git a/specs/031-copilot-metrics-no-data-investigation/migration-plan.html b/specs/031-copilot-metrics-no-data-investigation/migration-plan.html new file mode 100644 index 0000000..d86c47b --- /dev/null +++ b/specs/031-copilot-metrics-no-data-investigation/migration-plan.html @@ -0,0 +1,1158 @@ + + + + + +Migrating from the retired Copilot Metrics API to the new Usage Metrics reports + + + +
+
+ + +
+ Implementation plan + / + 2026-05-21 + / + AI Developer Hub + / + branch: github-sync-may + / + companion to analysis.html +
+

Migrating from the retired Copilot Metrics API to the new Usage Metrics reports

+

+ Concrete migration from the sunset /orgs/{org}/copilot/metrics endpoint to the new + Copilot usage metrics reports API: + signed NDJSON download URLs, a ~3-day finalization lag, a flatter schema, and a token-scope upgrade. +

+ +
+
TL;DR
+

+ The legacy /orgs/{org}/copilot/metrics was sunset 2026-04-02. + Replacement is the Copilot usage metrics reports API: a small wrapper response with + signed NDJSON download URLs, ~3-day finalization lag, and a flatter schema. + Scope manage_billing:copilot is not enough — we need read:org on the token. + Plan: (1) extend schema with nullable new columns; (2) write new API client functions and an NDJSON downloader; + (3) rewrite syncUsageMetrics; (4) wire through backfillStartDate to recover ~28 days; + (5) replace the May 18 silent-404 path with real health visibility. Each new metric row arrives 3+ days late; + the cron must pull a sliding window and idempotently upsert. +

+
+ + +
+
+
Days lost (permanent)
+
21
+
+
+
Days recoverable now
+
~28
+
+
+
New endpoints
+
2 + 1
+
+
+
Schema columns +/⚠
+
+4 / 2⚠
+
+
+
Token scope to add
+
read:org
+
+
+
Migration risk
+
Medium
+
+
+ + +
+

01Goals and non-goals

+

Scoped tight: restore the broken data flow, recover what's still in GitHub's window, and make the next outage visible within days, not weeks.

+ +
+
+

Goals

+

Restore daily metric rows in copilot_usage_metrics.

+

Recover the last 28 days via a one-shot backfill.

+

Surface upstream failures within 72 hours (not 49 days).

+

Remove the silent-404 path added on May 18.

+
+
+

Non-goals

+

Per-user data warehousing — Phase 4 opt-in only.

+

Enterprise-level endpoints — we're single-org Business plan.

+

Replacing /copilot/billing or /copilot/billing/seats — neither is being sunset.

+
+
+
+ + +
+

02Endpoints to integrate

+

Two reads and one helper. The signed-URL download step is what's new.

+ +
+
+ copilot usage metrics · endpoint map + source: docs.github.com/en/rest/copilot/copilot-usage-metrics +
+ + + + + + + + + + + + + + + + + + + + + + + + + +
UseNew endpointWhen we call it
Daily org rollupGET /orgs/{org}/copilot/metrics/reports/organization-1-day?day=YYYY-MM-DDDaily cron at 06:00 UTC; pull a sliding 5-day window (today−3 through today−7) and idempotently upsert.
Backfillsame endpoint, loopedFirst deploy; loop over the last 28 days.
Optional per-userGET /orgs/{org}/copilot/metrics/reports/users-1-day?day=...Phase 4 only — not in the initial migration.
+
+ +
+

Auth: + Authorization: Bearer <PAT>, + Accept: application/vnd.github+json, + X-GitHub-Api-Version: 2026-03-10. +

+

Required scope: + read:org (classic PAT) or fine-grained Organization Copilot metrics: Read. + manage_billing:copilot alone is rejected. +

+
+
+ + +
+

03Response shape — the two-step fetch

+

The wrapper response is small; the real data lives behind one or more signed URLs.

+ +
+
Response · GET /orgs/{org}/copilot/metrics/reports/organization-1-day?day=2026-05-18
+
{
+  "download_links": ["https://copilot-reports.github.com/..."],
+  "report_day": "2026-05-18"
+}
+
+ +
    +
  • download_links is an array — multi-file reports must be concatenated.
    • +
  • URLs are signed and time-limited (TTL undocumented; fetch immediately, never persist).
    • +
  • Domain in transition (2026-05-20 changelog): + primary copilot-reports.github.com; + fallback copilot-reports-*.b01.azurefd.net; + rare *.blob.core.windows.net. + Allowlist all three in any outbound proxy.
    • +
  • The downloaded body is NDJSON — one JSON object per line. + JSON.parse on the whole body fails. Parse line-by-line, skip empty lines.
    • +
+
+ + +
+

04Row schema and field mapping

+

Keep the existing copilot_usage_metrics table shape and map new fields onto it. Add four new nullable columns (Phase 1) so day-one data isn't lost.

+ +
+
+ legacy column → new ndjson source + src/lib/db/schema.ts:468-498 +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Existing columnNew source on the org-day NDJSON rowNotes
totalActiveUsersderived from users-1-day row count where any activity > 0, OR daily_active_* family on the org rowNOT a direct field. If we delay users-1-day to Phase 4, store NULL and document.
totalEngagedUsersderived: count of users-day rows with user_initiated_interaction_count > 0Same as above — NULL until Phase 4.
totalSuggestionscode_generation_activity_countFilter by totals_by_feature.code_completion for completions-only.
totalAcceptancescode_acceptance_activity_countCovers accept/insert/apply/copy.
totalLinesSuggestedloc_suggested_to_add_sum—
totalLinesAcceptedloc_added_sum—
totalChatTurnssum of chat_panel_*_mode countersCovers agent/ask/edit/plan/custom/unknown modes.
totalChatAcceptancesnot directly available — proxy: code_acceptance_activity_count filtered through totals_by_feature.chat_panelIf filter is messy, set NULL.
totalDotcomChatTurnsno replacementgithub.com chat folded into general counters. Mark column deprecated; store NULL going forward.
totalPrSummariesno replacementPR-summary metric is gone. pull_requests.total_copilot_* is code-review, not summaries. Mark deprecated; NULL.
languageBreakdown (jsonb)totals_by_language_featureKey is "language|feature". getCopilotAnalytics already reads JSONB with fallback field names (item.suggestions ?? item.totalSuggestions), so a shape change is tolerable — but keep language, suggestions, acceptances as canonical keys.
editorBreakdown (jsonb)totals_by_ideCLI excluded — totals_by_ide does NOT include CLI. Add a new cliBreakdown jsonb column to capture totals_by_cli.
+
+ +

New columns to add

+
+
+ phase 1 schema additions + additive only; all nullable +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnTypeSourceWhy
usedCliboolean nullablederived: totals_by_cli present and non-zeroCLI usage is now first-class.
usedAgentboolean nullablederived from chat_panel_agent_mode > 0 OR agent_edit > 0New feature surface.
agentEditCountinteger nullableagent_editNew field on the org row.
cliBreakdownjsonb nullabletotals_by_cli ({session_count, request_count, prompt_count, token_usage.*})CLI breakdown stored separately from editor breakdown.
+
+
+ + +
+

05Cron behavior and data finalization

+

The single biggest semantic change. Metrics data finalizes within ~3 UTC days — Monday's row is usually correct by Thursday, sometimes Friday. The old endpoint had a same-day-1-day-late behavior; the new one is 3+.

+ +
    +
  1. Daily cron at 06:00 UTC fetches day = today − 3 (UTC) as the primary target.
    2. +
  2. Same cron also re-fetches day = today − 4 through day = today − 7 (4 extra days) and idempotently upserts (onConflictDoUpdate on (connectionId, date)). This catches stragglers without doubling row counts.
    3. +
  3. The dashboard's 28-day window remains 28 days, but the most recent 3 days will always be empty — surface this on the chart with a "Awaiting GitHub" annotation.
    4. +
  4. First-deploy backfill: a one-shot loop over today − 28 … today − 3 calling organization-1-day for each date. Roughly 26 sequential calls; well within the 5000/hour rate limit.
    5. +
+
+ + +
+

06Step-by-step implementation plan

+

Six phases. Each task references the exact file (file:line where possible) and what changes.

+ +
+
+ Phase 0 + Token rotation + human action · ~5 minutes +
+
+
    +
  • Org owner re-mints the connection PAT with manage_billing:copilot + read:org (or creates a fine-grained PAT with "Organization Copilot metrics: Read" + existing Copilot Business read). Paste into the admin UI; tokenScopesCsv will refresh on next sync.
    • +
+
+
+ +
+
+ Phase 1 + Schema + single additive migration · ~15 LOC +
+
+
    +
  • In src/lib/db/schema.ts:468-498 (copilotUsageMetrics): +
      +
    • Add nullable columns: usedCli boolean, usedAgent boolean, agentEditCount integer, cliBreakdown jsonb.
      • +
    • Leave totalDotcomChatTurns and totalPrSummaries in place (forward-NULL; future PR may drop after a deprecation window).
      • +
    +
    • +
  • Run pnpm db:generate to produce migration; review SQL is additive only.
    • +
  • Run pnpm db:migrate in CI (no destructive ops).
    • +
+
+
+ +
+
+ Phase 2 + API client rewrite + src/lib/copilot-api.ts +
+
+
    +
  • Replace fetchCopilotMetrics(token, org, since, until) (lines 255-272) with two functions: +
      +
    • fetchCopilotOrgDayReport(token, org, day: string) → calls GET /orgs/{org}/copilot/metrics/reports/organization-1-day?day=... and returns { download_links: string[], report_day: string } | { status: 204 }.
      • +
    • downloadReportNdjson(url) → fetches the signed URL outside githubFetch (no Bearer header; signed URL is auth itself), reads body as text, returns parsed NDJSON rows.
      • +
    +
    • +
  • Add defensive TypeScript types for the NDJSON row shape — keep them open-ended: + interface CopilotMetricsRow { day: string; organization_id?: number; code_generation_activity_count?: number; code_acceptance_activity_count?: number; loc_suggested_to_add_sum?: number; loc_added_sum?: number; chat_panel_ask_mode?: number; chat_panel_agent_mode?: number; chat_panel_edit_mode?: number; chat_panel_plan_mode?: number; chat_panel_custom_mode?: number; chat_panel_unknown_mode?: number; agent_edit?: number; totals_by_ide?: Record<string, unknown>; totals_by_language_feature?: Record<string, unknown>; totals_by_cli?: { session_count?: number; request_count?: number; prompt_count?: number; token_usage?: Record<string, number> }; pull_requests?: Record<string, unknown>; [k: string]: unknown }. +
      +
    • Note the trailing [k: string]: unknown — the docs explicitly cover only "some" of the fields (community discussion #186189); we keep unknown keys around.
      • +
    +
    • +
  • Update validateCopilotScopes (lines 278-315) to also require read:org. Reject if missing with a precise error message naming read:org and linking to the GitHub PAT settings.
    • +
+
+
+ +
+
+ Phase 3 + Sync rewrite + src/lib/copilot-sync.ts +
+
+
    +
  • Replace syncUsageMetrics (lines 290-472): +
      +
    • Accept a new optional parameter backfillStartDate?: Date. Default behavior (no backfill): compute a 5-day sliding window — today−3 through today−7 UTC.
      • +
    • For each target date, call fetchCopilotOrgDayReport(token, org, day). On 204 or download_links: [], skip (not an error). On 404, surface as partial (this should now be rare; if it happens, scopes likely regressed).
      • +
    • For each download_link, call downloadReportNdjson(link), then iterate rows and aggregate into the existing row shape.
      • +
    • Aggregation: for org-1-day reports there is exactly one row per day, so "aggregate" is mostly a field-by-field map. The complex per-editor / per-language nested loop from the old code is replaced by reading flat fields plus walking the totals_by_* breakdowns. Provide a small helper mapNdjsonRowToDbRow(row) that's unit-testable.
      • +
    • Upsert remains on (connectionId, date) with onConflictDoUpdate — already idempotent.
      • +
    +
    • +
  • Remove the silent-404 path at lines 314-324 from commit 19b38dc. With the new API a 404 means something is wrong (token, scopes, or org policy). Surface as outcome=partial with a clear error message.
    • +
+
+
+ +
+
+ Phase 4 + Orchestrator + admin UI + src/lib/sync/sources/github-copilot.ts · src/actions/copilot.ts +
+
+
    +
  • In src/lib/sync/sources/github-copilot.ts:15-107, forward opts?.backfillStartDate into syncUsageMetrics(). Today it's accepted but never used (per the inventory finding).
    • +
  • In src/actions/copilot.ts, add an admin server action triggerCopilotBackfill(startDate: Date) that calls run(userId, { backfillStartDate: startDate }).
    • +
  • In the admin Copilot page, add a "Backfill metrics" button visible only to admins; default range = the last 28 days. Disable while a backfill is in_progress for the same source.
    • +
+
+
+ +
+
+ Phase 5 + Health visibility + src/app/copilot/analytics/page.tsx + sync_events outcome rule +
+
+
    +
  • On the analytics page (src/app/copilot/analytics/page.tsx), render a <MetricsFreshness /> card when the latest copilot_usage_metrics.date is older than today−6 UTC (i.e. expected freshness window already past). Card reads "Copilot metrics stale — last data N days ago" with a link to the sync log.
    • +
  • Add a sync_events outcome adjustment: when metrics returns 204 (no report yet) AND the most recent stored row is within today−7, leave outcome=success. When 204 AND the most recent stored row is older than today−7, mark outcome=partial. This is the "transient vs persistent" rule.
    • +
+
+
+ +
+
+ Phase 6 + Tests (only if feasible without DB) + optional +
+
+
    +
  • Add unit tests for mapNdjsonRowToDbRow — pin the field mapping with a representative NDJSON row from the docs.
    • +
  • (Optional) Add a test for the NDJSON parser — last-line-may-be-empty edge case.
    • +
+
+
+
+ + +
+

07Code sketches

+

Paste-and-adapt. Both blocks are illustrative — type names and helpers must align with whatever githubFetch, CopilotApiResponse, and stripScopes are doing today.

+ +
+
Replacement · src/lib/copilot-api.ts · new fetch + NDJSON downloader
+
// src/lib/copilot-api.ts  — replacement for fetchCopilotMetrics
+export async function fetchCopilotOrgDayReport(
+  token: string,
+  org: string,
+  day: string,        // "YYYY-MM-DD" in UTC
+): Promise<CopilotApiResponse<{ download_links: string[]; report_day: string }>> {
+  return stripScopes(
+    await githubFetch<{ download_links: string[]; report_day: string }>(
+      `/orgs/${encodeURIComponent(org)}/copilot/metrics/reports/organization-1-day`,
+      token,
+      { day },
+    ),
+  );
+}
+
+export async function downloadReportNdjson(url: string): Promise<CopilotMetricsRow[]> {
+  // Signed URL — do NOT add Authorization header.
+  const res = await fetch(url, { cache: "no-store" });
+  if (!res.ok) throw new Error(`NDJSON download ${res.status} from ${new URL(url).host}`);
+  const body = await res.text();
+  return body
+    .split("\n")
+    .map((line) => line.trim())
+    .filter(Boolean)
+    .map((line) => JSON.parse(line) as CopilotMetricsRow);
+}
+
+ +
+
Replacement · src/lib/copilot-sync.ts · loop body inside syncUsageMetrics
+
// src/lib/copilot-sync.ts  — replacement loop body inside syncUsageMetrics
+const targetDays = backfillStartDate
+  ? eachDayInRange(backfillStartDate, addDaysUTC(today, -3))
+  : [-3, -4, -5, -6, -7].map((d) => addDaysUTC(today, d));
+
+for (const day of targetDays) {
+  const dayStr = day.toISOString().slice(0, 10);
+  const meta = await fetchCopilotOrgDayReport(token, connection.orgLogin, dayStr);
+  if (meta.status === 204 || !meta.data?.download_links?.length) continue;
+
+  const rows: CopilotMetricsRow[] = [];
+  for (const link of meta.data.download_links) {
+    rows.push(...(await downloadReportNdjson(link)));
+  }
+
+  for (const row of rows) {
+    const mapped = mapNdjsonRowToDbRow(connection.id, row);
+    await db.insert(copilotUsageMetrics).values(mapped).onConflictDoUpdate({
+      target: [copilotUsageMetrics.connectionId, copilotUsageMetrics.date],
+      set: mapped,
+    });
+  }
+}
+
+
+ + +
+

08Rollout sequence

+

Ship the migration in small, reversible steps. The schema goes first; the legacy code goes last.

+ +
    +
  1. Deploy schema migration (Phase 1) — additive only, safe to ship in isolation.
    2. +
  2. Deploy code changes behind a feature flag (Phases 2–3). Flag default OFF.
    3. +
  3. Manually trigger one backfill in production (Phase 4 admin UI) to recover the last 28 days. Verify rows appear in copilot_usage_metrics.
    4. +
  4. Flip the cron flag ON so the daily 06:00 UTC run uses the new path. Watch one full day of sync_events.
    5. +
  5. Remove the legacy code paths in a follow-up PR after one week of stable operation. The old types in src/lib/copilot-api.ts:53-154 (CopilotIdeCodeCompletions, etc.) can be deleted then.
    6. +
  6. Add monitoring — the <MetricsFreshness /> card (Phase 5) is the standing canary going forward.
    7. +
+
+ + +
+

09Risks and mitigations

+

Most of the surface area is well-known. The two unknowns are signed-URL TTL and any unannounced field additions in the NDJSON.

+ +
+
+ risk register + scored against Phase 1–3 scope +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RiskMitigation
Signed URL TTL is undocumented; reports may 403 if we delayAlways fetch immediately after receiving the wrapper response. No queueing between steps.
NDJSON schema is incompletely documented (community #186189)Defensive TypeScript types with [k: string]: unknown; runtime parser ignores unknown keys instead of throwing.
download_links domain churn (now copilot-reports.github.com, was Azure Front Door)Allowlist copilot-reports.github.com, copilot-reports-*.b01.azurefd.net, *.blob.core.windows.net if an outbound proxy gates fetches.
Token scope upgrade requires human actionAdmin must re-mint PAT with read:org before code deploy. Validate via validateCopilotScopes on first sync and fail loudly.
total_active_users / total_engaged_users columns become NULL unless we also pull users-1-dayAcceptable for Phase 1–3; dashboard already tolerates NULLs. Plan Phase-7 follow-up to add per-user fetch.
3-day finalization lag confuses dashboard usersUI annotation on the trailing 3 days: Awaiting GitHub pill on the chart axis.
total_pr_summaries and total_dotcom_chat_turns permanently brokenMark deprecated in column comments; future PR drops them with a separate migration.
Multi-file download_links — rare in practice but supportedConcatenate before parsing; ordering between files is not significant for our per-day upsert.
+
+
+ + +
+

10Test plan

+

Manual verification (in order). Each step gates the next.

+ +
    +
  1. After Phase 0 token swap: from /admin/sync page click "Test connection" — verify validateCopilotScopes now requires and finds read:org.
    2. +
  2. After Phase 1 schema deploy: run pnpm db:push against a Neon branch first; verify only ALTER TABLE ADD COLUMN statements, no destructive ops.
    3. +
  3. After Phase 2–3 code deploy (flag off): unit-test mapNdjsonRowToDbRow against a fixture NDJSON row.
    4. +
  4. Trigger backfill (Phase 4 button) with startDate = today − 28. Confirm ~26 new rows in copilot_usage_metrics.
    5. +
  5. Watch the next cron run at 06:00 UTC the following morning. Confirm sync_events.outcome = success and at least one new row landed for today − 3.
    6. +
  6. After 7 days of clean cron runs, deploy Phase-5 removal of legacy code paths.
    7. +
+
+ + + + +
+ + +
+ + diff --git a/specs/031-copilot-metrics-no-data-investigation/verification-analytics-page.png b/specs/031-copilot-metrics-no-data-investigation/verification-analytics-page.png new file mode 100644 index 0000000..bd13489 Binary files /dev/null and b/specs/031-copilot-metrics-no-data-investigation/verification-analytics-page.png differ diff --git a/specs/031-copilot-metrics-no-data-investigation/verification-backfill-dialog.png b/specs/031-copilot-metrics-no-data-investigation/verification-backfill-dialog.png new file mode 100644 index 0000000..8b9ac78 Binary files /dev/null and b/specs/031-copilot-metrics-no-data-investigation/verification-backfill-dialog.png differ diff --git a/specs/031-copilot-metrics-no-data-investigation/verification-backfill-result.png b/specs/031-copilot-metrics-no-data-investigation/verification-backfill-result.png new file mode 100644 index 0000000..950a016 Binary files /dev/null and b/specs/031-copilot-metrics-no-data-investigation/verification-backfill-result.png differ diff --git a/specs/031-copilot-metrics-no-data-investigation/verification-integrations-page.png b/specs/031-copilot-metrics-no-data-investigation/verification-integrations-page.png new file mode 100644 index 0000000..d8b71fc Binary files /dev/null and b/specs/031-copilot-metrics-no-data-investigation/verification-integrations-page.png differ diff --git a/specs/031-copilot-metrics-no-data-investigation/verification-login-page.png b/specs/031-copilot-metrics-no-data-investigation/verification-login-page.png new file mode 100644 index 0000000..3ecf38c Binary files /dev/null and b/specs/031-copilot-metrics-no-data-investigation/verification-login-page.png differ diff --git a/specs/031-copilot-metrics-no-data-investigation/verification-sync-page.png b/specs/031-copilot-metrics-no-data-investigation/verification-sync-page.png new file mode 100644 index 0000000..48390b5 Binary files /dev/null and b/specs/031-copilot-metrics-no-data-investigation/verification-sync-page.png differ diff --git a/src/actions/copilot-data.ts b/src/actions/copilot-data.ts index 21c4b1a..d11892d 100644 --- a/src/actions/copilot-data.ts +++ b/src/actions/copilot-data.ts @@ -34,6 +34,50 @@ async function getActiveConnection() { }); } +/** + * Days behind today (UTC) before metrics are considered stale. + * + * GitHub finalizes Copilot usage data within ~3 days; we add a 3-day buffer to + * stay quiet during normal latency, surfacing only persistent freshness issues. + */ +const FRESHNESS_STALENESS_THRESHOLD_DAYS = 6; + +export interface CopilotFreshness { + /** Latest metric date in `copilot_usage_metrics`, or null if no rows. */ + latestDate: string | null; + /** Whole UTC days between today and `latestDate`. Null when latestDate null. */ + daysBehind: number | null; + /** True when `daysBehind` exceeds the staleness threshold. */ + stale: boolean; +} + +export async function getCopilotMetricsFreshness(): Promise { + const connection = await getActiveConnection(); + if (!connection) { + return { latestDate: null, daysBehind: null, stale: false }; + } + + const [row] = await db + .select({ latest: sql`MAX(${copilotUsageMetrics.date})` }) + .from(copilotUsageMetrics) + .where(eq(copilotUsageMetrics.connectionId, connection.id)); + + if (!row?.latest) { + return { latestDate: null, daysBehind: null, stale: false }; + } + + const today = new Date(); + const latest = new Date(row.latest); + const msPerDay = 24 * 60 * 60 * 1000; + const daysBehind = Math.floor((today.getTime() - latest.getTime()) / msPerDay); + + return { + latestDate: row.latest, + daysBehind, + stale: daysBehind > FRESHNESS_STALENESS_THRESHOLD_DAYS, + }; +} + function derivePlanType(tierName: string | null): "business" | "enterprise" { return (tierName ?? "").toLowerCase().includes("enterprise") ? "enterprise" diff --git a/src/app/copilot/analytics/page.tsx b/src/app/copilot/analytics/page.tsx index 3b935a4..db01179 100644 --- a/src/app/copilot/analytics/page.tsx +++ b/src/app/copilot/analytics/page.tsx @@ -1,13 +1,20 @@ -import { getCopilotAnalytics } from "@/actions/copilot-data"; +import { + getCopilotAnalytics, + getCopilotMetricsFreshness, +} from "@/actions/copilot-data"; import { LanguageChart } from "@/components/copilot/language-chart"; import { EditorChart } from "@/components/copilot/editor-chart"; import { ActivityDistribution } from "@/components/copilot/activity-distribution"; +import { MetricsFreshnessCard } from "@/components/copilot/metrics-freshness-card"; import { Card, CardContent } from "@/components/ui/card"; import { BarChart3 } from "lucide-react"; import Link from "next/link"; export default async function CopilotAnalyticsPage() { - const result = await getCopilotAnalytics(); + const [result, freshness] = await Promise.all([ + getCopilotAnalytics(), + getCopilotMetricsFreshness(), + ]); if (!result.success) { return ( @@ -20,18 +27,22 @@ export default async function CopilotAnalyticsPage() { if (!hasData) { return ( - - - -

No analytics data yet

-

Enable Copilot sync in Settings to start collecting usage analytics.

- - +
+ + + + +

No analytics data yet

+

Enable Copilot sync in Settings to start collecting usage analytics.

+ + +
); } return (
+
diff --git a/src/app/settings/integrations/github-integration-client.tsx b/src/app/settings/integrations/github-integration-client.tsx index b5b3c7f..84af13f 100644 --- a/src/app/settings/integrations/github-integration-client.tsx +++ b/src/app/settings/integrations/github-integration-client.tsx @@ -173,7 +173,9 @@ export function GitHubIntegrationClient({ initialConnection }: Props) { aria-describedby="token-help" />

- Requires read:org and read:user scopes. + Requires read:org, read:user, and{" "} + manage_billing:copilot scopes (the last for Copilot + billing and usage metrics).

@@ -291,7 +293,16 @@ export function GitHubIntegrationClient({ initialConnection }: Props) { placeholder="ghp_..." value={updateToken} onChange={(e) => setUpdateTokenValue(e.target.value)} + aria-describedby="update-token-help" /> +

+ Requires read:org, read:user, and{" "} + manage_billing:copilot scopes (the last for Copilot + billing and usage metrics). +

- - - ); - } - - // Render ENABLED state - return ( - - -
- Copilot Data Sync - Enabled -
- - Copilot data is synced daily. Last sync:{" "} - {status.lastSyncAt - ? new Date(status.lastSyncAt).toLocaleString() - : "Never"} - {status.lastSyncStatus && ( - <> - {" "} - · Status:{" "} - - - )} - - - - {/* Data Range */} - {status.dataRange && ( -
- Data range:{" "} - {status.dataRange.earliest} to {status.dataRange.latest} -
- )} - - {/* Next Scheduled Sync */} - {status.nextScheduledSync && ( -
- Next scheduled sync:{" "} - {new Date(status.nextScheduledSync).toLocaleString()} -
- )} - - {/* Record Counts */} -
-
-
- {status.recordCounts.metrics} -
-
Metric Days
-
-
-
- {status.recordCounts.billing} -
-
- Billing Snapshots -
-
-
-
- {status.recordCounts.seats} -
-
- Seat Assignments -
-
-
- - {/* Actions */} -
- - - -
- - - ); -} diff --git a/src/components/copilot/metrics-freshness-card.tsx b/src/components/copilot/metrics-freshness-card.tsx new file mode 100644 index 0000000..cc5a5c3 --- /dev/null +++ b/src/components/copilot/metrics-freshness-card.tsx @@ -0,0 +1,35 @@ +import { AlertTriangle } from "lucide-react"; +import Link from "next/link"; +import { Card, CardContent } from "@/components/ui/card"; +import type { CopilotFreshness } from "@/actions/copilot-data"; + +export function MetricsFreshnessCard({ freshness }: { freshness: CopilotFreshness }) { + if (!freshness.stale || freshness.daysBehind == null) return null; + + return ( + + + +
+

+ Copilot metrics are stale — last data {freshness.daysBehind} days ago + {freshness.latestDate ? ` (${freshness.latestDate})` : ""}. +

+

+ GitHub finalizes Copilot usage data within ~3 days, so a 3–4 day lag + is normal. Anything beyond that usually means the sync is failing or + the token is missing the read:org scope. + Check the{" "} + + integration status + + {" "}or run a backfill. +

+
+ + + ); +} diff --git a/src/lib/copilot-api.ts b/src/lib/copilot-api.ts index 197a98e..e187206 100644 --- a/src/lib/copilot-api.ts +++ b/src/lib/copilot-api.ts @@ -51,106 +51,71 @@ interface CopilotSeatsPage { } // --------------------------------------------------------------------------- -// Copilot Metrics types +// Copilot Usage Metrics types (new reports API, replaces sunset 2026-04-02 endpoint) // --------------------------------------------------------------------------- -// Top-level language summary (no suggestion/acceptance counts) -export interface CopilotMetricsLanguageSummary { - name: string; - total_engaged_users: number; -} - -// Detailed language metrics (only at editors[].models[].languages[]) -export interface CopilotMetricsModelLanguage { - name: string; - total_engaged_users: number; - total_code_suggestions: number; - total_code_acceptances: number; - total_code_lines_suggested: number; - total_code_lines_accepted: number; -} - -export interface CopilotMetricsEditorModel { - name: string; - is_custom_model: boolean; - custom_model_training_date?: string | null; - total_engaged_users: number; - languages?: CopilotMetricsModelLanguage[]; -} - -export interface CopilotMetricsEditor { - name: string; - total_engaged_users: number; - models?: CopilotMetricsEditorModel[]; -} - -export interface CopilotIdeCodeCompletions { - total_engaged_users: number; - languages?: CopilotMetricsLanguageSummary[]; - editors?: CopilotMetricsEditor[]; -} - -export interface CopilotIdeChatModel { - name: string; - is_custom_model: boolean; - custom_model_training_date?: string | null; - total_engaged_users: number; - total_chats: number; - total_chat_insertion_events: number; - total_chat_copy_events: number; -} - -export interface CopilotIdeChatEditor { - name: string; - total_engaged_users: number; - models?: CopilotIdeChatModel[]; -} - -export interface CopilotIdeChat { - total_engaged_users: number; - editors?: CopilotIdeChatEditor[]; -} - -export interface CopilotDotcomChatModel { - name: string; - is_custom_model: boolean; - custom_model_training_date?: string | null; - total_engaged_users: number; - total_chats: number; -} - -export interface CopilotDotcomChat { - total_engaged_users: number; - models?: CopilotDotcomChatModel[]; -} - -export interface CopilotDotcomPrModel { - name: string; - is_custom_model: boolean; - custom_model_training_date?: string | null; - total_engaged_users: number; - total_pr_summaries_created: number; -} - -export interface CopilotDotcomPrRepository { - name: string; - total_engaged_users: number; - models?: CopilotDotcomPrModel[]; +/** + * Wrapper response from /orgs/{org}/copilot/metrics/reports/organization-1-day. + * The actual data lives behind one or more signed URLs in `download_links`. + */ +export interface CopilotReportLinks { + download_links: string[]; + report_day?: string; + report_start_day?: string; + report_end_day?: string; } -export interface CopilotDotcomPullRequests { - total_engaged_users: number; - repositories?: CopilotDotcomPrRepository[]; -} +/** + * One row in an NDJSON Copilot usage metrics report. The docs cover "only some" + * fields (community discussion #186189), so this type is intentionally loose: + * known fields are typed, unknown keys are preserved via the index signature. + */ +export interface CopilotMetricsRow { + day?: string; + organization_id?: number; + enterprise_id?: number | null; + user_id?: number; + user_login?: string; + + user_initiated_interaction_count?: number; + code_generation_activity_count?: number; + code_acceptance_activity_count?: number; + + loc_suggested_to_add_sum?: number; + loc_added_sum?: number; + loc_suggested_to_delete_sum?: number; + loc_deleted_sum?: number; + agent_edit?: number; + + chat_panel_agent_mode?: number; + chat_panel_ask_mode?: number; + chat_panel_edit_mode?: number; + chat_panel_plan_mode?: number; + chat_panel_custom_mode?: number; + chat_panel_unknown_mode?: number; + + used_cli?: boolean; + used_agent?: boolean; + used_chat?: boolean; + + totals_by_ide?: Record>; + totals_by_feature?: Record>; + totals_by_language_feature?: Record>; + totals_by_language_model?: Record>; + totals_by_model_feature?: Record>; + totals_by_cli?: { + session_count?: number; + request_count?: number; + prompt_count?: number; + last_known_cli_version?: string; + token_usage?: { + output_tokens_sum?: number; + prompt_tokens_sum?: number; + avg_tokens_per_request?: number; + }; + }; -export interface CopilotDailyMetrics { - date: string; - total_active_users: number; - total_engaged_users: number; - copilot_ide_code_completions: CopilotIdeCodeCompletions | null; - copilot_ide_chat?: CopilotIdeChat | null; - copilot_dotcom_chat?: CopilotDotcomChat | null; - copilot_dotcom_pull_requests?: CopilotDotcomPullRequests | null; + [k: string]: unknown; } // --------------------------------------------------------------------------- @@ -249,31 +214,77 @@ export async function fetchCopilotSeats( } /** - * Fetch Copilot usage metrics for an organization. - * GET /orgs/{org}/copilot/metrics + * Fetch the wrapper response for a single-day org Copilot usage metrics report. + * GET /orgs/{org}/copilot/metrics/reports/organization-1-day?day=YYYY-MM-DD + * + * Replaces the sunset (2026-04-02) endpoint /orgs/{org}/copilot/metrics. The + * actual metric rows live behind signed URLs in the returned `download_links`; + * call `downloadReportNdjson` on each link to fetch the NDJSON data. */ -export async function fetchCopilotMetrics( +export async function fetchCopilotOrgDayReport( token: string, org: string, - since?: string, - until?: string, -): Promise> { - const params: Record = {}; - if (since) params.since = since; - if (until) params.until = until; + day: string, +): Promise> { + return stripScopes( + await githubFetch( + `/orgs/${encodeURIComponent(org)}/copilot/metrics/reports/organization-1-day`, + token, + { day }, + ), + ); +} +/** + * Fetch the wrapper response for a single-day per-user Copilot usage metrics + * report. GET /orgs/{org}/copilot/metrics/reports/users-1-day?day=YYYY-MM-DD + * + * Each row in the downloaded NDJSON represents one (user, day) tuple. Used to + * derive `total_active_users` and `total_engaged_users` counters that the + * org-level report does not expose directly. + */ +export async function fetchCopilotUsersDayReport( + token: string, + org: string, + day: string, +): Promise> { return stripScopes( - await githubFetch( - `/orgs/${encodeURIComponent(org)}/copilot/metrics`, + await githubFetch( + `/orgs/${encodeURIComponent(org)}/copilot/metrics/reports/users-1-day`, token, - Object.keys(params).length > 0 ? params : undefined, + { day }, ), ); } /** - * Validate that the provided token has the `manage_billing:copilot` scope - * by making a lightweight request and inspecting the `x-oauth-scopes` header. + * Download and parse the NDJSON body behind a signed Copilot report URL. + * + * Signed URLs authenticate via the signature itself — do NOT add an + * Authorization header (doing so triggers 403). TTL is undocumented; fetch + * immediately and never cache the URL. + */ +export async function downloadReportNdjson( + url: string, +): Promise { + const res = await fetch(url, { cache: "no-store" }); + if (!res.ok) { + throw new Error( + `Copilot report download ${res.status} from ${new URL(url).host}`, + ); + } + const body = await res.text(); + return body + .split("\n") + .map((line) => line.trim()) + .filter(Boolean) + .map((line) => JSON.parse(line) as CopilotMetricsRow); +} + +/** + * Validate that the provided token has the scopes needed to call both the + * Copilot billing endpoints (`manage_billing:copilot` or `admin:org`) and the + * new Copilot usage metrics reports endpoints (`read:org`). */ export async function validateCopilotScopes( token: string, @@ -281,9 +292,6 @@ export async function validateCopilotScopes( const result = await githubFetch("/user", token); const scopes = result.scopes; - const hasScope = scopes.some( - (s) => s === "manage_billing:copilot" || s === "admin:org", - ); if (result.error) { return { @@ -295,10 +303,21 @@ export async function validateCopilotScopes( }; } - if (!hasScope) { + const hasBillingScope = scopes.some( + (s) => s === "manage_billing:copilot" || s === "admin:org", + ); + const hasOrgReadScope = scopes.some( + (s) => s === "read:org" || s === "admin:org", + ); + + const missing: string[] = []; + if (!hasBillingScope) missing.push("manage_billing:copilot"); + if (!hasOrgReadScope) missing.push("read:org"); + + if (missing.length > 0) { return { data: { valid: false, scopes }, - error: `Token missing required scope: manage_billing:copilot. Current scopes: ${scopes.join(", ")}`, + error: `Token missing required scope(s): ${missing.join(", ")}. The new Copilot usage metrics API requires read:org in addition to manage_billing:copilot. Current scopes: ${scopes.join(", ") || "(none)"}`, status: result.status, rateLimitRemaining: result.rateLimitRemaining, rateLimitReset: result.rateLimitReset, diff --git a/src/lib/copilot-sync.ts b/src/lib/copilot-sync.ts index aa05808..58a7360 100644 --- a/src/lib/copilot-sync.ts +++ b/src/lib/copilot-sync.ts @@ -10,9 +10,12 @@ import { import { fetchCopilotBilling, fetchCopilotSeats, - fetchCopilotMetrics, + fetchCopilotOrgDayReport, + fetchCopilotUsersDayReport, + downloadReportNdjson, + type CopilotMetricsRow, } from "@/lib/copilot-api"; -import { eq, and, desc } from "drizzle-orm"; +import { eq, and } from "drizzle-orm"; // --------------------------------------------------------------------------- // Types @@ -287,187 +290,172 @@ export async function syncSeatAssignments( // Function 3: syncUsageMetrics // --------------------------------------------------------------------------- -export async function syncUsageMetrics( - connection: SyncConnection, - token: string, -): Promise { - // Find latest metric date for this connection - const latestRow = await db.query.copilotUsageMetrics.findFirst({ - where: eq(copilotUsageMetrics.connectionId, connection.id), - orderBy: desc(copilotUsageMetrics.date), - }); - - let since: string | undefined; - if (latestRow) { - // Use latest + 1 day as since - const latestDate = new Date(latestRow.date); - latestDate.setUTCDate(latestDate.getUTCDate() + 1); - since = latestDate.toISOString().split("T")[0]; - } +/** + * Number of UTC days before today that we wait before trusting a daily report. + * GitHub finalizes Copilot usage data within ~3 days; days more recent than + * `today - FINALIZATION_LAG_DAYS` are not yet stable. + * + * https://docs.github.com/en/copilot/reference/copilot-usage-metrics/reconciling-usage-metrics + */ +const FINALIZATION_LAG_DAYS = 3; + +/** + * How many additional days to re-fetch behind `FINALIZATION_LAG_DAYS` on each + * regular run. Catches late-arriving telemetry rows; idempotent upserts make + * the re-fetch a no-op when nothing changed. + */ +const RESTABILIZE_WINDOW_DAYS = 4; + +function utcDay(d: Date): string { + return d.toISOString().slice(0, 10); +} - const metricsResponse = await fetchCopilotMetrics( - token, - connection.orgLogin, - since, - ); +function addUtcDays(d: Date, days: number): Date { + const next = new Date(d); + next.setUTCDate(next.getUTCDate() + days); + return next; +} - if (metricsResponse.error || !metricsResponse.data) { - // GitHub returns 404 when the org has the Copilot Metrics API policy - // disabled, or when fewer than 5 members have generated telemetry in the - // window. The admin has to flip that on GitHub's side — there's nothing - // we can do from here, so don't fail the whole sync over it. - if (metricsResponse.status === 404) { - console.warn( - `[copilot-sync] Copilot metrics not available for org "${connection.orgLogin}" (404). The "Copilot Metrics API access" policy may be disabled, or the org has fewer than 5 users emitting telemetry. See https://docs.github.com/en/copilot/managing-copilot/managing-policies-and-features-for-copilot-in-your-organization`, - ); - return { metricsProcessed: 0 }; - } - throw new Error( - metricsResponse.error ?? "Failed to fetch Copilot usage metrics", - ); +function daysInRange(start: Date, end: Date): string[] { + const out: string[] = []; + for (let cur = new Date(start); cur <= end; cur = addUtcDays(cur, 1)) { + out.push(utcDay(cur)); } + return out; +} - const metrics = metricsResponse.data; - - for (const day of metrics) { - const completions = day.copilot_ide_code_completions; +/** + * Map one NDJSON row from the org-1-day report onto a row of + * `copilot_usage_metrics`. Exported for unit testing. + * + * `activeUsers` and `engagedUsers` come from a separate users-1-day fetch + * (the org-level report does not expose them as flat counters), supplied by + * the caller. + */ +export function mapNdjsonRowToDbRow( + connectionId: number, + date: string, + row: CopilotMetricsRow, + userCounts: { active: number; engaged: number }, +) { + const chatModeKeys = [ + "chat_panel_agent_mode", + "chat_panel_ask_mode", + "chat_panel_edit_mode", + "chat_panel_plan_mode", + "chat_panel_custom_mode", + "chat_panel_unknown_mode", + ] as const; + const hasChatData = chatModeKeys.some((k) => row[k] !== undefined); + const chatTurns = chatModeKeys.reduce( + (sum, k) => sum + ((row[k] as number | undefined) ?? 0), + 0, + ); - // Aggregate language metrics from editors[].models[].languages[] - // (the top-level languages[] only has name + engaged_users, no counts) - const langTotals = new Map< - string, - { - suggestions: number; - acceptances: number; - linesSuggested: number; - linesAccepted: number; - } - >(); - - let totalSuggestions = 0; - let totalAcceptances = 0; - let totalLinesSuggested = 0; - let totalLinesAccepted = 0; - - for (const editor of completions?.editors ?? []) { - for (const model of editor.models ?? []) { - for (const lang of model.languages ?? []) { - totalSuggestions += lang.total_code_suggestions ?? 0; - totalAcceptances += lang.total_code_acceptances ?? 0; - totalLinesSuggested += lang.total_code_lines_suggested ?? 0; - totalLinesAccepted += lang.total_code_lines_accepted ?? 0; - - const existing = langTotals.get(lang.name); - if (existing) { - existing.suggestions += lang.total_code_suggestions ?? 0; - existing.acceptances += lang.total_code_acceptances ?? 0; - existing.linesSuggested += lang.total_code_lines_suggested ?? 0; - existing.linesAccepted += lang.total_code_lines_accepted ?? 0; - } else { - langTotals.set(lang.name, { - suggestions: lang.total_code_suggestions ?? 0, - acceptances: lang.total_code_acceptances ?? 0, - linesSuggested: lang.total_code_lines_suggested ?? 0, - linesAccepted: lang.total_code_lines_accepted ?? 0, - }); - } - } - } - } + return { + connectionId, + date, + totalActiveUsers: userCounts.active, + totalEngagedUsers: userCounts.engaged, + totalSuggestions: row.code_generation_activity_count ?? 0, + totalAcceptances: row.code_acceptance_activity_count ?? 0, + totalLinesSuggested: row.loc_suggested_to_add_sum ?? 0, + totalLinesAccepted: row.loc_added_sum ?? 0, + totalChatTurns: hasChatData ? chatTurns : null, + totalChatAcceptances: null, + // GitHub removed these on 2026-04-02; written as null going forward. + totalDotcomChatTurns: null, + totalPrSummaries: null, + languageBreakdown: row.totals_by_language_feature ?? null, + editorBreakdown: row.totals_by_ide ?? null, + usedCli: row.totals_by_cli !== undefined, + usedAgent: + (row.chat_panel_agent_mode ?? 0) > 0 || (row.agent_edit ?? 0) > 0, + agentEditCount: row.agent_edit ?? null, + cliBreakdown: row.totals_by_cli ?? null, + }; +} - const languageBreakdown = [...langTotals.entries()].map( - ([language, totals]) => ({ - language, - ...totals, - }), +/** + * When `backfillStartDate` is set, sync every UTC day from that date through + * today minus the finalization lag. When unset, sync the rolling restabilization + * window: `today − (FINALIZATION_LAG_DAYS + RESTABILIZE_WINDOW_DAYS)` through + * `today − FINALIZATION_LAG_DAYS`. + */ +export async function syncUsageMetrics( + connection: SyncConnection, + token: string, + opts: { backfillStartDate?: Date } = {}, +): Promise { + const newest = addUtcDays(new Date(), -FINALIZATION_LAG_DAYS); + const oldest = opts.backfillStartDate + ? new Date(opts.backfillStartDate) + : addUtcDays(newest, -RESTABILIZE_WINDOW_DAYS); + const targetDays = daysInRange(oldest, newest); + + let processed = 0; + + for (const day of targetDays) { + const orgMeta = await fetchCopilotOrgDayReport( + token, + connection.orgLogin, + day, ); - // Build editor breakdown by summing across models[].languages[] - const editorBreakdown = - completions?.editors?.map((e) => { - let suggestions = 0; - let acceptances = 0; - for (const m of e.models ?? []) { - for (const l of m.languages ?? []) { - suggestions += l.total_code_suggestions ?? 0; - acceptances += l.total_code_acceptances ?? 0; - } - } - return { - editor: e.name, - engagedUsers: e.total_engaged_users, - suggestions, - acceptances, - }; - }) ?? []; - - // Aggregate chat metrics from editors[].models[] - let totalChatTurns = 0; - let totalChatAcceptances = 0; - let hasChatData = false; - for (const editor of day.copilot_ide_chat?.editors ?? []) { - for (const model of editor.models ?? []) { - hasChatData = true; - totalChatTurns += model.total_chats ?? 0; - totalChatAcceptances += model.total_chat_insertion_events ?? 0; - } + if (orgMeta.error) { + throw new Error( + `Copilot org-day report failed for ${day}: ${orgMeta.error}`, + ); } - - // Aggregate dotcom chat from models[] - let totalDotcomChatTurns = 0; - let hasDotcomChat = false; - for (const model of day.copilot_dotcom_chat?.models ?? []) { - hasDotcomChat = true; - totalDotcomChatTurns += model.total_chats ?? 0; + // 204 No Content or empty links means GitHub hasn't generated a report for + // this day yet; that's expected behind the finalization lag. + if ( + orgMeta.status === 204 || + !orgMeta.data || + !orgMeta.data.download_links?.length + ) { + continue; } - // Aggregate PR summaries from repositories[].models[] - let totalPrSummaries = 0; - let hasPrData = false; - for (const repo of day.copilot_dotcom_pull_requests?.repositories ?? []) { - for (const model of repo.models ?? []) { - hasPrData = true; - totalPrSummaries += model.total_pr_summaries_created ?? 0; + const orgRows: CopilotMetricsRow[] = []; + for (const link of orgMeta.data.download_links) { + orgRows.push(...(await downloadReportNdjson(link))); + } + if (orgRows.length === 0) continue; + const orgRow = orgRows[0]; + + // Per-user counts come from the users-1-day report. Required to keep the + // schema's NOT NULL invariant on `total_active_users` / `total_engaged_users`. + const usersMeta = await fetchCopilotUsersDayReport( + token, + connection.orgLogin, + day, + ); + let userCounts = { active: 0, engaged: 0 }; + if (usersMeta.data?.download_links?.length) { + const userRows: CopilotMetricsRow[] = []; + for (const link of usersMeta.data.download_links) { + userRows.push(...(await downloadReportNdjson(link))); } + userCounts = { + active: userRows.length, + engaged: userRows.filter( + (r) => (r.user_initiated_interaction_count ?? 0) > 0, + ).length, + }; } + const mapped = mapNdjsonRowToDbRow(connection.id, day, orgRow, userCounts); await db .insert(copilotUsageMetrics) - .values({ - connectionId: connection.id, - date: day.date, - totalActiveUsers: day.total_active_users, - totalEngagedUsers: day.total_engaged_users, - totalSuggestions, - totalAcceptances, - totalLinesSuggested, - totalLinesAccepted, - totalChatTurns: hasChatData ? totalChatTurns : null, - totalChatAcceptances: hasChatData ? totalChatAcceptances : null, - totalDotcomChatTurns: hasDotcomChat ? totalDotcomChatTurns : null, - totalPrSummaries: hasPrData ? totalPrSummaries : null, - languageBreakdown, - editorBreakdown, - }) + .values(mapped) .onConflictDoUpdate({ target: [copilotUsageMetrics.connectionId, copilotUsageMetrics.date], - set: { - totalActiveUsers: day.total_active_users, - totalEngagedUsers: day.total_engaged_users, - totalSuggestions, - totalAcceptances, - totalLinesSuggested, - totalLinesAccepted, - totalChatTurns: hasChatData ? totalChatTurns : null, - totalChatAcceptances: hasChatData ? totalChatAcceptances : null, - totalDotcomChatTurns: hasDotcomChat ? totalDotcomChatTurns : null, - totalPrSummaries: hasPrData ? totalPrSummaries : null, - languageBreakdown, - editorBreakdown, - }, + set: mapped, }); + processed++; } - return { metricsProcessed: metrics.length }; + return { metricsProcessed: processed }; } diff --git a/src/lib/db/migrations/0020_daily_rachel_grey.sql b/src/lib/db/migrations/0020_daily_rachel_grey.sql new file mode 100644 index 0000000..1268a8b --- /dev/null +++ b/src/lib/db/migrations/0020_daily_rachel_grey.sql @@ -0,0 +1,4 @@ +ALTER TABLE "copilot_usage_metrics" ADD COLUMN "used_cli" boolean;--> statement-breakpoint +ALTER TABLE "copilot_usage_metrics" ADD COLUMN "used_agent" boolean;--> statement-breakpoint +ALTER TABLE "copilot_usage_metrics" ADD COLUMN "agent_edit_count" integer;--> statement-breakpoint +ALTER TABLE "copilot_usage_metrics" ADD COLUMN "cli_breakdown" jsonb; \ No newline at end of file diff --git a/src/lib/db/migrations/meta/0020_snapshot.json b/src/lib/db/migrations/meta/0020_snapshot.json new file mode 100644 index 0000000..b27668c --- /dev/null +++ b/src/lib/db/migrations/meta/0020_snapshot.json @@ -0,0 +1,3753 @@ +{ + "id": "a9298155-c5c5-42ac-9ab0-3f67e0cd6c9f", + "prevId": "75224b1d-f37d-4283-8b4d-6fd85c802ec5", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.access_tiers": { + "name": "access_tiers", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "tool_id": { + "name": "tool_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "monthly_cost_cents": { + "name": "monthly_cost_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "is_active": { + "name": "is_active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "access_tiers_tool_id_idx": { + "name": "access_tiers_tool_id_idx", + "columns": [ + { + "expression": "tool_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "access_tiers_tool_name_idx": { + "name": "access_tiers_tool_name_idx", + "columns": [ + { + "expression": "tool_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "access_tiers_tool_id_ai_tools_id_fk": { + "name": "access_tiers_tool_id_ai_tools_id_fk", + "tableFrom": "access_tiers", + "tableTo": "ai_tools", + "columnsFrom": [ + "tool_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ai_tools": { + "name": "ai_tools", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "vendor": { + "name": "vendor", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "max_licenses": { + "name": "max_licenses", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "tool_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "ai_tools_name_idx": { + "name": "ai_tools_name_idx", + "columns": [ + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "ai_tools_vendor_idx": { + "name": "ai_tools_vendor_idx", + "columns": [ + { + "expression": "vendor", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.annual_budgets": { + "name": "annual_budgets", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "fiscal_year": { + "name": "fiscal_year", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_amount_cents": { + "name": "total_amount_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "period_type": { + "name": "period_type", + "type": "period_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "budget_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "annual_budgets_fiscal_year_idx": { + "name": "annual_budgets_fiscal_year_idx", + "columns": [ + { + "expression": "fiscal_year", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "annual_budgets_status_idx": { + "name": "annual_budgets_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.anthropic_alert_state": { + "name": "anthropic_alert_state", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "billing_month": { + "name": "billing_month", + "type": "varchar(7)", + "primaryKey": false, + "notNull": true + }, + "threshold_80_fired_at": { + "name": "threshold_80_fired_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "threshold_100_fired_at": { + "name": "threshold_100_fired_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "threshold_120_fired_at": { + "name": "threshold_120_fired_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "forecast_at_risk": { + "name": "forecast_at_risk", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "forecast_changed_at": { + "name": "forecast_changed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "anthropic_alert_state_workspace_month_idx": { + "name": "anthropic_alert_state_workspace_month_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "billing_month", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_alert_state\".\"workspace_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_alert_state_default_month_idx": { + "name": "anthropic_alert_state_default_month_idx", + "columns": [ + { + "expression": "billing_month", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_alert_state\".\"workspace_id\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_alert_state_month_idx": { + "name": "anthropic_alert_state_month_idx", + "columns": [ + { + "expression": "billing_month", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "anthropic_alert_state_billing_month_format": { + "name": "anthropic_alert_state_billing_month_format", + "value": "\"anthropic_alert_state\".\"billing_month\" ~ '^[0-9]{4}-(0[1-9]|1[0-2])$'" + } + }, + "isRLSEnabled": false + }, + "public.anthropic_org_config": { + "name": "anthropic_org_config", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "integer", + "primaryKey": true, + "notNull": true, + "default": 1 + }, + "billing_budget_limit_cents": { + "name": "billing_budget_limit_cents", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_by": { + "name": "updated_by", + "type": "integer", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "anthropic_org_config_updated_by_users_id_fk": { + "name": "anthropic_org_config_updated_by_users_id_fk", + "tableFrom": "anthropic_org_config", + "tableTo": "users", + "columnsFrom": [ + "updated_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "anthropic_org_config_id_check": { + "name": "anthropic_org_config_id_check", + "value": "\"anthropic_org_config\".\"id\" = 1" + } + }, + "isRLSEnabled": false + }, + "public.anthropic_sync_status": { + "name": "anthropic_sync_status", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_sync_completed_at": { + "name": "last_sync_completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_sync_error": { + "name": "last_sync_error", + "type": "varchar(500)", + "primaryKey": false, + "notNull": false + }, + "synced_days": { + "name": "synced_days", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "resolved_api_key_id": { + "name": "resolved_api_key_id", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "resolved_workspace_id": { + "name": "resolved_workspace_id", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "workspace_sync_completed_at": { + "name": "workspace_sync_completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "anthropic_sync_status_user_id_idx": { + "name": "anthropic_sync_status_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.anthropic_usage_metrics": { + "name": "anthropic_usage_metrics", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "date": { + "name": "date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "model": { + "name": "model", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "uncached_input_tokens": { + "name": "uncached_input_tokens", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "cache_read_input_tokens": { + "name": "cache_read_input_tokens", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "cache_creation_input_tokens": { + "name": "cache_creation_input_tokens", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "output_tokens": { + "name": "output_tokens", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "computed_cost_cents": { + "name": "computed_cost_cents", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "pricing_resolved": { + "name": "pricing_resolved", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "anthropic_usage_metrics_user_date_model_idx": { + "name": "anthropic_usage_metrics_user_date_model_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "model", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_usage_metrics_user_date_idx": { + "name": "anthropic_usage_metrics_user_date_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_usage_metrics_date_idx": { + "name": "anthropic_usage_metrics_date_idx", + "columns": [ + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_usage_metrics_pricing_resolved_idx": { + "name": "anthropic_usage_metrics_pricing_resolved_idx", + "columns": [ + { + "expression": "pricing_resolved", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "anthropic_usage_metrics_user_id_users_id_fk": { + "name": "anthropic_usage_metrics_user_id_users_id_fk", + "tableFrom": "anthropic_usage_metrics", + "tableTo": "users", + "columnsFrom": [ + "user_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.anthropic_workspace_costs": { + "name": "anthropic_workspace_costs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "date": { + "name": "date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "cost_cents": { + "name": "cost_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "anthropic_workspace_costs_workspace_date_idx": { + "name": "anthropic_workspace_costs_workspace_date_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_workspace_costs\".\"workspace_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_workspace_costs_default_date_idx": { + "name": "anthropic_workspace_costs_default_date_idx", + "columns": [ + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_workspace_costs\".\"workspace_id\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_workspace_costs_date_idx": { + "name": "anthropic_workspace_costs_date_idx", + "columns": [ + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_workspace_costs_workspace_id_idx": { + "name": "anthropic_workspace_costs_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "anthropic_workspace_costs_cost_cents_check": { + "name": "anthropic_workspace_costs_cost_cents_check", + "value": "\"anthropic_workspace_costs\".\"cost_cents\" >= 0" + } + }, + "isRLSEnabled": false + }, + "public.anthropic_workspace_limits": { + "name": "anthropic_workspace_limits", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "limit_cents": { + "name": "limit_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "anthropic_workspace_limits_workspace_id_idx": { + "name": "anthropic_workspace_limits_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_workspace_limits\".\"workspace_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_workspace_limits_default_idx": { + "name": "anthropic_workspace_limits_default_idx", + "columns": [ + { + "expression": "(1)", + "asc": true, + "isExpression": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_workspace_limits\".\"workspace_id\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.anthropic_workspaces": { + "name": "anthropic_workspaces", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "display_color": { + "name": "display_color", + "type": "varchar(20)", + "primaryKey": false, + "notNull": false + }, + "is_default": { + "name": "is_default", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_archived": { + "name": "is_archived", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "anthropic_created_at": { + "name": "anthropic_created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_seen_at": { + "name": "last_seen_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "anthropic_workspaces_workspace_id_idx": { + "name": "anthropic_workspaces_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_workspaces\".\"workspace_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_workspaces_is_default_idx": { + "name": "anthropic_workspaces_is_default_idx", + "columns": [ + { + "expression": "is_default", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"anthropic_workspaces\".\"is_default\" = true", + "concurrently": false, + "method": "btree", + "with": {} + }, + "anthropic_workspaces_archived_idx": { + "name": "anthropic_workspaces_archived_idx", + "columns": [ + { + "expression": "is_archived", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.assignment_comments": { + "name": "assignment_comments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "assignment_id": { + "name": "assignment_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "author_id": { + "name": "author_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "body": { + "name": "body", + "type": "varchar(2000)", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "assignment_comments_assignment_id_idx": { + "name": "assignment_comments_assignment_id_idx", + "columns": [ + { + "expression": "assignment_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "assignment_comments_author_id_idx": { + "name": "assignment_comments_author_id_idx", + "columns": [ + { + "expression": "author_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "assignment_comments_created_at_idx": { + "name": "assignment_comments_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "assignment_comments_assignment_id_license_assignments_id_fk": { + "name": "assignment_comments_assignment_id_license_assignments_id_fk", + "tableFrom": "assignment_comments", + "tableTo": "license_assignments", + "columnsFrom": [ + "assignment_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "assignment_comments_author_id_users_id_fk": { + "name": "assignment_comments_author_id_users_id_fk", + "tableFrom": "assignment_comments", + "tableTo": "users", + "columnsFrom": [ + "author_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.billed_costs": { + "name": "billed_costs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "period_id": { + "name": "period_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "amount_cents": { + "name": "amount_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "invoice_date": { + "name": "invoice_date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "varchar(500)", + "primaryKey": false, + "notNull": true + }, + "vendor_reference": { + "name": "vendor_reference", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "billed_costs_period_id_idx": { + "name": "billed_costs_period_id_idx", + "columns": [ + { + "expression": "period_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "billed_costs_invoice_date_idx": { + "name": "billed_costs_invoice_date_idx", + "columns": [ + { + "expression": "invoice_date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "billed_costs_period_id_budget_periods_id_fk": { + "name": "billed_costs_period_id_budget_periods_id_fk", + "tableFrom": "billed_costs", + "tableTo": "budget_periods", + "columnsFrom": [ + "period_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.budget_periods": { + "name": "budget_periods", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "budget_id": { + "name": "budget_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "period_label": { + "name": "period_label", + "type": "varchar(20)", + "primaryKey": false, + "notNull": true + }, + "period_index": { + "name": "period_index", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "start_date": { + "name": "start_date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "end_date": { + "name": "end_date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "planned_amount_cents": { + "name": "planned_amount_cents", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "budget_periods_budget_id_idx": { + "name": "budget_periods_budget_id_idx", + "columns": [ + { + "expression": "budget_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "budget_periods_budget_period_idx": { + "name": "budget_periods_budget_period_idx", + "columns": [ + { + "expression": "budget_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "period_index", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "budget_periods_budget_id_annual_budgets_id_fk": { + "name": "budget_periods_budget_id_annual_budgets_id_fk", + "tableFrom": "budget_periods", + "tableTo": "annual_budgets", + "columnsFrom": [ + "budget_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.change_history": { + "name": "change_history", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "entity_type": { + "name": "entity_type", + "type": "varchar(50)", + "primaryKey": false, + "notNull": true + }, + "entity_id": { + "name": "entity_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "change_type": { + "name": "change_type", + "type": "change_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "field_name": { + "name": "field_name", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "previous_value": { + "name": "previous_value", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "new_value": { + "name": "new_value", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "changed_by": { + "name": "changed_by", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "change_history_entity_idx": { + "name": "change_history_entity_idx", + "columns": [ + { + "expression": "entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "change_history_changed_by_idx": { + "name": "change_history_changed_by_idx", + "columns": [ + { + "expression": "changed_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "change_history_created_at_idx": { + "name": "change_history_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "change_history_changed_by_users_id_fk": { + "name": "change_history_changed_by_users_id_fk", + "tableFrom": "change_history", + "tableTo": "users", + "columnsFrom": [ + "changed_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_billing_snapshots": { + "name": "copilot_billing_snapshots", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "connection_id": { + "name": "connection_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "billing_month": { + "name": "billing_month", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "plan_type": { + "name": "plan_type", + "type": "varchar(50)", + "primaryKey": false, + "notNull": true + }, + "total_seats": { + "name": "total_seats", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "active_seats": { + "name": "active_seats", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "seat_cost_cents": { + "name": "seat_cost_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_cost_cents": { + "name": "total_cost_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_billing_snapshots_connection_month_idx": { + "name": "copilot_billing_snapshots_connection_month_idx", + "columns": [ + { + "expression": "connection_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "billing_month", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_billing_snapshots_connection_id_github_connections_id_fk": { + "name": "copilot_billing_snapshots_connection_id_github_connections_id_fk", + "tableFrom": "copilot_billing_snapshots", + "tableTo": "github_connections", + "columnsFrom": [ + "connection_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_usage_metrics": { + "name": "copilot_usage_metrics", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "connection_id": { + "name": "connection_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "date": { + "name": "date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "total_active_users": { + "name": "total_active_users", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_engaged_users": { + "name": "total_engaged_users", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_suggestions": { + "name": "total_suggestions", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_acceptances": { + "name": "total_acceptances", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_lines_suggested": { + "name": "total_lines_suggested", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_lines_accepted": { + "name": "total_lines_accepted", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "total_chat_turns": { + "name": "total_chat_turns", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "total_chat_acceptances": { + "name": "total_chat_acceptances", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "total_dotcom_chat_turns": { + "name": "total_dotcom_chat_turns", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "total_pr_summaries": { + "name": "total_pr_summaries", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "language_breakdown": { + "name": "language_breakdown", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "editor_breakdown": { + "name": "editor_breakdown", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "used_cli": { + "name": "used_cli", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "used_agent": { + "name": "used_agent", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "agent_edit_count": { + "name": "agent_edit_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "cli_breakdown": { + "name": "cli_breakdown", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_usage_metrics_connection_date_idx": { + "name": "copilot_usage_metrics_connection_date_idx", + "columns": [ + { + "expression": "connection_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_usage_metrics_date_idx": { + "name": "copilot_usage_metrics_date_idx", + "columns": [ + { + "expression": "date", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_usage_metrics_connection_id_github_connections_id_fk": { + "name": "copilot_usage_metrics_connection_id_github_connections_id_fk", + "tableFrom": "copilot_usage_metrics", + "tableTo": "github_connections", + "columnsFrom": [ + "connection_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.github_connections": { + "name": "github_connections", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "org_login": { + "name": "org_login", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "org_id": { + "name": "org_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "org_avatar_url": { + "name": "org_avatar_url", + "type": "varchar(500)", + "primaryKey": false, + "notNull": false + }, + "token_encrypted": { + "name": "token_encrypted", + "type": "varchar(700)", + "primaryKey": false, + "notNull": true + }, + "token_scopes_csv": { + "name": "token_scopes_csv", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "github_connection_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "connected_by": { + "name": "connected_by", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "connected_at": { + "name": "connected_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "disconnected_at": { + "name": "disconnected_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_sync_at": { + "name": "last_sync_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "copilot_sync_enabled": { + "name": "copilot_sync_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "copilot_sync_schedule": { + "name": "copilot_sync_schedule", + "type": "varchar(50)", + "primaryKey": false, + "notNull": true, + "default": "'daily'" + } + }, + "indexes": { + "github_connections_status_idx": { + "name": "github_connections_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "github_connections_connected_by_users_id_fk": { + "name": "github_connections_connected_by_users_id_fk", + "tableFrom": "github_connections", + "tableTo": "users", + "columnsFrom": [ + "connected_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.github_profiles": { + "name": "github_profiles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "github_id": { + "name": "github_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "github_login": { + "name": "github_login", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "avatar_url": { + "name": "avatar_url", + "type": "varchar(500)", + "primaryKey": false, + "notNull": false + }, + "bio": { + "name": "bio", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "public_repos": { + "name": "public_repos", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "profile_url": { + "name": "profile_url", + "type": "varchar(500)", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "email": { + "name": "email", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "last_synced_at": { + "name": "last_synced_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "github_profiles_user_id_idx": { + "name": "github_profiles_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "github_profiles_github_id_idx": { + "name": "github_profiles_github_id_idx", + "columns": [ + { + "expression": "github_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "github_profiles_github_login_idx": { + "name": "github_profiles_github_login_idx", + "columns": [ + { + "expression": "github_login", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "github_profiles_user_id_users_id_fk": { + "name": "github_profiles_user_id_users_id_fk", + "tableFrom": "github_profiles", + "tableTo": "users", + "columnsFrom": [ + "user_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.github_sync_events": { + "name": "github_sync_events", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "connection_id": { + "name": "connection_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "triggered_by": { + "name": "triggered_by", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "github_sync_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "total_members": { + "name": "total_members", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "matched_count": { + "name": "matched_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "imported_count": { + "name": "imported_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "unmatched_count": { + "name": "unmatched_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "conflict_count": { + "name": "conflict_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "manually_matched_count": { + "name": "manually_matched_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "created_count": { + "name": "created_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_type": { + "name": "sync_type", + "type": "copilot_sync_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'members'" + }, + "seats_processed": { + "name": "seats_processed", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "metrics_processed": { + "name": "metrics_processed", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "billing_processed": { + "name": "billing_processed", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "billing_linked": { + "name": "billing_linked", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "billing_skipped": { + "name": "billing_skipped", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "github_sync_events_connection_id_idx": { + "name": "github_sync_events_connection_id_idx", + "columns": [ + { + "expression": "connection_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "github_sync_events_triggered_by_idx": { + "name": "github_sync_events_triggered_by_idx", + "columns": [ + { + "expression": "triggered_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "github_sync_events_connection_id_github_connections_id_fk": { + "name": "github_sync_events_connection_id_github_connections_id_fk", + "tableFrom": "github_sync_events", + "tableTo": "github_connections", + "columnsFrom": [ + "connection_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "github_sync_events_triggered_by_users_id_fk": { + "name": "github_sync_events_triggered_by_users_id_fk", + "tableFrom": "github_sync_events", + "tableTo": "users", + "columnsFrom": [ + "triggered_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_filters": { + "name": "ingestion_filters", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "field": { + "name": "field", + "type": "filter_field", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "mode": { + "name": "mode", + "type": "filter_mode", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "value": { + "name": "value", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "created_by": { + "name": "created_by", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "ingestion_filters_enabled_idx": { + "name": "ingestion_filters_enabled_idx", + "columns": [ + { + "expression": "enabled", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "ingestion_filters_created_by_users_id_fk": { + "name": "ingestion_filters_created_by_users_id_fk", + "tableFrom": "ingestion_filters", + "tableTo": "users", + "columnsFrom": [ + "created_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_log": { + "name": "ingestion_log", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "filename": { + "name": "filename", + "type": "varchar(500)", + "primaryKey": false, + "notNull": false + }, + "vendor": { + "name": "vendor", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "invoice_number": { + "name": "invoice_number", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "invoice_date": { + "name": "invoice_date", + "type": "date", + "primaryKey": false, + "notNull": false + }, + "amount_cents": { + "name": "amount_cents", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "outcome": { + "name": "outcome", + "type": "ingestion_outcome", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "channel": { + "name": "channel", + "type": "ingestion_channel", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "blob_pathname": { + "name": "blob_pathname", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "linked_invoice_id": { + "name": "linked_invoice_id", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "uploaded_by": { + "name": "uploaded_by", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "ingestion_log_outcome_idx": { + "name": "ingestion_log_outcome_idx", + "columns": [ + { + "expression": "outcome", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "ingestion_log_created_at_idx": { + "name": "ingestion_log_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "ingestion_log_vendor_idx": { + "name": "ingestion_log_vendor_idx", + "columns": [ + { + "expression": "vendor", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "ingestion_log_channel_idx": { + "name": "ingestion_log_channel_idx", + "columns": [ + { + "expression": "channel", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "ingestion_log_linked_invoice_id_invoices_id_fk": { + "name": "ingestion_log_linked_invoice_id_invoices_id_fk", + "tableFrom": "ingestion_log", + "tableTo": "invoices", + "columnsFrom": [ + "linked_invoice_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "ingestion_log_uploaded_by_users_id_fk": { + "name": "ingestion_log_uploaded_by_users_id_fk", + "tableFrom": "ingestion_log", + "tableTo": "users", + "columnsFrom": [ + "uploaded_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.invite_tokens": { + "name": "invite_tokens", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token_hash": { + "name": "token_hash", + "type": "varchar(64)", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "invite_token_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "consumed_at": { + "name": "consumed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "invite_tokens_token_hash_idx": { + "name": "invite_tokens_token_hash_idx", + "columns": [ + { + "expression": "token_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invite_tokens_user_id_idx": { + "name": "invite_tokens_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invite_tokens_active_user_idx": { + "name": "invite_tokens_active_user_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"invite_tokens\".\"status\" = 'active'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "invite_tokens_user_id_users_id_fk": { + "name": "invite_tokens_user_id_users_id_fk", + "tableFrom": "invite_tokens", + "tableTo": "users", + "columnsFrom": [ + "user_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.invoices": { + "name": "invoices", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "invoice_number": { + "name": "invoice_number", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "invoice_date": { + "name": "invoice_date", + "type": "date", + "primaryKey": false, + "notNull": true + }, + "amount_cents": { + "name": "amount_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "vendor": { + "name": "vendor", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "linked_billed_cost_id": { + "name": "linked_billed_cost_id", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "blob_url": { + "name": "blob_url", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "blob_pathname": { + "name": "blob_pathname", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "filtered_out": { + "name": "filtered_out", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "uploaded_by": { + "name": "uploaded_by", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "invoices_invoice_number_idx": { + "name": "invoices_invoice_number_idx", + "columns": [ + { + "expression": "invoice_number", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invoices_created_at_idx": { + "name": "invoices_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invoices_linked_billed_cost_id_idx": { + "name": "invoices_linked_billed_cost_id_idx", + "columns": [ + { + "expression": "linked_billed_cost_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "invoices_linked_billed_cost_id_billed_costs_id_fk": { + "name": "invoices_linked_billed_cost_id_billed_costs_id_fk", + "tableFrom": "invoices", + "tableTo": "billed_costs", + "columnsFrom": [ + "linked_billed_cost_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "invoices_uploaded_by_users_id_fk": { + "name": "invoices_uploaded_by_users_id_fk", + "tableFrom": "invoices", + "tableTo": "users", + "columnsFrom": [ + "uploaded_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.license_assignments": { + "name": "license_assignments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "tool_id": { + "name": "tool_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "tier_id": { + "name": "tier_id", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "cost_at_assignment_cents": { + "name": "cost_at_assignment_cents", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "assignment_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "assigned_at": { + "name": "assigned_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "revoked_at": { + "name": "revoked_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "workspace": { + "name": "workspace", + "type": "varchar(200)", + "primaryKey": false, + "notNull": false + }, + "api_key_encrypted": { + "name": "api_key_encrypted", + "type": "varchar(700)", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "source": { + "name": "source", + "type": "varchar(50)", + "primaryKey": false, + "notNull": true, + "default": "'manual'" + } + }, + "indexes": { + "license_assignments_user_id_idx": { + "name": "license_assignments_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "license_assignments_tool_id_idx": { + "name": "license_assignments_tool_id_idx", + "columns": [ + { + "expression": "tool_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "license_assignments_tier_id_idx": { + "name": "license_assignments_tier_id_idx", + "columns": [ + { + "expression": "tier_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "license_assignments_status_idx": { + "name": "license_assignments_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "license_assignments_active_lookup_idx": { + "name": "license_assignments_active_lookup_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "tool_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "license_assignments_user_id_users_id_fk": { + "name": "license_assignments_user_id_users_id_fk", + "tableFrom": "license_assignments", + "tableTo": "users", + "columnsFrom": [ + "user_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + }, + "license_assignments_tool_id_ai_tools_id_fk": { + "name": "license_assignments_tool_id_ai_tools_id_fk", + "tableFrom": "license_assignments", + "tableTo": "ai_tools", + "columnsFrom": [ + "tool_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + }, + "license_assignments_tier_id_access_tiers_id_fk": { + "name": "license_assignments_tier_id_access_tiers_id_fk", + "tableFrom": "license_assignments", + "tableTo": "access_tiers", + "columnsFrom": [ + "tier_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sync_events": { + "name": "sync_events", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "source_type": { + "name": "source_type", + "type": "sync_source_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "operation_type": { + "name": "operation_type", + "type": "sync_operation_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'regular'" + }, + "backfill_start_date": { + "name": "backfill_start_date", + "type": "date", + "primaryKey": false, + "notNull": false + }, + "outcome": { + "name": "outcome", + "type": "sync_outcome", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'in_progress'" + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "triggered_by": { + "name": "triggered_by", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "created_count": { + "name": "created_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "updated_count": { + "name": "updated_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "skipped_count": { + "name": "skipped_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "error_count": { + "name": "error_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "sync_events_source_type_idx": { + "name": "sync_events_source_type_idx", + "columns": [ + { + "expression": "source_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "sync_events_outcome_idx": { + "name": "sync_events_outcome_idx", + "columns": [ + { + "expression": "outcome", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "sync_events_started_at_idx": { + "name": "sync_events_started_at_idx", + "columns": [ + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "sync_events_source_started_idx": { + "name": "sync_events_source_started_idx", + "columns": [ + { + "expression": "source_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "sync_events_triggered_by_users_id_fk": { + "name": "sync_events_triggered_by_users_id_fk", + "tableFrom": "sync_events", + "tableTo": "users", + "columnsFrom": [ + "triggered_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sync_sources": { + "name": "sync_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "source_type": { + "name": "source_type", + "type": "sync_source_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "cron_schedule": { + "name": "cron_schedule", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "sync_sources_source_type_idx": { + "name": "sync_sources_source_type_idx", + "columns": [ + { + "expression": "source_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.users": { + "name": "users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "email": { + "name": "email", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "password_hash": { + "name": "password_hash", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "github_username": { + "name": "github_username", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "circle": { + "name": "circle", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "role": { + "name": "role", + "type": "user_role", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'viewer'" + }, + "status": { + "name": "status", + "type": "user_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "preferences": { + "name": "preferences", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'{\"theme\":\"system\"}'::jsonb" + }, + "profile": { + "name": "profile", + "type": "user_profile", + "typeSchema": "public", + "primaryKey": false, + "notNull": false + }, + "must_change_password": { + "name": "must_change_password", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "is_agent": { + "name": "is_agent", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "users_email_idx": { + "name": "users_email_idx", + "columns": [ + { + "expression": "email", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "users_circle_idx": { + "name": "users_circle_idx", + "columns": [ + { + "expression": "circle", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "users_status_idx": { + "name": "users_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.assignment_status": { + "name": "assignment_status", + "schema": "public", + "values": [ + "active", + "inactive" + ] + }, + "public.budget_status": { + "name": "budget_status", + "schema": "public", + "values": [ + "active", + "archived" + ] + }, + "public.change_type": { + "name": "change_type", + "schema": "public", + "values": [ + "created", + "updated", + "deleted", + "status_change" + ] + }, + "public.copilot_sync_type": { + "name": "copilot_sync_type", + "schema": "public", + "values": [ + "members", + "copilot" + ] + }, + "public.filter_field": { + "name": "filter_field", + "schema": "public", + "values": [ + "vendor", + "invoice_number" + ] + }, + "public.filter_mode": { + "name": "filter_mode", + "schema": "public", + "values": [ + "whitelist", + "blacklist" + ] + }, + "public.github_connection_status": { + "name": "github_connection_status", + "schema": "public", + "values": [ + "active", + "disconnected" + ] + }, + "public.github_sync_status": { + "name": "github_sync_status", + "schema": "public", + "values": [ + "in_progress", + "completed", + "partial", + "failed" + ] + }, + "public.ingestion_channel": { + "name": "ingestion_channel", + "schema": "public", + "values": [ + "manual", + "api", + "bulk" + ] + }, + "public.ingestion_outcome": { + "name": "ingestion_outcome", + "schema": "public", + "values": [ + "success", + "failed", + "filtered" + ] + }, + "public.invite_token_status": { + "name": "invite_token_status", + "schema": "public", + "values": [ + "active", + "consumed", + "invalidated" + ] + }, + "public.period_type": { + "name": "period_type", + "schema": "public", + "values": [ + "monthly", + "quarterly" + ] + }, + "public.sync_operation_type": { + "name": "sync_operation_type", + "schema": "public", + "values": [ + "regular", + "backfill" + ] + }, + "public.sync_outcome": { + "name": "sync_outcome", + "schema": "public", + "values": [ + "in_progress", + "success", + "partial", + "failed" + ] + }, + "public.sync_source_type": { + "name": "sync_source_type", + "schema": "public", + "values": [ + "github_copilot_billing", + "anthropic_api_usage", + "anthropic_team_invoices", + "github_members", + "invoice_period_matching", + "anthropic_api_costs" + ] + }, + "public.tool_status": { + "name": "tool_status", + "schema": "public", + "values": [ + "active", + "archived" + ] + }, + "public.user_profile": { + "name": "user_profile", + "schema": "public", + "values": [ + "boost", + "maxed", + "indie" + ] + }, + "public.user_role": { + "name": "user_role", + "schema": "public", + "values": [ + "admin", + "viewer" + ] + }, + "public.user_status": { + "name": "user_status", + "schema": "public", + "values": [ + "active", + "inactive" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} \ No newline at end of file diff --git a/src/lib/db/migrations/meta/_journal.json b/src/lib/db/migrations/meta/_journal.json index 9150a27..f3d8440 100644 --- a/src/lib/db/migrations/meta/_journal.json +++ b/src/lib/db/migrations/meta/_journal.json @@ -141,6 +141,13 @@ "when": 1779370363541, "tag": "0019_bouncy_scourge", "breakpoints": true + }, + { + "idx": 20, + "version": "7", + "when": 1779433030575, + "tag": "0020_daily_rachel_grey", + "breakpoints": true } ] } \ No newline at end of file diff --git a/src/lib/db/schema.ts b/src/lib/db/schema.ts index 208648f..b9f9981 100644 --- a/src/lib/db/schema.ts +++ b/src/lib/db/schema.ts @@ -482,10 +482,17 @@ export const copilotUsageMetrics = pgTable( totalLinesAccepted: integer("total_lines_accepted").notNull(), totalChatTurns: integer("total_chat_turns"), totalChatAcceptances: integer("total_chat_acceptances"), + // Deprecated 2026-04-02: GitHub removed the dotcom-chat and PR-summary counters + // from the new Copilot usage metrics API. Columns retained for historical rows. totalDotcomChatTurns: integer("total_dotcom_chat_turns"), totalPrSummaries: integer("total_pr_summaries"), languageBreakdown: jsonb("language_breakdown"), editorBreakdown: jsonb("editor_breakdown"), + // Added 2026-05-21 for new Copilot usage metrics API. See spec 031. + usedCli: boolean("used_cli"), + usedAgent: boolean("used_agent"), + agentEditCount: integer("agent_edit_count"), + cliBreakdown: jsonb("cli_breakdown"), createdAt: timestamp("created_at").notNull().defaultNow(), }, (table) => [ diff --git a/src/lib/sync/sources/github-copilot.ts b/src/lib/sync/sources/github-copilot.ts index 3d02c8e..6e42f66 100644 --- a/src/lib/sync/sources/github-copilot.ts +++ b/src/lib/sync/sources/github-copilot.ts @@ -88,7 +88,9 @@ export async function run( // Sync usage metrics try { - const metricsResult = await syncUsageMetrics(syncConnection, token); + const metricsResult = await syncUsageMetrics(syncConnection, token, { + backfillStartDate: opts?.backfillStartDate, + }); counts.updatedCount += metricsResult.metricsProcessed; } catch (err) { errors.push( diff --git a/tests/unit/sync/copilot-metrics-mapping.test.ts b/tests/unit/sync/copilot-metrics-mapping.test.ts new file mode 100644 index 0000000..4768fbd --- /dev/null +++ b/tests/unit/sync/copilot-metrics-mapping.test.ts @@ -0,0 +1,102 @@ +import { describe, it, expect } from "vitest"; +import { mapNdjsonRowToDbRow } from "@/lib/copilot-sync"; +import type { CopilotMetricsRow } from "@/lib/copilot-api"; + +describe("mapNdjsonRowToDbRow", () => { + const fullRow: CopilotMetricsRow = { + day: "2026-05-18", + organization_id: 12345, + code_generation_activity_count: 4200, + code_acceptance_activity_count: 1800, + loc_suggested_to_add_sum: 32000, + loc_added_sum: 14500, + chat_panel_ask_mode: 320, + chat_panel_agent_mode: 95, + chat_panel_edit_mode: 40, + chat_panel_plan_mode: 12, + chat_panel_custom_mode: 3, + chat_panel_unknown_mode: 0, + agent_edit: 18, + totals_by_ide: { vscode: { completions: 3000 } }, + totals_by_language_feature: { + "TypeScript|code_completion": { suggestions: 1200, acceptances: 480 }, + }, + totals_by_cli: { + session_count: 12, + request_count: 84, + prompt_count: 84, + }, + }; + + it("maps activity counters straight across", () => { + const out = mapNdjsonRowToDbRow(1, "2026-05-18", fullRow, { + active: 32, + engaged: 24, + }); + expect(out.connectionId).toBe(1); + expect(out.date).toBe("2026-05-18"); + expect(out.totalSuggestions).toBe(4200); + expect(out.totalAcceptances).toBe(1800); + expect(out.totalLinesSuggested).toBe(32000); + expect(out.totalLinesAccepted).toBe(14500); + expect(out.totalActiveUsers).toBe(32); + expect(out.totalEngagedUsers).toBe(24); + }); + + it("sums chat-panel mode counters into totalChatTurns", () => { + const out = mapNdjsonRowToDbRow(1, "2026-05-18", fullRow, { + active: 0, + engaged: 0, + }); + // 320 + 95 + 40 + 12 + 3 + 0 = 470 + expect(out.totalChatTurns).toBe(470); + }); + + it("derives usedCli + usedAgent + agentEditCount from new fields", () => { + const out = mapNdjsonRowToDbRow(1, "2026-05-18", fullRow, { + active: 0, + engaged: 0, + }); + expect(out.usedCli).toBe(true); + expect(out.usedAgent).toBe(true); + expect(out.agentEditCount).toBe(18); + expect(out.cliBreakdown).toEqual(fullRow.totals_by_cli); + }); + + it("writes null for deprecated dotcom-chat and PR-summary columns", () => { + const out = mapNdjsonRowToDbRow(1, "2026-05-18", fullRow, { + active: 0, + engaged: 0, + }); + expect(out.totalDotcomChatTurns).toBeNull(); + expect(out.totalPrSummaries).toBeNull(); + }); + + it("forwards JSONB breakdown shapes verbatim", () => { + const out = mapNdjsonRowToDbRow(1, "2026-05-18", fullRow, { + active: 0, + engaged: 0, + }); + expect(out.editorBreakdown).toEqual(fullRow.totals_by_ide); + expect(out.languageBreakdown).toEqual(fullRow.totals_by_language_feature); + }); + + it("returns 0 for missing activity fields and null chat when no mode keys present", () => { + const sparse: CopilotMetricsRow = { day: "2026-05-18" }; + const out = mapNdjsonRowToDbRow(7, "2026-05-18", sparse, { + active: 0, + engaged: 0, + }); + expect(out.totalSuggestions).toBe(0); + expect(out.totalAcceptances).toBe(0); + expect(out.totalLinesSuggested).toBe(0); + expect(out.totalLinesAccepted).toBe(0); + expect(out.totalChatTurns).toBeNull(); + expect(out.usedCli).toBe(false); + expect(out.usedAgent).toBe(false); + expect(out.agentEditCount).toBeNull(); + expect(out.cliBreakdown).toBeNull(); + expect(out.editorBreakdown).toBeNull(); + expect(out.languageBreakdown).toBeNull(); + }); +});