Skip to content

fix(insights): add missing lifecycle column compatibility check#43

Merged
gsxdsm merged 5 commits intoRunfusion:mainfrom
timothyjlaurent:timothyjlaurent/broken-insights
May 5, 2026
Merged

fix(insights): add missing lifecycle column compatibility check#43
gsxdsm merged 5 commits intoRunfusion:mainfrom
timothyjlaurent:timothyjlaurent/broken-insights

Conversation

@timothyjlaurent
Copy link
Copy Markdown
Contributor

@timothyjlaurent timothyjlaurent commented May 5, 2026

Summary

  • Adds ensureInsightRunsSchemaCompatibility() to fix "table project_insight_runs has no column named lifecycle" error when clicking Generate Insights
  • Follows the existing ensureRoutinesSchemaCompatibility() pattern — idempotent addColumnIfMissing calls that run unconditionally on every init()

Root Cause

The lifecycle and cancelledAt columns were retroactively added to migration v33 and as a safety-net in the version < 59 block. Databases already at v59+ never re-ran that migration, leaving the columns missing.

Test plan

  • All 117 insight-related tests pass
  • Verified central-integration test failure is pre-existing flake
  • Manual: open existing database, click "Generate Insights", confirm no error

Fixes #42

Summary by CodeRabbit

  • Bug Fixes

    • Database initialization now applies a compatibility backfill so older installations get missing schema pieces and indexes automatically, improving data consistency and preventing compatibility issues.
    • Improved handling of legacy database states to ensure features depending on newer columns work correctly.
  • Tests

    • Added migration tests that validate schema restoration and runtime behavior for legacy databases.

…ifecycle column

Databases already at schema v59+ when the lifecycle column was added never
re-ran the v59 migration, causing "no column named lifecycle" errors on
insight generation. Adds an unconditional compatibility check following the
existing ensureRoutinesSchemaCompatibility pattern.

Fixes Runfusion#42

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 5, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds an idempotent compatibility backfill, ensureInsightRunsSchemaCompatibility(), invoked during Database.init() to add missing lifecycle and cancelledAt columns and ensure idxInsightRunsProjectTriggerStatus exists on project_insight_runs for databases that missed earlier migrations. A test verifies the backfill restores columns and that InsightStore.createRun() works.

Changes

Schema Compatibility Backfill

Layer / File(s) Summary
Init hook
packages/core/src/db.ts
Call to this.ensureInsightRunsSchemaCompatibility() added to Database.init() after routine backfills.
Core Implementation / Compatibility
packages/core/src/db.ts
New private method ensureInsightRunsSchemaCompatibility() checks for project_insight_runs, idempotently adds lifecycle (TEXT) and cancelledAt (TEXT) columns if missing, and creates idxInsightRunsProjectTriggerStatus index.
Tests
packages/core/src/__tests__/insight-store.test.ts
New test "ensureInsightRunsSchemaCompatibility adds lifecycle column to legacy table": simulates legacy table lacking columns, runs init, asserts columns are present, and verifies InsightStore.createRun() produces a run with lifecycle defined.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

  • #42: Fix: project_insight_runs table missing lifecycle column — Implements the same compatibility backfill pattern to add missing lifecycle/cancelledAt columns (direct match).
  • Fix: project_insight_runs table missing lifecycle column #41 — Describes the same missing-column problem and proposes adding the same compatibility backfill (closely related).

Poem

🐰
I nibbled through the schema soil,
Found columns lost in legacy soil.
A gentle nudge at init's gate,
Restored the fields, now all is straight.
Hooray — migrations celebrate!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: adding a compatibility check for missing lifecycle column in the insights schema.
Linked Issues check ✅ Passed The PR implementation fully addresses issue #42 by adding ensureInsightRunsSchemaCompatibility() that runs unconditionally on init() with idempotent addColumnIfMissing calls, matching the described fix pattern.
Out of Scope Changes check ✅ Passed All changes are directly related to addressing the missing lifecycle column issue: the db.ts changes implement the compatibility backfill, and the test changes verify the fix works correctly.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 5, 2026

Greptile Summary

This PR fixes a "table project_insight_runs has no column named lifecycle" crash by adding ensureInsightRunsSchemaCompatibility(), which unconditionally backfills the lifecycle and cancelledAt columns (plus their index) on every init(). This mirrors the pre-existing ensureRoutinesSchemaCompatibility() pattern for the same class of schema-drift problem.

  • db.ts: New private method ensureInsightRunsSchemaCompatibility() guards on table existence then calls addColumnIfMissing for both missing columns and idempotently recreates idxInsightRunsProjectTriggerStatus.
  • insight-store.test.ts: End-to-end integration test reconstructs the affected legacy-DB state by dropping and recreating the table without the new columns, then re-opens the DB and confirms the columns are present and a full createRun round-trip succeeds.

Confidence Score: 5/5

Safe to merge — the change is a narrow, idempotent backfill that cannot regress databases which already have the columns, and the new test proves the fix works end-to-end.

Both addColumnIfMissing and CREATE INDEX IF NOT EXISTS are no-ops when the column or index already exists, so fresh databases and already-fixed databases are unaffected. The implementation matches the established compatibility-fix pattern used for routines, the column types are consistent with SCHEMA_SQL, and the integration test accurately simulates the real-world failure scenario.

No files require special attention.

Important Files Changed

Filename Overview
packages/core/src/db.ts Adds ensureInsightRunsSchemaCompatibility() following the established ensureRoutinesSchemaCompatibility() pattern — idempotent addColumnIfMissing calls for lifecycle and cancelledAt plus a CREATE INDEX IF NOT EXISTS guard, invoked after migrate() on every init().
packages/core/src/tests/insight-store.test.ts Adds an integration test that simulates a pre-fix database by stripping the lifecycle/cancelledAt columns, reopens the database, and asserts the compatibility method restores them and allows a successful createRun call. Schema-version assertion correctly uses 62.

Sequence Diagram

sequenceDiagram
    participant App
    participant Database
    participant SQLite

    App->>Database: init()
    Database->>SQLite: run migrations (migrate())
    Note over Database,SQLite: version < SCHEMA_VERSION → apply missing migrations
    Database->>Database: ensureRoutinesSchemaCompatibility()
    Database->>Database: ensureInsightRunsSchemaCompatibility()
    activate Database
    Database->>SQLite: hasTable("project_insight_runs")?
    SQLite-->>Database: true / false
    alt table exists
        Database->>SQLite: addColumnIfMissing("lifecycle", "TEXT")
        Database->>SQLite: addColumnIfMissing("cancelledAt", "TEXT")
        Database->>SQLite: CREATE INDEX IF NOT EXISTS idxInsightRunsProjectTriggerStatus
    end
    deactivate Database
    Database-->>App: ready
Loading

Reviews (5): Last reviewed commit: "Update packages/core/src/__tests__/insig..." | Re-trigger Greptile

timothyjlaurent and others added 2 commits May 5, 2026 12:51
…ibility

Simulates a database where project_insight_runs exists without lifecycle
and cancelledAt columns (the state that caused the original bug), then
verifies that re-opening the database adds the columns and creating a run
succeeds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-existing lint error on main — the `any` cast in __setCreateFnAgent
mirrors the eslint-disable already on the adjacent variable declarations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/core/src/__tests__/insight-store.test.ts (1)

970-1015: ⚡ Quick win

Always close DB handles via finally to avoid flaky cleanup on failed assertions.

If an assertion fails before the explicit close() calls, open SQLite handles can leak and make directory cleanup flaky. Wrapping each opened DB in try/finally keeps this deterministic.

Suggested refactor
-      const db1 = createDatabase(compatDir);
-      db1.init();
-      expect(db1.getSchemaVersion()).toBe(61);
+      const db1 = createDatabase(compatDir);
+      try {
+        db1.init();
+        expect(db1.getSchemaVersion()).toBe(61);
 
-      // Step 2: Strip lifecycle and cancelledAt columns by recreating the
-      // table without them. This simulates a DB that was created before the
-      // lifecycle columns were added and already past v59 when they landed.
-      db1.exec(`
-        CREATE TABLE project_insight_runs_legacy AS
-          SELECT id, projectId, trigger, status, summary, error,
-                 insightsCreated, insightsUpdated, inputMetadata, outputMetadata,
-                 createdAt, startedAt, completedAt
-          FROM project_insight_runs
-      `);
-      db1.exec("DROP TABLE project_insight_runs");
-      db1.exec("ALTER TABLE project_insight_runs_legacy RENAME TO project_insight_runs");
+        // Step 2: Strip lifecycle and cancelledAt columns by recreating the
+        // table without them. This simulates a DB that was created before the
+        // lifecycle columns were added and already past v59 when they landed.
+        db1.exec(`
+          CREATE TABLE project_insight_runs_legacy AS
+            SELECT id, projectId, trigger, status, summary, error,
+                   insightsCreated, insightsUpdated, inputMetadata, outputMetadata,
+                   createdAt, startedAt, completedAt
+            FROM project_insight_runs
+        `);
+        db1.exec("DROP TABLE project_insight_runs");
+        db1.exec("ALTER TABLE project_insight_runs_legacy RENAME TO project_insight_runs");
 
-      // Verify lifecycle column is gone
-      const colsBefore = db1.prepare("PRAGMA table_info(project_insight_runs)").all() as Array<{ name: string }>;
-      const colNamesBefore = colsBefore.map((c) => c.name);
-      expect(colNamesBefore).not.toContain("lifecycle");
-      expect(colNamesBefore).not.toContain("cancelledAt");
-      db1.close();
+        // Verify lifecycle column is gone
+        const colsBefore = db1.prepare("PRAGMA table_info(project_insight_runs)").all() as Array<{ name: string }>;
+        const colNamesBefore = colsBefore.map((c) => c.name);
+        expect(colNamesBefore).not.toContain("lifecycle");
+        expect(colNamesBefore).not.toContain("cancelledAt");
+      } finally {
+        db1.close();
+      }
 
       // Step 3: Re-open — ensureInsightRunsSchemaCompatibility should add the
       // missing columns unconditionally.
       const db2 = createDatabase(compatDir);
-      db2.init();
+      try {
+        db2.init();
 
-      const colsAfter = db2.prepare("PRAGMA table_info(project_insight_runs)").all() as Array<{ name: string }>;
-      const colNamesAfter = colsAfter.map((c) => c.name);
-      expect(colNamesAfter).toContain("lifecycle");
-      expect(colNamesAfter).toContain("cancelledAt");
+        const colsAfter = db2.prepare("PRAGMA table_info(project_insight_runs)").all() as Array<{ name: string }>;
+        const colNamesAfter = colsAfter.map((c) => c.name);
+        expect(colNamesAfter).toContain("lifecycle");
+        expect(colNamesAfter).toContain("cancelledAt");
 
-      // Step 4: Creating a run must not throw — proves the INSERT path works
-      // with the restored columns.
-      const s = new InsightStore(db2);
-      const run = s.createRun("proj", { trigger: "manual" });
-      expect(run.id).toBeTruthy();
-      expect(run.lifecycle).toBeDefined();
-
-      db2.close();
+        // Step 4: Creating a run must not throw — proves the INSERT path works
+        // with the restored columns.
+        const s = new InsightStore(db2);
+        const run = s.createRun("proj", { trigger: "manual" });
+        expect(run.id).toBeTruthy();
+        expect(run.lifecycle).toBeDefined();
+      } finally {
+        db2.close();
+      }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/__tests__/insight-store.test.ts` around lines 970 - 1015,
The test opens two DB handles (created via createDatabase and referenced as db1
and db2) but only calls close() directly, which can leak handles if assertions
throw; wrap each DB lifecycle in its own try/finally (or at least ensure db1 and
db2 are closed in a finally) so that after db1.init() and after db2.init() you
always call db1.close() and db2.close() respectively even on failure—adjust the
test around the blocks that use db1 and db2 (and the InsightStore usage) to
guarantee deterministic cleanup.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/core/src/__tests__/insight-store.test.ts`:
- Around line 970-1015: The test opens two DB handles (created via
createDatabase and referenced as db1 and db2) but only calls close() directly,
which can leak handles if assertions throw; wrap each DB lifecycle in its own
try/finally (or at least ensure db1 and db2 are closed in a finally) so that
after db1.init() and after db2.init() you always call db1.close() and
db2.close() respectively even on failure—adjust the test around the blocks that
use db1 and db2 (and the InsightStore usage) to guarantee deterministic cleanup.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0e626c19-00bd-4276-bd0b-28c4fa100637

📥 Commits

Reviewing files that changed from the base of the PR and between 46c6b7a and b0c1084.

📒 Files selected for processing (1)
  • packages/core/src/__tests__/insight-store.test.ts

Comment thread packages/core/src/db.ts
Comment on lines +1070 to +1078
private ensureInsightRunsSchemaCompatibility(): void {
if (!this.hasTable("project_insight_runs")) {
return;
}

this.addColumnIfMissing("project_insight_runs", "lifecycle", "TEXT");
this.addColumnIfMissing("project_insight_runs", "cancelledAt", "TEXT");
this.db.exec(`CREATE INDEX IF NOT EXISTS idxInsightRunsProjectTriggerStatus ON project_insight_runs(projectId, trigger, status)`);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Missing changeset for user-facing bug fix

AGENTS.md requires a changeset file for any change that affects @runfusion/fusion — including bug fixes. Because @fusion/core is bundled into the published CLI via noExternal, this fix to db.ts ships to end-users and needs a patch changeset. None of the three commits in this PR include one. A missing changeset means the next pnpm release will silently skip version bumping and changelog entries for this fix.

Context Used: AGENTS.md (source)

@gsxdsm
Copy link
Copy Markdown
Collaborator

gsxdsm commented May 5, 2026

@copilot resolve the merge conflicts in this pull request

Comment thread packages/core/src/__tests__/insight-store.test.ts Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/src/__tests__/insight-store.test.ts`:
- Line 974: The test assertion uses the wrong expected schema version; update
the expectation on db1.getSchemaVersion() from 61 to 62 (or better, import and
use the SCHEMA_VERSION constant from db.ts) so it matches the actual
SCHEMA_VERSION used by the database; locate the assertion on
db1.getSchemaVersion() in the test and change the expected value to 62 (or
replace the literal with the SCHEMA_VERSION symbol) to make it consistent with
other tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 283142d9-4076-476c-91d3-60a2abad55d6

📥 Commits

Reviewing files that changed from the base of the PR and between 06cbb74 and 4677612.

📒 Files selected for processing (2)
  • packages/core/src/__tests__/insight-store.test.ts
  • packages/core/src/db.ts

Comment thread packages/core/src/__tests__/insight-store.test.ts Outdated
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/core/src/__tests__/insight-store.test.ts (1)

1001-1004: ⚡ Quick win

Cover the index backfill in this compatibility test.

ensureInsightRunsSchemaCompatibility() also recreates idxInsightRunsProjectTriggerStatus, but this test only asserts the restored columns and insert path. A regression in the index creation would still pass here.

♻️ Suggested assertion
       const colsAfter = db2.prepare("PRAGMA table_info(project_insight_runs)").all() as Array<{ name: string }>;
       const colNamesAfter = colsAfter.map((c) => c.name);
       expect(colNamesAfter).toContain("lifecycle");
       expect(colNamesAfter).toContain("cancelledAt");
+
+      const indexesAfter = db2.prepare(
+        "SELECT name FROM sqlite_master WHERE type = 'index' AND tbl_name = 'project_insight_runs'"
+      ).all() as Array<{ name: string }>;
+      expect(indexesAfter.map((i) => i.name)).toContain("idxInsightRunsProjectTriggerStatus");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/__tests__/insight-store.test.ts` around lines 1001 - 1004,
The test currently checks restored columns but not the recreated index; update
the test that calls ensureInsightRunsSchemaCompatibility() to also verify the
index idxInsightRunsProjectTriggerStatus exists and covers the expected columns:
after the migration query PRAGMA index_list(project_insight_runs) and assert the
returned index names include "idxInsightRunsProjectTriggerStatus", then query
PRAGMA index_info('idxInsightRunsProjectTriggerStatus') and assert its column
sequence contains "project", "trigger", and "status" to fully cover the index
backfill path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/core/src/__tests__/insight-store.test.ts`:
- Around line 1001-1004: The test currently checks restored columns but not the
recreated index; update the test that calls
ensureInsightRunsSchemaCompatibility() to also verify the index
idxInsightRunsProjectTriggerStatus exists and covers the expected columns: after
the migration query PRAGMA index_list(project_insight_runs) and assert the
returned index names include "idxInsightRunsProjectTriggerStatus", then query
PRAGMA index_info('idxInsightRunsProjectTriggerStatus') and assert its column
sequence contains "project", "trigger", and "status" to fully cover the index
backfill path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 12c53b62-1c07-4acf-a6f0-0fcc183161c7

📥 Commits

Reviewing files that changed from the base of the PR and between 4677612 and ca432fe.

📒 Files selected for processing (1)
  • packages/core/src/__tests__/insight-store.test.ts

@gsxdsm gsxdsm merged commit 2e29d44 into Runfusion:main May 5, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix: project_insight_runs table missing lifecycle column

2 participants