Skip to content

Test isolation: migration 4→5 seeds extraction tables from ~/.claude/MEMORY/*.json on every fresh DB #29

@edheltzel

Description

@edheltzel

What happened

Two tests started failing after ~/.claude/MEMORY/SESSION_INDEX.json accumulated real session data during a long Claude Code session:

  • tests/hooks/sqlite-writers.test.ts > writeExtractionSession > upserts on duplicate session_id — expected count=1, got 4
  • tests/hooks/extraction-parsers.test.ts > dualWriteToSqlite > populates all six surfaces from one fixture

Root cause: migration 4→5 in src/db/migrations.ts:48 reads ~/.claude/MEMORY/.extraction_tracker.json, SESSION_INDEX.json, and ERROR_PATTERNS.json and inserts their contents into the extraction_* tables on every initDb() run. Since setupTestDb() calls initDb() on a fresh tempdir DB, the migration always seeds it from the user's real ~/.claude/MEMORY/. Tests that insert 1-2 rows and then count get N + machine_history rows, not 1-2.

This was masked until session data accumulated on the dev machine — first day on a fresh install, tests pass; later, they don't.

What we've already done

Patched in c46cc90: tests/helpers/setup.ts now scrubs the extraction_* tables immediately after initDb() so tests start truly empty. Fix verified — 402/402 pass.

What this issue tracks

The scrub is a workaround. The deeper problem is that a schema migration silently reaches into the user's filesystem and copies data into the DB. That's surprising:

  1. It runs on every fresh DB, not just upgrades from schema v4.
  2. It tightly couples the DB initialization to the user's ~/.claude/MEMORY/ state.
  3. Tests can't opt out without explicit cleanup.

Proposals:

  • Gate migration 4→5 on PRAGMA user_version so it only runs once per DB, not every initDb().
  • Add an env hook (RECALL_SKIP_LEGACY_DATA_MIGRATIONS=1) that tests/CI/sandboxes can set to skip the JSON ingestion entirely.
  • Long-term: migration 4→5 is a one-shot legacy import. Once we're confident nobody's still on pre-v5 schema, this whole block can be removed.

Related:

  • src/db/migrations.ts:44-110 (migration 4→5)
  • tests/helpers/setup.ts (current workaround)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions