ISS-141 - feat: overlay system — merge remote base graph with local branch diffs

[patelatharva]

feat: overlay system — merge remote base graph with local branch diffs

Summary

Adds an overlay architecture that lets teams share a single pre-built base graph (main/development branch, built once by CI) while each developer only indexes their feature-branch diffs locally. LLM queries see a seamless, merged view of both databases — no full re-index needed per developer.

Existing local-only CodeGraph.init() / open() behavior is completely unaffected; the overlay is opt-in via a new CodeGraph.openOverlay() static method.

Motivation

Full-repo indexing is expensive. On large codebases every developer re-indexes the same unchanged files. The overlay system eliminates that redundancy:

1. CI builds the base graph once for main/development and publishes the .db file.
2. Developers download it (file://, HTTP, or shared filesystem) with TTL-based caching.
3. Only feature-branch diffs are indexed locally — typically a handful of files.
4. Queries merge both layers transparently — the LLM gets a complete, up-to-date graph.

This pull request addresses this issue:
#141

Architecture

┌──────────────────────────────────────────────────┐
│                CodeGraph.openOverlay()            │
├──────────────────────────────────────────────────┤
│  RemoteGraphClient   →  fetch & cache base .db   │
│  BranchDiffIndexer   →  git diff --name-status   │
│  OverlayQueryEngine  →  merged reads, local writes│
│         extends QueryBuilder (drop-in)           │
└──────────────────────────────────────────────────┘

Merge semantics in OverlayQueryEngine:

• Nodes/edges in overlay files → served from overlay DB
• Nodes/edges in deleted files → hidden entirely
• Everything else → served from base DB
• Search results → merged from both, overlay wins on conflict
• Stale base edges (referencing overlay/deleted files) → filtered out
• Cross-boundary edges (overlay → base) work seamlessly

Changes

File	Status	Lines	Description
src/overlay/types.ts	Added	72	RemoteGraphConfig and BranchDiffResult interfaces
src/overlay/remote-client.ts	Added	198	RemoteGraphClient — fetch, TTL cache, open base graph
src/overlay/branch-diff.ts	Added	144	BranchDiffIndexer — git merge-base + diff detection
src/overlay/overlay-engine.ts	Added	543	OverlayQueryEngine extends QueryBuilder — merged reads
src/overlay/index.ts	Added	12	Barrel re-exports
src/index.ts	Modified	+123	openOverlay(), remoteClient cleanup, re-exports
__tests__/overlay.test.ts	Added	1092	74 tests across 5 suites
package-lock.json	Modified	-1	Lockfile churn

Total: 8 files changed, +2,183 lines, -2 lines

New Public API

// Open in overlay mode — replaces the full-index workflow for team devs
const cg = await CodeGraph.openOverlay('/path/to/repo', {
  url: 'https://ci.example.com/codegraph/main.db', // or file:// path
  baseBranch: 'main',
  cacheTTL: 3_600_000,   // optional, default 1h
  cacheDir: '.codegraph', // optional
});

// All existing queries work identically — backed by merged overlay engine
const results = cg.searchNodes('AuthService');
const callers = cg.getCallers('some-node-id');
const context = cg.buildContext({ query: 'login flow' });

cg.close(); // cleans up both overlay DB and remote client

Key Design Decisions

1. OverlayQueryEngine extends QueryBuilder — drop-in compatible with all existing consumers (GraphTraverser, ContextBuilder, ReferenceResolver, MCP tools). No changes needed downstream.

2. FK constraints disabled on overlay DB — overlay edges legitimately reference nodes in the base DB (a separate SQLite file). PRAGMA foreign_keys = OFF is scoped only to the overlay connection.

3. Merge-base diffing — uses git merge-base + git diff --name-status so only the developer's actual changes appear, regardless of upstream commits since the branch point. Handles renamed files (R status) as add + delete.

4. Write isolation — all write operations (insertNode, insertEdge, indexFile, etc.) go to the overlay DB only. The base DB is read-only.

5. Stale edge filtering — base edges that reference overlay or deleted files are detected and excluded from results, preventing ghost references.

Test Coverage

74 new tests across 5 suites, 0 regressions (811 total pass):

Suite	Tests	Coverage
BranchDiffIndexer	4	Added/modified/deleted detection, empty diffs, renamed files
RemoteGraphClient	9	file:// fetch, TTL caching, cache expiry, error paths, close
OverlayQueryEngine	47	Node routing, stale filtering, deleted masking, edge merging, search dedup, file ops, stats, write isolation, cache clearing
GraphTraverser integration	5	Cross-boundary traversal, overlay→base callers, stale caller hiding
Search methods	9	findNodesByExactName, findNodesByNameSubstring, searchNodes merging

Tests use real SQLite databases and real git repos (created via mkdtempSync + git init), matching the project's existing test conventions. No mocking.

How to Test

# Run only overlay tests
npx vitest run __tests__/overlay.test.ts

# Run full suite to confirm no regressions
npm test

Backward Compatibility

• No breaking changes. All existing CodeGraph.init(), open(), initSync(), openSync() paths are untouched.
• The overlay module is purely additive — new files in src/overlay/, new static method on CodeGraph, new re-exports.
• The only modification to existing code is src/index.ts (+123 lines for imports, re-exports, the openOverlay() method, and close() cleanup).