diff --git a/docs/spec-auto-run-on-test-change.md b/docs/spec-auto-run-on-test-change.md new file mode 100644 index 0000000..dfe42cc --- /dev/null +++ b/docs/spec-auto-run-on-test-change.md @@ -0,0 +1,105 @@ +# Spec: Auto-Run Coverage on Test File Save + +## Objective + +When a developer writes or modifies a test file, the coverage dashboard and status bar should +automatically update with fresh numbers — without the user having to manually run pytest or +trigger any command. + +**User:** Python developer using VS Code with coverage-visualizer installed. They are actively +writing tests and want immediate feedback on whether their new test improves coverage. + +**Problem today:** User saves `test_calculator.py` → nothing happens → they manually run pytest +→ coverage files update → dashboard refreshes. The gap between save and updated numbers is +entirely manual friction. + +**Success criteria:** +- User saves any test file → pytest runs in the background within 2s → dashboard shows updated % +- Status bar shows a spinner while pytest is running so the user knows something is happening +- If pytest fails, a warning notification appears (reusing existing behaviour from `handleNoCoverage`) +- A setting `coverageVisualizer.autoRunOnTestChange` (boolean, default `true`) lets users with + slow test suites opt out +- Behaviour is additive — existing manual `coverage-visualizer.show` command is unchanged + +## Tech Stack + +TypeScript, VS Code Extension API 1.90+, Node.js `child_process.spawn` + +## Commands + +```bash +npm run compile # compile TypeScript → out/ +npm test # run Jest unit tests +npm run lint # ESLint on src/ +``` + +## Project Structure + +``` +src/ + extension.ts ← file watcher setup lives here (setupWatchers) + config.ts ← add autoRunOnTestChange setting here + ui/ + statusBar.ts ← add spinner state here +package.json ← add new configuration property +``` + +## Code Style + +Follow existing patterns exactly. New watchers go inside `setupWatchers()`. New config keys go +in `getConfig()`. No new files unless unavoidable. + +```typescript +// Good — matches existing watcher pattern +const testWatcher = vscode.workspace.createFileSystemWatcher( + new vscode.RelativePattern(root, '{test_*.py,*_test.py,tests/**/*.py,test/**/*.py}') +); +testWatcher.onDidChange(debouncedTestRun); +testWatcher.onDidCreate(debouncedTestRun); +context.subscriptions.push(testWatcher); +``` + +## Testing Strategy + +- Unit tests in `tests/` using Jest (existing framework) +- `vscode` is mocked at `tests/__mocks__/vscode.ts` +- New logic to test: debounce behaviour, setting read, watcher glob pattern +- `extension.ts` integration tested manually via F5 + +## Boundaries + +- **Always:** run `npm run compile` before committing, keep changes inside existing functions +- **Ask first:** adding new npm dependencies, changing existing watcher patterns +- **Never:** auto-run on non-test source file changes, run pytest without checking + `autoRunOnTestChange` setting first, break the existing manual `show` command + +## Assumptions + +1. Test files are identified by filename pattern only (`test_*.py`, `*_test.py`) or by living + inside `tests/` or `test/` directories — same logic already used in `isTestFile()` +2. Debounce is 2 seconds after the last save event (not per-save) +3. Auto-run reuses the existing `handleNoCoverage` spawn infrastructure — no new process manager +4. The spinner uses VS Code's existing `$(sync~spin)` ThemeIcon in the status bar +5. We do NOT run only the changed test file — full suite always, for accurate numbers + +## Not Doing (and Why) + +- **Run only the changed test file** — gives partial/misleading coverage numbers; requires + coverage merging complexity; not worth it for v1 +- **pytest-watch / `--watch` mode** — external dependency we don't control; overkill +- **Per-test CodeLens "▶ Run"** — different feature, much larger scope, separate PR +- **Watch non-test source files** — source changes don't add tests; false positives everywhere +- **Auto-install pytest-cov if missing** — out of scope; existing `handleNoCoverage` already + handles missing dependencies + +## Open Questions + +None — direction confirmed in brainstorm session. + +## Success Criteria (testable) + +- [ ] Save `test_foo.py` → within 2s pytest runs → status bar shows spinner → dashboard % updates +- [ ] Set `autoRunOnTestChange: false` → save test file → nothing happens +- [ ] pytest exits non-zero → warning notification shown → coverage not cleared +- [ ] Saving a non-test `.py` file → no auto-run triggered +- [ ] Multiple rapid saves → only one pytest run (debounce working) diff --git a/package-lock.json b/package-lock.json index 45ad781..c639780 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "coverage-visualizer", - "version": "1.0.2", + "version": "1.1.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "coverage-visualizer", - "version": "1.0.2", + "version": "1.1.0", "license": "MIT", "dependencies": { "sql.js": "^1.14.1" diff --git a/package.json b/package.json index 3b8c5e3..0e09430 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "coverage-visualizer", "displayName": "Python Coverage Visualizer", "description": "Visualize Python test coverage inline in VS Code — highlights, CodeLens, dashboard, and sidebar tree view", - "version": "1.0.2", + "version": "1.1.0", "publisher": "kool7", "engines": { "vscode": "^1.90.0" @@ -109,6 +109,11 @@ "type": "boolean", "default": true, "description": "Skip decorations and CodeLens on test files (test_*.py, *_test.py, files inside tests/ directories)." + }, + "coverageVisualizer.autoRunOnTestChange": { + "type": "boolean", + "default": true, + "description": "Automatically re-run pytest and refresh coverage when a test file is saved. Disable if your test suite is too slow to run on every save." } } } diff --git a/src/config.ts b/src/config.ts index dbc06b4..1fda0ee 100644 --- a/src/config.ts +++ b/src/config.ts @@ -9,6 +9,7 @@ export interface Config { enableHoverMessages: boolean; autoReloadOnChange: boolean; excludeTestFiles: boolean; + autoRunOnTestChange: boolean; } export function getConfig(): Config { @@ -22,5 +23,6 @@ export function getConfig(): Config { enableHoverMessages: cfg.get('enableHoverMessages', true), autoReloadOnChange: cfg.get('autoReloadOnChange', true), excludeTestFiles: cfg.get('excludeTestFiles', true), + autoRunOnTestChange: cfg.get('autoRunOnTestChange', true), }; } diff --git a/src/extension.ts b/src/extension.ts index 56424a9..10c52bf 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -11,7 +11,7 @@ import { CoverageReport, RawCoverageJson, } from './parsers/coverageParser.js'; -import { initStatusBar, updateStatusBar, clearStatusBar } from './ui/statusBar.js'; +import { initStatusBar, updateStatusBar, clearStatusBar, showRunningStatusBar } from './ui/statusBar.js'; import { showDashboard, updateDashboard } from './ui/dashboardPanel.js'; import { getConfig } from './config.js'; import { CoverageCodeLensProvider } from './providers/codeLensProvider.js'; @@ -24,13 +24,19 @@ let currentReport: CoverageReport | undefined; let coverageRunInProgress = false; let noCoveragePromptActive = false; let reloadTimer: ReturnType | undefined; +let testChangeTimer: ReturnType | undefined; let coverageOutputChannel: vscode.OutputChannel | undefined; +let runningProc: ReturnType | undefined; +let cachedPytestEnv: { hasPytestCov: boolean; hasCoverage: boolean } | undefined; +let extensionContext: vscode.ExtensionContext | undefined; +let showDashboardPending = false; const codeLensProvider = new CoverageCodeLensProvider(); const hoverProvider = new CoverageHoverProvider(); const treeProvider = new CoverageTreeProvider(); export function activate(context: vscode.ExtensionContext) { + extensionContext = context; createDecorations(); initStatusBar(context); @@ -44,19 +50,16 @@ export function activate(context: vscode.ExtensionContext) { vscode.commands.registerCommand('coverage-visualizer.show', loadAndApply), vscode.commands.registerCommand('coverage-visualizer.clear', clearCoverage), vscode.commands.registerCommand('coverage-visualizer.showDashboard', () => { - if (currentReport) { - showDashboard(currentReport, context); - } else { - loadAndApply().then(() => { - if (currentReport) showDashboard(currentReport, context); - }); - } + if (currentReport) showDashboard(currentReport, context); + showDashboardPending = true; + loadAndApply(); }), vscode.window.onDidChangeActiveTextEditor(editor => { if (editor && currentReport) applyToEditor(editor, currentReport); }), vscode.workspace.onDidChangeConfiguration(e => { if (!e.affectsConfiguration('coverageVisualizer')) return; + cachedPytestEnv = undefined; coveredDecoration.dispose(); uncoveredDecoration.dispose(); createDecorations(); @@ -108,6 +111,19 @@ function setupWatchers(context: vscode.ExtensionContext) { }); context.subscriptions.push(w); }); + + const testWatcher = vscode.workspace.createFileSystemWatcher( + new vscode.RelativePattern(root, '{test_*.py,*_test.py,tests/**/*.py,test/**/*.py,**/conftest.py}') + ); + const debouncedTestRun = () => { + if (!getConfig().autoRunOnTestChange) return; + clearTimeout(testChangeTimer); + testChangeTimer = setTimeout(() => runPytestSilently(root.uri.fsPath), 2000); + }; + testWatcher.onDidChange(debouncedTestRun); + testWatcher.onDidCreate(debouncedTestRun); + testWatcher.onDidDelete(debouncedTestRun); + context.subscriptions.push(testWatcher); } async function loadAndApply() { @@ -144,7 +160,12 @@ async function loadAndApply() { numStatements: filteredTotal, }); - updateDashboard(report); + if (showDashboardPending && extensionContext) { + showDashboard(report, extensionContext); + showDashboardPending = false; + } else { + updateDashboard(report); + } } function resolvePython(workspaceFolder: string): string { @@ -164,6 +185,54 @@ function checkPython(python: string, cwd: string, code: string): Promise coverageOutputChannel!.append(d.toString())); + runningProc.stderr!.on('data', (d: Buffer) => coverageOutputChannel!.append(d.toString())); + runningProc.on('close', code => { + runningProc = undefined; + coverageRunInProgress = false; + coverageOutputChannel!.appendLine(`\n[exited ${code ?? '?'}]`); + if (code !== 0) { + clearStatusBar(); + vscode.window.showWarningMessage('pytest failed — check the Coverage Run output panel.'); + } else { + loadAndApply(); + } + }); +} + +async function runPytestSilently(workspaceFolder: string) { + if (coverageRunInProgress) return; + coverageRunInProgress = true; + try { + const python = resolvePython(workspaceFolder); + if (!cachedPytestEnv) { + const [hasPytestCov, hasCoverage] = await Promise.all([ + checkPython(python, workspaceFolder, 'import pytest_cov'), + checkPython(python, workspaceFolder, 'import coverage'), + ]); + cachedPytestEnv = { hasPytestCov, hasCoverage }; + } + if (!cachedPytestEnv.hasCoverage) { + coverageRunInProgress = false; + return; + } + const args = cachedPytestEnv.hasPytestCov + ? ['-m', 'pytest', '--cov=.', '--cov-report='] + : ['-m', 'coverage', 'run', '-m', 'pytest']; + spawnPytest(python, args, workspaceFolder); + } catch { + coverageRunInProgress = false; + } +} + async function handleNoCoverage(workspaceFolder: string) { if (noCoveragePromptActive) return; noCoveragePromptActive = true; @@ -188,25 +257,11 @@ async function handleNoCoverage(workspaceFolder: string) { if (choice !== 'Run pytest') return; const args = hasPytestCov - ? ['-m', 'pytest', '--cov=.', '--cov-report=json'] + ? ['-m', 'pytest', '--cov=.', '--cov-report='] : ['-m', 'coverage', 'run', '-m', 'pytest']; - coverageRunInProgress = true; - coverageOutputChannel ??= vscode.window.createOutputChannel('Coverage Run'); - coverageOutputChannel.clear(); - coverageOutputChannel.show(true); - coverageOutputChannel.appendLine(`$ ${python} ${args.join(' ')}\n`); - - const proc = spawn(python, args, { cwd: workspaceFolder }); - proc.stdout.on('data', (d: Buffer) => coverageOutputChannel!.append(d.toString())); - proc.stderr.on('data', (d: Buffer) => coverageOutputChannel!.append(d.toString())); - proc.on('close', code => { - coverageRunInProgress = false; - coverageOutputChannel!.appendLine(`\n[exited ${code ?? '?'}]`); - if (code !== 0) { - vscode.window.showWarningMessage('pytest failed — check the Coverage Run output panel.'); - } - }); + coverageOutputChannel?.show(true); + spawnPytest(python, args, workspaceFolder); } finally { noCoveragePromptActive = false; } @@ -217,22 +272,17 @@ async function detectAndParse( ): Promise<{ report: CoverageReport; formatUsed: string } | undefined> { const jsonPath = path.join(workspaceFolder, 'coverage.json'); - if (fs.existsSync(jsonPath)) { - try { - const raw = JSON.parse(fs.readFileSync(jsonPath, 'utf-8')) as RawCoverageJson; - return { report: parseCoverageJson(raw), formatUsed: 'coverage.json' }; - } catch { /* fall through */ } - } + const sqlitePath = path.join(workspaceFolder, '.coverage'); - const xmlPath = path.join(workspaceFolder, 'coverage.xml'); - if (fs.existsSync(xmlPath)) { - try { - return { report: parseCoverageXml(fs.readFileSync(xmlPath, 'utf-8')), formatUsed: 'coverage.xml' }; - } catch { /* fall through */ } - } + const jsonExists = fs.existsSync(jsonPath); + const sqliteExists = fs.existsSync(sqlitePath); - const sqlitePath = path.join(workspaceFolder, '.coverage'); - if (fs.existsSync(sqlitePath)) { + // Prefer .coverage → coverage json when .coverage is newer than coverage.json. + // This ensures branch coverage config from the project is always respected. + const sqliteIsNewer = sqliteExists && jsonExists && + fs.statSync(sqlitePath).mtimeMs > fs.statSync(jsonPath).mtimeMs; + + if (sqliteExists && (!jsonExists || sqliteIsNewer)) { const python = resolvePython(workspaceFolder); const generated = await new Promise(resolve => { exec(`"${python}" -m coverage json`, { cwd: workspaceFolder }, err => resolve(!err)); @@ -249,9 +299,24 @@ async function detectAndParse( vscode.window.showErrorMessage( `Coverage Visualizer: Failed to read .coverage — ${err instanceof Error ? err.message : String(err)}` ); + return undefined; } } + if (jsonExists) { + try { + const raw = JSON.parse(fs.readFileSync(jsonPath, 'utf-8')) as RawCoverageJson; + return { report: parseCoverageJson(raw), formatUsed: 'coverage.json' }; + } catch { /* fall through */ } + } + + const xmlPath = path.join(workspaceFolder, 'coverage.xml'); + if (fs.existsSync(xmlPath)) { + try { + return { report: parseCoverageXml(fs.readFileSync(xmlPath, 'utf-8')), formatUsed: 'coverage.xml' }; + } catch { /* fall through */ } + } + return undefined; } @@ -302,6 +367,9 @@ function linesToDecorations(lines: number[]): vscode.DecorationOptions[] { } export function deactivate() { + clearTimeout(reloadTimer); + clearTimeout(testChangeTimer); + runningProc?.kill(); coveredDecoration?.dispose(); uncoveredDecoration?.dispose(); coverageOutputChannel?.dispose(); diff --git a/src/ui/statusBar.ts b/src/ui/statusBar.ts index 641c544..602d935 100644 --- a/src/ui/statusBar.ts +++ b/src/ui/statusBar.ts @@ -24,6 +24,13 @@ export function updateStatusBar(stats: { percentCovered: number; coveredStatemen statusBarItem.show(); } +export function showRunningStatusBar() { + if (!statusBarItem) return; + statusBarItem.text = '$(sync~spin) Running coverage…'; + statusBarItem.backgroundColor = undefined; + statusBarItem.show(); +} + export function clearStatusBar() { statusBarItem?.hide(); } diff --git a/tasks/plan-auto-run-on-test-change.md b/tasks/plan-auto-run-on-test-change.md new file mode 100644 index 0000000..9516d5a --- /dev/null +++ b/tasks/plan-auto-run-on-test-change.md @@ -0,0 +1,77 @@ +# Plan: Auto-Run Coverage on Test File Save + +Spec: `docs/spec-auto-run-on-test-change.md` + +## Dependency Order + +``` +Task 1 (config) + ↓ +Task 2 (status bar spinner) + ↓ +Task 3 (test file watcher + debounced run) + ↓ +Task 4 (manual test + compile check) + ↓ +Task 5 (ship gate) + ↓ +Task 6 (PR + release) +``` + +--- + +## Tasks + +- [ ] **Task 1 — Add config setting** + - Add `coverageVisualizer.autoRunOnTestChange` boolean (default `true`) to `package.json` + `contributes.configuration.properties` + - Add `autoRunOnTestChange: boolean` to `getConfig()` return value in `src/config.ts` + - Acceptance: `getConfig().autoRunOnTestChange` returns `true` by default, `false` when set + - Verify: `npm run compile` passes; read config value in extension console + - Files: `package.json`, `src/config.ts` + +- [ ] **Task 2 — Add spinner to status bar** + - Extend `updateStatusBar` in `src/ui/statusBar.ts` to accept an optional `running: boolean` + flag; when `true`, prepend `$(sync~spin)` to the status bar text + - Acceptance: status bar shows spinner icon while pytest is in progress, normal % when done + - Verify: `npm run compile` passes; manually trigger via F5 and confirm visual + - Files: `src/ui/statusBar.ts`, `src/extension.ts` (call site) + +- [ ] **Task 3 — Test file watcher + debounced auto-run** + - Inside `setupWatchers()` in `src/extension.ts`, add a `FileSystemWatcher` for the glob + `{test_*.py,*_test.py,tests/**/*.py,test/**/*.py}` + - On `onDidChange` / `onDidCreate`: if `autoRunOnTestChange` is false, return early; + otherwise debounce 2s then call the existing `handleNoCoverage` / `loadAndApply` flow + - Show spinner (Task 2) while running; clear it on completion + - Acceptance: save a test file → 2s delay → pytest runs → dashboard updates + - Verify: F5 host, save a test file, observe status bar spinner and dashboard refresh + - Files: `src/extension.ts` + +- [ ] **Task 4 — Compile + manual acceptance check** + - Run `npm run compile` — zero errors + - Run `npm test` — all existing tests pass + - Open F5 host, write a new test, save → confirm spinner → confirm dashboard % changes + - Confirm saving a non-test `.py` file does NOT trigger a run + - Confirm multiple rapid saves = one run (debounce) + +- [ ] **Task 5 — /ship quality gate** + - Run `/ship` — parallel fan-out to code-reviewer + security-auditor + test-engineer + - Resolve any Critical or Important findings before raising PR + - Files: all modified files above + +- [ ] **Task 6 — PR + release** + - Create branch `feat/auto-run-on-test-change` + - Run `/release patch` to bump `package.json` to next version on the branch + - Run `/create-pr CV` — title format `[CV-N]: [Enhancement] Auto-Refresh Coverage on Test Save` + - After merge: `git checkout main && git pull && git tag vX.Y.Z && git push origin vX.Y.Z` + +--- + +## Risks + +| Risk | Mitigation | +|------|-----------| +| User has large test suite — 30s run per save | `autoRunOnTestChange` setting defaults true; spinner makes wait visible | +| `handleNoCoverage` prompts user instead of silently running | Separate the "run pytest silently" path from the "ask user" prompt path | +| Watcher glob too broad — catches conftest.py, fixtures | `isTestFile()` already handles this; apply same filter before triggering | +| Windows glob separator issues | Use `RelativePattern` — VS Code handles separators internally |