Skip to content

feat(testing): skill activation integration test infrastructure#88

Open
dean0x wants to merge 1 commit intomainfrom
feat/skill-activation-testing
Open

feat(testing): skill activation integration test infrastructure#88
dean0x wants to merge 1 commit intomainfrom
feat/skill-activation-testing

Conversation

@dean0x
Copy link
Owner

@dean0x dean0x commented Mar 6, 2026

Summary

  • Adds integration test framework using claude -p for verifying ambient mode classification
  • 4 test cases: QUICK tier (silent for "thanks", git ops) and STANDARD tier (BUILD classification, DEBUG classification)
  • Separate vitest config with 60s timeout and retry support
  • Gracefully skips when claude CLI not available
  • Not part of npm test — runs via npm run test:integration

Test Cases

Test Expected
"thanks" QUICK — no classification output
"commit the current changes" QUICK — git op, no overhead
"add a login form with email and password fields" BUILD/STANDARD — TDD skill referenced
"fix the authentication error" DEBUG/STANDARD — diagnostic skills

Design Decisions

  • Non-determinism mitigation: Tests assert presence/absence of classification markers, not exact output
  • Cost control: Only 4 tests (each is an API call), retry: 2 for flaky cases
  • CI skip: describe.skipIf(!isClaudeAvailable()) — no setup needed for CI

Test plan

  • npm test — all 178 unit tests pass (integration excluded)
  • npm run test:integration — runs if claude CLI available
  • Integration tests use separate vitest config

Closes #83
Part of #78
Depends on #84 (ambient session injection)

Adds integration test framework for verifying ambient mode classification
using `claude -p` with output parsing. Tests are separate from the unit
test suite and require `claude` CLI to be installed.

Changes:
- tests/integration/helpers.ts: runClaude(), isClaudeAvailable(),
  classification assertion helpers
- tests/integration/ambient-activation.test.ts: 4 test cases covering
  QUICK (silent) and STANDARD (skills loaded) tiers
- vitest.integration.config.ts: separate config with 60s timeout + retry
- vitest.config.ts: exclude integration/ from default test suite
- package.json: add test:integration script

Run: npm run test:integration (manual, not part of npm test)

Closes #83
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(testing): skill activation integration test infrastructure

1 participant