feat(test): coverage gate + golden snapshots + handler/polish error paths (test-strategy pass 1)#23
Merged
Merged
Conversation
Moved from ~/attune/specs/regen-pipeline/. The spec primarily targets attune-author (the `attune-author regenerate` CLI command in src/attune_author/cli.py:507, plus maintenance.py and maintenance_batch.py). Phase 3 status bumped from "approved" to "complete" — the CLI ships, the batch variant ships, CHANGELOG documents both. A pointer to the implementation files is added under the "Phase 3: Tasks" header. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…polish errors Pass 1 of the test-strategy spec for the attune product family — Layer 4 (attune-author, the AI-assisted authoring layer and reference for the family's LLM mocking pattern). Coverage gate: - Added pytest-cov branch coverage + fail_under = 85 in pyproject.toml (measure-then-lock at baseline-2pp; current 87.15% with new tests). - tests.yml: coverage runs on the ubuntu x py3.11 cell only; coverage.xml uploaded as an artifact. - Stretch target for pass 2: 92% (already close at 87%). New test files (26 new tests; total 788 passing + 32 skipped): - tests/test_generated_templates_golden.py — syrupy snapshots of generated concept/task/reference output. Schema drift in Jinja2 templates or in the frontmatter contract surfaces immediately rather than weeks later via the cross-repo workflow. Snapshots in tests/__snapshots__/. Helper strips volatile fields (timestamps + source_hash) before comparison so reruns are deterministic. - tests/test_mcp_handlers_integration.py — exercises the AttuneAuthorHandlers methods through their full request → orchestration → response lifecycle. Path-validation contract, author_init bootstrap (empty / discovered / already-init paths), author_status, author_generate, author_lookup envelope shapes. No real LLM calls. - tests/test_parallel_polish_errors.py — error injection in _parallel_polish: single-worker PolishError, TimeoutError, and all-workers-fail cascade. Confirms exceptions propagate via Future.result(); also pins use_rag and template_type pass-through semantics. Test infra: - New tests/README.md positions attune-author as the LLM-mocking reference (the autouse _lenient_polish_by_default + _reset_rag_pipeline conftest fixtures). Documents the snapshot policy and the live marker. - Added syrupy to dev deps. Spec: /Users/patrickroebuck/attune/specs/test-strategy/ Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The new tests/test_mcp_handlers_integration.py added in this PR uses `async def` test functions with @pytest.mark.asyncio decorators, but pytest-asyncio was not in the [dev] extras. CI across all 12 OS/Py combinations failed with: Failed: async def functions are not natively supported. Fix: add `pytest-asyncio>=0.21,<2.0` to [dev] extras and set `asyncio_mode = "auto"` in [tool.pytest.ini_options] to auto-collect the marker. Sibling attune-ai uses pytest-asyncio>=0.21,<2.0 and asyncio_mode = "auto" — matching that pattern.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pass 1 of the test-strategy spec — Layer 4 (attune-author, the AI-assisted authoring layer and reference for the attune family's LLM mocking pattern).
What changed
Coverage gate
New test files
Tests README — reference layer
attune-author is the reference for LLM mocking in the family. The new `tests/README.md` documents the three autouse fixtures (`_lenient_polish_by_default`, `_reset_rag_pipeline`, import-time `anthropic.Anthropic` patches) that the other layers' READMEs point at.
Dependencies
Test plan
Out of scope (deferred to subsequent passes)
🤖 Generated with Claude Code