feat(fact-check): Phase 1 — AST-based post-polish fact-check

[silversurfer562]

Summary

Phase 1 of the polish-fact-check spec (umbrella PR #27). Adds an AST-based post-polish verification layer that catches LLM-fabricated technical detail without calling an LLM.

Four checks, zero LLM cost, ~600 LOC including tests + regression fixture:

Check	What it catches	Fixture coverage
check_python_refs	Hallucinated imports + dotted paths (attune.ops._readers)	2 of 6
check_cli_refs	Invented CLI flags (--allow-run)	1 of 6
check_md_links	Missing relative link targets	1 of 6
check_numeric_refs	Wrong counts (498 templates)	1 of 6
Phase 1 total		5 of 6 (the 6th — missing security callout — is Phase 3)

Behavior

Wired into generator.apply_polish_results so every polished template runs through the check immediately after being written.

• Soft-fail (default) — findings are appended as an ## Unresolved references table at the bottom of the polished file.
• Strict — FactCheckError raised after the write; the bad file is left on disk for inspection.
• Off — short-circuits before reading the file.

Controlled via ATTUNE_AUTHOR_FACT_CHECK env var (off | soft | strict, default soft) and [tool.attune-author.fact-check] in pyproject.toml:

[tool.attune-author.fact-check]
enabled = true
soft_fail = true
check_python_refs = true
check_cli_refs = true
check_md_links = true
check_numeric_refs = true

[tool.attune-author.fact-check.skip]
"docs/architecture/some-feature.md" = ["check_md_links"]

Version coupling

check_cli_refs resolves against whichever attune-ai is installed in the active venv. Every finding includes proactive context so an operator running against a different attune-ai version can resolve false positives without spelunking:

Line 17: `attune ops --allow-run` — flag not found in `attune ops --help`

Detected against attune 6.8.0 (installed in active venv). If you
are regenerating against a different version, verify the flag
exists in that version's `attune --help`.
To override:
  - One-off:  attune-author generate FEATURE --skip-check check_cli_refs
  - Per file: [tool.attune-author.fact-check.skip]
              "docs/how-to/foo.md" = ["check_cli_refs"]

Regression fixture

tests/fixtures/fact_check_ops_dashboard/ ships pre-fix and post-fix versions of the four attune-ai PR #351 docs. The fixture-based test suite asserts each check fires on the pre-fix files and is silent on the post-fix files, exercising the spec's "5/6 ops-dashboard errors caught" exit gate.

Test plan

• 55 new tests in tests/unit/fact_check/ — all green locally on Py 3.10
• ruff check clean on new code
• No regressions in unrelated test suites (10 pre-existing failures verified unchanged with my changes stashed)
• CI matrix (Ubuntu / macOS / Windows × Py 3.10–3.13)
• Live regen of a feature to validate the soft-fail block format in real output

Spec tasks

Completed in this PR: 1.1–1.8, 1.10, 1.11, 1.11.1, 1.12, 1.13, 1.14, 1.15, 1.16.

Deferred: 1.9 (CLI flags --fact-check=strict / --no-fact-check) — the env var ATTUNE_AUTHOR_FACT_CHECK ships in this PR; the named CLI flags can land as a small follow-up that wraps the existing env hook.

Phase 2 (ground-truth context injection), Phase 3 (faithfulness judge), and Phase 4 (tutorial static check) remain to ship as separate PRs per the spec.

🤖 Generated with Claude Code