docs: Docs/api pipeline improvements

[planetf1]

API docs pipeline improvements

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes API Reference Documentation #532

Improves the tooling/docs-autogen/ pipeline (build.py, generate-ast.py, decorate_api_mdx.py, audit_coverage.py):

• Fix 4 Mintlify parse errors — bare >>> doctest blocks in module docstrings were parsed as nested blockquotes by MDX; wrapped in fenced code blocks via wrap_doctest_blocks(). Unescaped {/} in prose (e.g. inline Python/JSON examples) caused acorn parse errors; now escaped outside code fences.
• Fix GitHub source links — fix_source_links() now handles the HTML <a href="..."> format that mdxify actually emits. The old regex only matched Markdown [View source](...) format and never fired, leaving all links on blob/main/ instead of the versioned tag.
• Richer landing page — generate_landing_page() now emits a prose bullet list with bold linked module names and descriptions, replacing the previous plain bullet list. Descriptions are extracted from the decorated MDX body (the full module preamble text) rather than the short frontmatter one-liner.
• --source-dir support — build.py and audit_coverage.py now accept --source-dir to generate or audit docs for a different checkout without touching that repo. The docs.json navigation is updated in the source repo (bug fix: previously always wrote to mellea-d).
• Docstring quality audit — audit_coverage.py --quality detects 8 categories of incomplete docstrings (missing, short, no Args/Returns/Raises sections, param name mismatches). Scoped to documented public symbols only — 322/322 documented (100% coverage).
• Navigation orphan audit — audit_coverage.py --orphans finds MDX files that exist on disk but are absent from the docs.json navigation tree.
• CI/pre-commit support — --fail-on-quality flag exits 1 when quality issues are found, making the audit usable as a hard gate. When run in GitHub Actions (GITHUB_ACTIONS=true), each issue is also emitted as a ::error:: annotation so problems surface inline on the PR diff. Groundwork for ci: gate doc quality in CI and pre-commit to prevent future degradation #616.
• *args/**kwargs forwarder exemption — functions that forward via *args/**kwargs (e.g. mify) are correctly exempt from no_args and param_mismatch checks. The tool uses Griffe's ParameterKind rather than a name-prefix heuristic, which was previously a no-op.

Generated API docs removed from the repo

docs/docs/api/ and docs/docs/api-reference.mdx are fully generated artefacts — they are now both added to .gitignore and removed from git tracking in this PR. The previous committed snapshot (28 MDX files) is deleted; going forward these files are generated on demand via uv run poe apidocs or by the docs-autogen-pr.yml CI workflow on each release tag. They should never be hand-edited or re-committed directly.

uv run poe commands for API docs

Command	Description
uv run poe apidocs	Generate + decorate API docs (version auto-read from pyproject.toml)
uv run poe apidocs-preview	Generate fresh docs to /tmp/mellea-preview, then run full quality audit
uv run poe apidocs-quality	Audit docstring quality for all documented public symbols including methods — reports missing docstrings, short docstrings, missing Args/Returns/Raises sections, and param name mismatches
uv run poe apidocs-orphans	Find MDX files present on disk but missing from docs.json navigation
uv run poe apidocs-validate	Validate generated docs: coverage ≥ 80%, MDX syntax, internal links
uv run poe apidocs-test	Run docs tooling unit tests
uv run poe apidocs-clean	Remove all generated API docs and build artefacts

Cross-repo usage (direct script invocation)

Generate and audit docs for a different checkout without touching that repo:

# Generate docs from mellea-b source, write to /tmp
uv run python tooling/docs-autogen/build.py \
    --source-dir ../mellea-b \
    --output-dir /tmp/mellea-b-preview/api

# Audit quality against the generated output
uv run python tooling/docs-autogen/audit_coverage.py \
    --quality \
    --docs-dir /tmp/mellea-b-preview/api \
    --source-dir ../mellea-b

# Fail if any quality issues found (for CI/pre-commit)
uv run python tooling/docs-autogen/audit_coverage.py \
    --quality --fail-on-quality \
    --docs-dir docs/docs/api

Related issues

• docs: improve module-level docstrings for API reference #612 — improve module-level docstrings for API reference
• docs: add Args/Returns sections to public function docstrings #613 — add Args/Returns sections to public function docstrings
• fix: add missing type annotations to public API functions #615 — add missing type annotations to public API (phase 2; surfaces additional no_returns issues once fixed)
• ci: gate doc quality in CI and pre-commit to prevent future degradation #616 — gate doc quality in CI and pre-commit (depends on this PR merging first)

Testing

• Pipeline tested end-to-end against ../mellea-b with the docstring improvements from PR docs: improve docstrings for API reference (#612) #614 — audit reports 0 quality issues after those fixes
• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[planetf1]

Note: Currently this removes the old generated files.
I propose if we want to regen the docs and checkin (deployment discussion) we do this in a new PR? (but it could be same if required)

[jakelorocco]

looks mostly good to me; I'm confused on the api doc update mechanism though. We have the api docs in our git ignore, but we are still pushing them to the github repo? I don't understand how that works with the automation.

[psschwei]

Summarizing from discussions earlier today, this PR:

• adds tooling to assist with building the API docs
• removes the out of date API docs that were currently there
• updates the action to generate the new API docs and push them to main

I think, based on discussions earlier, that instead of pushing to main the idea is to push to a dedicated docs branch that will be used for mintlify to grab the files from (@planetf1 keep me honest here)

In general, I'm good with that process (I think there's still a few minor things to clean up like some extra files / bob tags, so not officially approving).

[planetf1]

• Removed stray watch_20260210-162335 diagnostic file committed by accident
• Removed tooling/docs-autogen/requirements.txt (deps now in pyproject.toml)
• Removed # Made with Bob attribution comment from workflow file
• Disabled CI auto-trigger on tag push pending branch strategy decision; added TODO comment referencing PR docs: Docs/api pipeline improvements #611
• Updated cache-dependency-glob in workflow from deleted requirements.txt to uv.lock
• Squashed 29 commits to 1 and rebased onto upstream/main

Now ready for re-review @HendrikStrobelt @psschwei @jakelorocco

[jakelorocco]

I think I'm still lost on where / when the docs will be, and how Mintlify will grab them. I don't see a concrete answer in this PR or the linked improvement issue. If that get's sorted out, I think this PR looks good.

[psschwei]

I think I'm still lost on where / when the docs will be, and how Mintlify will grab them. I don't see a concrete answer in this PR or the linked improvement issue. If that get's sorted out, I think this PR looks good.

I believe the plan was a dedicated docs branch (can you confirm @planetf1 ?)

[planetf1]

The intent of this PR is to create tools that can build docs from the source content (our source code) with validation checks to ensure we're creating a good quality API reference.

Yes. Intent is to publish to a dedicated docs branch (or rather two - one for each site). For example we'll write the full docs tree to docs/docs in that target branch. That will include both our 'written' conceptrual docs & the api reference. Mintlify will be pointed there so it publishes docs when they change

When we've merged this, and pr 601 (I'm implementing some of the review comments) I'll do that part which hopefully will become clearer.

[jakelorocco]

Our usual pytest commands won't catch these tests. I'm assuming it was intentional to keep them separate (which I'm fine with). We should think about if we need something to ensure they don't break.

[planetf1]

Our usual pytest commands won't catch these tests. I'm assuming it was intentional to keep them separate (which I'm fine with). We should think about if we need something to ensure they don't break.

current plan is add to CI and potentially pre-commit.
Initially soft:warnings, but once another pass has been done in #645 we can probably make them hard failures -- this is the reason to add the checks, to keep the doc quality. Additionally i've opened #638 for harder code validation

[planetf1]

Removed all 7 # Made with Bob attribution comments from tooling/docs-autogen/ files (3 changed in this PR + 4 pre-existing on main): audit_coverage.py, validate.py, test_escape_mdx.py, test_cross_references.py, test_mintlify_anchors.py, test_validate.py, test_anchor_collisions.py.

[psschwei]

I ran this locally to build the docs and preview against mintlify and everything seemed to work fine (I did not run the tests, just the basic uv run poe apidocs).

I think we're in a good spot to merge this and then iterate if there are small details we need to fix once its in action.

[jakelorocco]

lgtm; I'm fine if y'all want to iterate further in this PR. But I also think the remaining potential changes are small and can be handled in future PRs as we improve the process.