Skip to content

Auto-Generate Service API Docs from Source#217

Merged
wpak-ai merged 3 commits into
cppalliance:developfrom
jonathanMLDev:feat/auto-gen-service-doc
May 19, 2026
Merged

Auto-Generate Service API Docs from Source#217
wpak-ai merged 3 commits into
cppalliance:developfrom
jonathanMLDev:feat/auto-gen-service-doc

Conversation

@jonathanMLDev
Copy link
Copy Markdown
Collaborator

@jonathanMLDev jonathanMLDev commented May 18, 2026

1. scripts/generate_service_docs.py

  • Walks repo top-level apps with services.py (skips .venv, docs, scripts, etc.).
  • Uses ast.parse (no Django) for public top-level def / async def (names not starting with _).
  • Renders a markdown table (function, parameters, return type, first docstring paragraph).
  • core/protocols.py → docs/service_api/core_protocols.md: @runtime_checkable Protocol types + properties + require_* helpers.
  • Splices only between and .
  • --check: exit 1 if any file would change; --app NAME: one module.

2. Docs layout (docs/service_api/)

  • All 12 apps with services.py have a matching *.md (added boost_mailing_list_tracker, cppa_slack_tracker, wg21_paper_tracker).
  • Existing pages were migrated: header above markers, generated block, manual content below END (GitHub “Not yet / sync”, Discord commands/Pinecone, YouTube preprocessor/fetcher, etc.).

3. Pre-commit / CI

  • .pre-commit-config.yaml: local hook generate-service-docs with language: system, pass_filenames: false, always_run: true so CI’s pre-commit run -a always runs python scripts/generate_service_docs.py --check.

4. Contributor / index docs

  • docs/Contributing.md: extended which apps table, ### Regenerating service API docs, link to core_protocols.md, docs bullet updated.
  • docs/service_api/README.md: new rows + quick reference + regenerate pointer.
  • docs/Service_API.md: full index + per-app bullets + link to Contributing’s regen section.
  • README.md: Contributing bullet mentions regenerating service API docs.

Related Issue

close #204

Summary by CodeRabbit

  • Documentation
    • Regenerated and consolidated many Service API docs for consistency; added new service module pages (Slack, Mailing List, WG21 paper, Core protocols, YouTube script, and others). Updated contributing docs and README to document regenerating and committing service API docs.
  • Chores
    • Added a pre-commit check that validates generated service documentation.

Review Change Stack

@jonathanMLDev jonathanMLDev self-assigned this May 18, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 18, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 43b99150-389e-45f4-8673-bb58f9514ae1

📥 Commits

Reviewing files that changed from the base of the PR and between 73eb71a and 9940cac.

📒 Files selected for processing (1)
  • scripts/generate_service_docs.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • scripts/generate_service_docs.py

📝 Walkthrough

Walkthrough

Adds an AST-based CLI generator (scripts/generate_service_docs.py) that extracts public service function signatures and runtime-checkable protocol types, injects generated Markdown between SERVICE_API markers into docs/service_api/*.md (or verifies with --check), wires a pre-commit hook, and updates/regenerates many service API docs and contributing/README guidance.

Changes

Service API Documentation Auto-Generation

Layer / File(s) Summary
Generation script and CLI
scripts/generate_service_docs.py
Python tool using AST parsing to extract public async/sync function signatures and docstrings from services.py modules and @runtime_checkable protocols from core/protocols.py. Generates Markdown API tables bounded by marker comments, splices output into existing docs while preserving manual content, and provides --check mode for CI validation with distinct exit codes for different failure modes.
Pre-commit hook and contributing workflow
.pre-commit-config.yaml, docs/Contributing.md, README.md
New pre-commit hook runs generation in --check mode. Contributing guide adds service-layer app table expansion, "Regenerating service API docs" subsection explaining marker-based workflow, and explicit requirement to regenerate and commit docs after modifying services.py or core/protocols.py. README updated to reference generation step.
Documentation indices and navigation
docs/Service_API.md, docs/service_api/README.md
Service API indices extended to list additional service apps and core.protocols. Per-app documentation links and quick reference sections updated to index all documented services.
Generated service API documentation
docs/service_api/core_protocols.md, docs/service_api/boost_mailing_list_tracker.md, docs/service_api/cppa_slack_tracker.md, docs/service_api/wg21_paper_tracker.md, docs/service_api/boost_*.md, docs/service_api/clang_*.md, docs/service_api/cppa_*.md, docs/service_api/discord_*.md, docs/service_api/github_*.md
New core_protocols.md documents protocol types (ActivityRecord, IncrementalState, TrackerResult) and helper functions. New service docs created (boost_mailing_list_tracker, cppa_slack_tracker, wg21_paper_tracker). Existing service docs regenerated with unified "Public API (generated)" tables replacing per-model manual sections and normalizing signatures (e.g., optional params added, batch upsert and watermark helpers added for clang_github_tracker, nullable assignee accounts in github_activity_tracker).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested labels

enhancement

Suggested reviewers

  • snowfox1003
  • wpak-ai

Poem

🐇 I nibble lines and parse with care,

I stitch the docs from functions there,
Markers hold the generated bloom,
Pre-commit hums to check for doom,
Fresh API hops into the lair!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Auto-Generate Service API Docs from Source' accurately and clearly summarizes the main objective—adding automation to generate service API documentation from source code.
Linked Issues check ✅ Passed All acceptance criteria from #204 are met: the script extracts function signatures, generates markdown matching existing format, validates freshness via --check mode, preserves manual content, and updates contributor docs.
Out of Scope Changes check ✅ Passed All changes directly support the PR objective. Documentation updates consolidate generated API tables, contributing docs add regeneration guidance, pre-commit integration enforces doc freshness, and the script automates extraction from services.py and core/protocols.py.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/service_api/github_activity_tracker.md (1)

44-47: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remove stale “Not yet in API” entries that are now documented.

This section contradicts the generated table above (e.g., add_commit_file_change, create_or_update_github_file, create_or_update_issue_comment, create_or_update_pr_review, create_or_update_pr_comment are already listed). Keeping both will confuse contributors.

Suggested doc fix
-## Not yet in API
-
-- GitCommit, GitHubFile, GitCommitFileChange: add `create_commit`, `create_github_file`, `add_commit_file_change` when needed.
-- IssueComment, PullRequestReview, PullRequestComment: add `create_issue_comment`, `create_pr_review`, `create_pr_comment` when needed.
-
----
+---
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/service_api/github_activity_tracker.md` around lines 44 - 47, Remove the
stale "Not yet in API" list entries that are already documented to avoid
duplication and confusion: delete the bullet lines referencing
GitCommit/GitHubFile/GitCommitFileChange and
IssueComment/PullRequestReview/PullRequestComment and ensure only the canonical
function names remain (e.g., add_commit_file_change,
create_or_update_github_file, create_or_update_issue_comment,
create_or_update_pr_review, create_or_update_pr_comment) so the table and prose
are consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/service_api/wg21_paper_tracker.md`:
- Around line 15-18: The API table generator is emitting empty return/summary
cells (e.g., the row for mark_paper_downloaded) which makes docs incomplete;
update the generator logic that renders rows for functions like
get_or_create_mailing, get_or_create_paper, get_or_create_paper_author, and
mark_paper_downloaded to substitute explicit fallbacks when a return annotation
or docstring summary is missing (use a consistent placeholder such as "None" or
"Unknown" for return type and "—" for summary), ensuring every table cell is
populated even if annotations/docstrings are absent.

In `@scripts/generate_service_docs.py`:
- Around line 91-113: The _format_args routine currently omits the
positional-only "/" and the bare "*" separators, so update _format_args to
insert "/" after processing args.posonlyargs when any posonlyargs exist (and
ensure it’s omitted when none), and insert a bare "*" before kwonlyargs when
args.vararg is None but there are kwonlyargs; use the existing symbols
(args.posonlyargs, args.args, first_default, args.vararg, args.kwonlyargs,
args.kw_defaults, args.kwarg) and the same parts list/formatting logic so the
generated signature includes "/" after positional-only parameters and "*" before
keyword-only parameters when appropriate.
- Line 102: The zip call pairing args.kwonlyargs and args.kw_defaults should
include strict=True to satisfy Ruff B905; update the expression
zip(args.kwonlyargs, args.kw_defaults) used in the function that iterates over
keyword-only args to zip(args.kwonlyargs, args.kw_defaults, strict=True) so the
lengths are enforced (args.kwonlyargs and args.kw_defaults are index-aligned AST
fields and strict=True is safe).

---

Outside diff comments:
In `@docs/service_api/github_activity_tracker.md`:
- Around line 44-47: Remove the stale "Not yet in API" list entries that are
already documented to avoid duplication and confusion: delete the bullet lines
referencing GitCommit/GitHubFile/GitCommitFileChange and
IssueComment/PullRequestReview/PullRequestComment and ensure only the canonical
function names remain (e.g., add_commit_file_change,
create_or_update_github_file, create_or_update_issue_comment,
create_or_update_pr_review, create_or_update_pr_comment) so the table and prose
are consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 216d9e3e-8234-4d55-9957-615860be4aa3

📥 Commits

Reviewing files that changed from the base of the PR and between 2037bb5 and ad55c04.

📒 Files selected for processing (19)
  • .pre-commit-config.yaml
  • README.md
  • docs/Contributing.md
  • docs/Service_API.md
  • docs/service_api/README.md
  • docs/service_api/boost_library_docs_tracker.md
  • docs/service_api/boost_library_tracker.md
  • docs/service_api/boost_mailing_list_tracker.md
  • docs/service_api/boost_usage_tracker.md
  • docs/service_api/clang_github_tracker.md
  • docs/service_api/core_protocols.md
  • docs/service_api/cppa_pinecone_sync.md
  • docs/service_api/cppa_slack_tracker.md
  • docs/service_api/cppa_user_tracker.md
  • docs/service_api/cppa_youtube_script_tracker.md
  • docs/service_api/discord_activity_tracker.md
  • docs/service_api/github_activity_tracker.md
  • docs/service_api/wg21_paper_tracker.md
  • scripts/generate_service_docs.py

Comment thread docs/service_api/wg21_paper_tracker.md Outdated
Comment thread scripts/generate_service_docs.py
Comment thread scripts/generate_service_docs.py Outdated
@jonathanMLDev jonathanMLDev requested a review from snowfox1003 May 18, 2026 19:54
Comment thread scripts/generate_service_docs.py Outdated
Comment thread scripts/generate_service_docs.py Outdated
@snowfox1003 snowfox1003 self-requested a review May 19, 2026 20:19
@snowfox1003 snowfox1003 requested a review from wpak-ai May 19, 2026 20:19
@wpak-ai wpak-ai merged commit ba453d9 into cppalliance:develop May 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-Generate Service API Docs from Source

3 participants