feat(clio): auto-ingest documenter output (docs/**/*.md) into Clio#50
Merged
Conversation
Closes a long-standing inconsistency: cfcf auto-ingests almost
every workspace artifact (iteration logs, judge assessments,
reflection analyses, plan.md, decision-log, architect-review,
problem-pack, context-pack...) — but explicitly excluded the
documenter's docs/*.md output. The exclusion's stated rationale
("docs/ is canonical, Clio is redundant") applied equally to
plan.md, which IS auto-ingested. Carve-out was inconsistent and
cost cross-workspace discoverability of the most polished,
integrative artifact a workspace produces.
Behaviour:
- After documenter completes (both auto-document path in
iteration-loop AND standalone cfcf document), walk
<repo>/docs/ recursively; ingest every *.md as a separate
Clio doc.
- Stable per-file title: <workspace>: docs/<relative-path>
- updateIfExists: true → re-runs overwrite in place; sha256
dedup makes unchanged content a no-op.
- Author: documenter|<adapter>|<model>
- Metadata includes artifact_type=documenter-output + file_path
+ ingest_trigger ("loop-auto" | "manual") for filtering.
- Non-.md files skipped; dot-dirs skipped; empty files skipped.
- Per-file errors logged + counted but never fail the batch.
- Respects clio.ingestPolicy (off / summaries-only / all).
Documenter output runs on summaries-only — it IS a summary.
Implementation: new ingestDocumenterOutput() in loop-ingest.ts +
walkMarkdownFiles() helper; two call sites (iteration-loop
auto-document path, documenter-runner standalone path);
documenter template prose updated to match reality.
Test coverage: 11 new tests covering multi-file ingest, recursive
walk, updateIfExists round-trip, non-.md filtering, dot-dir
skipping, missing docs/ no-op, empty file skipping, author stamp,
trigger metadata, ingestPolicy gates. All 1049 tests pass.
Typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fstamatelopoulos
added a commit
that referenced
this pull request
May 15, 2026
Bumps version 0.24.3 -> 0.24.4 and consolidates the [Unreleased] section into a [0.24.4] release entry. Shipped via PR #50. Net-new since v0.24.3: - Clio auto-ingest of documenter docs/**/*.md output (loop-auto + standalone cfcf document, with updateIfExists, stable per-file titles, gated by clio.ingestPolicy) Test coverage: 1049 tests pass (+11 new). Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes a long-standing inconsistency: cfcf auto-ingests almost every workspace artifact into Clio (iteration logs, judge assessments, reflection analyses, plan.md, decision-log, architect-review, problem-pack, context-pack…), but explicitly excluded the documenter's
docs/*.mdoutput. The exclusion's stated rationale ("docs/is canonical, Clio is redundant") applied equally to plan.md — which IS auto-ingested. The carve-out was inconsistent and cost cross-workspace discoverability of the most polished, integrative artifact a workspace produces.Surfaced by user question while planning v0.24.4 dogfood of the freshly-finished gmbot run: "Is the documenter ingesting the generated docs into Clio? Or better the harness doing that?" — neither, by design. The design was wrong. This PR fixes it.
Behaviour
After the documenter completes (both auto-document path inside the iteration loop AND standalone
cfcf document):<repo>/docs/recursively*.mdfile: ingest into Clio as a separate document<workspace>: docs/<relative-path>(e.g.gmbot: docs/architecture.md,gmbot: docs/api/auth.md)updateIfExists: true— re-running the documenter overwrites in place, never duplicates; sha256 dedup makes unchanged content a no-opdocumenter|<adapter>|<model>{role: \"documenter\", artifact_type: \"documenter-output\", file_path: \"<rel>\", tier: \"semantic\", ingest_trigger: \"loop-auto\" | \"manual\", …}— thedocumenter-outputartifact_type makes the new docs filterable viacfcf clio search --metadata.mdfiles ignored (images, JSON config); dot-dirs (.git,.vscode) skipped; empty files skippedclio.ingestPolicy:\"off\"→ no-op;\"summaries-only\"and\"all\"→ runsFiles
packages/core/src/clio/loop-ingest.ts— newingestDocumenterOutput()+walkMarkdownFiles()helperpackages/core/src/iteration-loop.ts— call site inside auto-document branchpackages/core/src/documenter-runner.ts— call site for standalonecfcf documentpackages/core/src/templates/cfcf-documenter-instructions.md— prose updated to match realityTest plan
bun run test— 1049 tests pass (+11 new)bun run typecheck— cleanloop-ingest.test.tscover: multi-file ingest with stable titles, recursive walk,updateIfExistsround-trip (one doc per file, content updates in place), non-.mdfiltering, dot-dir skipping, missingdocs/no-op, empty file skipping, author stamp, trigger metadata (loop-autovsmanual),clio.ingestPolicy: \"off\"→ no-op,clio.ingestPolicy: \"summaries-only\"→ runscfcf documenton a workspace withdocs/architecture.md; verifycfcf clio search \"<term>\" --metadata '{\"artifact_type\":\"documenter-output\"}'returns the doc with the expected title formatVersioning
Target: v0.24.4 patch. New behaviour is additive and gated by the existing
clio.ingestPolicy(no opt-out is needed beyond the existing config knob). Same shape as v0.24.3's release.🤖 Generated with Claude Code