feat(rag,ui): index bundled project docs into the RAG corpus (#214)

[w7-mgfcode]

Implements PRP-23 — RAG Corpus Manager (MVP). Closes #214.

What

Adds one additive orchestration endpoint, POST /rag/index/project-docs, that discovers the repo's bundled markdown (docs/**, PRPs/**, and the root allow-list README.md / AGENTS.md / CHANGELOG.md) and indexes each file through the existing RAGService.index_document path — reusing its chunking, embedding, SHA-256 content-hash idempotency, and upsert. The Admin → RAG Sources tab gets an Index Project Docs button.

No Alembic migration (no schema change — category rides in the existing DocumentSource.metadata_ JSONB), no new slice, no .env var, no app/main.py change.

Changes

Backend (app/features/rag/)

• schemas.py — IndexProjectDocsRequest / ProjectDocResult / IndexProjectDocsResponse.
• service.py — pure _discover_project_doc_files() + async index_project_docs() orchestrator; _PROJECT_ROOT_FILES constant.
• routes.py — POST /rag/index/project-docs (mirrors the index_document route's EmbeddingError→502 / SQLAlchemyError→DatabaseError handling).
• Tests — discovery unit tests + TestIndexProjectDocsEndpoint integration class; conftest cleanup glob widened test-% → %test-% for nested fixture paths.

Frontend

• types/api.ts — 3 new interfaces.
• use-rag-sources.ts — useIndexProjectDocs mutation hook (invalidates ['rag-sources']).
• admin.tsx — Index Project Docs button (spinner, toast.success/toast.warning/toast.error).

Docs — API_CONTRACTS.md endpoint row; the PRPs/PRP-23-*.md plan.

Validation

• ✅ ruff check + ruff format --check clean; CRLF line endings preserved.
• ✅ mypy --strict + pyright --strict clean (218 files).
• ✅ 104 unit tests + 19 integration tests pass (5 new for this endpoint); test_strict_mode_policy.py still green.
• ✅ Frontend tsc --noEmit clean, lint 0 errors, 54 tests pass.
• ✅ Dogfooded in a browser against a live Ollama (qwen3-embedding:4b @ dim 1536): real index populated 112 sources / 1400 chunks; semantic search returns cited chunks; the Admin button shows the success toast (unchanged 112 — 1400 chunks on the idempotent re-run); Knowledge page renders the corpus; zero console errors.

Known limitation (follow-up candidate)

POST /rag/index/project-docs has no concurrency guard — two overlapping calls race on index_document's check-then-insert and one returns 500 on a uq_source_type_path violation. This is a pre-existing property of index_document (shared with POST /rag/index), not introduced here; a single click works correctly. A module-level asyncio.Lock (as the demo slice uses) would be a clean follow-up.

---

Added: ForecastLab user guides (#216)

A second commit (docs: add forecastlab user-facing guides (#216)) adds four curated, user-facing guides under docs/user-guide/ — getting-started.md, dashboard-guide.md, feature-reference.md, agents-and-rag-guide.md — synthesized from existing repo material and cross-checked against the codebase (routes, endpoints, seeder, agent safety limits). These replace the raw internal docs as the intended RAG-corpus content. Also addresses #216.

[sourcery-ai]

Sorry @w7-mgfcode, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5c2ae119-cb0e-4be8-bc52-df1ac65ae542

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/rag-index-project-docs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}