| Attribute | Value |
|---|---|
| System Name | Progressive Disclosure Knowledge System Generator |
| Version | 3.4 |
| Architecture Type | Three-Layer Decomposition System |
| Platform | Claude Projects (Anthropic) |
| Primary Function | Transform monolithic prompts into retrieval-optimised knowledge systems |
The Progressive Disclosure Knowledge System Generator v3.4 is a prompt decomposition methodology implemented as a Claude Project. It transforms large, monolithic system prompts into structured knowledge systems optimised for Claude Projects' Contextual Retrieval mechanism.
The Conductor Pattern: Instructions don't contain operational logic—they route to it. This enables any prompt size to be decomposed whilst maintaining a slim, effective routing layer.
| Target | Value |
|---|---|
| Instructions Size | 4,000–8,000 characters |
| Individual Knowledge Files | 2,000–5,000 characters |
| Total Domains | 8–15 per system |
| Test Coverage | 25+ verification queries |
Claude Projects has practical limitations:
- Long instructions dilute effectiveness — Instructions over ~10,000 characters reduce routing precision
- Monolithic prompts can't be selectively retrieved — All-or-nothing retrieval wastes context
- Operational logic competes with routing guidance — Workflows embedded in instructions crowd out retrieval direction
- No chunk boundary control — Content is split arbitrarily into ~400–800 token chunks
Decompose EVERYTHING—both operational logic AND reference knowledge—into retrievable files, leaving Instructions as a pure routing layer.
┌─────────────────────────────────────────────────────────────┐
│ BEFORE (Monolithic) │
│ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ 50,000+ character prompt containing: │ │
│ │ • Purpose statement │ │
│ │ • Phase 1 workflow (2,000 chars) │ │
│ │ • Phase 2 workflow (3,000 chars) │ │
│ │ • Decision framework (1,500 chars) │ │
│ │ • Methodology explanation (5,000 chars) │ │
│ │ • Examples (10,000 chars) │ │
│ │ • Constraints and principles (3,000 chars) │ │
│ │ • ...everything else... │ │
│ └───────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ AFTER (Decomposed) │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ INSTRUCTIONS (~6,000 chars) │ │
│ │ • Purpose (300 chars) │ │
│ │ • Content Index (1,500 chars) │ │
│ │ • Query Routing (3,000 chars) │ │
│ │ • Edge Cases (500 chars) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ OPERATIONAL FILES │ │ REFERENCE FILES │ │
│ │ (HOW it works) │ │ (WHAT it knows) │ │
│ │ • phase-1.md │ │ • methodology.md │ │
│ │ • phase-2.md │ │ • examples.md │ │
│ │ • decisions.md │ │ • patterns.md │ │
│ │ • constraints.md │ │ • troubleshooting.md│ │
│ └─────────────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
The system implements a three-layer architecture where each layer serves a distinct purpose:
┌─────────────────────────────────────────────────────────────┐
│ LAYER 1: INSTRUCTIONS (Routing Layer) │
│ Target: 4,000–8,000 characters │
│ │
│ The Conductor — knows what exists, routes to it: │
│ • Irreducible core purpose (~300 chars) │
│ • Content index (~1,500 chars) │
│ • Query routing (~3,000 chars) │
│ • Edge case handling (~500 chars) │
│ │
│ ❌ Does NOT contain: workflows, methodology, examples │
└─────────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ LAYER 2: OPERATIONAL │ │ LAYER 3: REFERENCE │
│ KNOWLEDGE │ │ KNOWLEDGE │
│ │ │ │
│ HOW the system works: │ │ WHAT the system knows: │
│ • Phases & workflows │ │ • Methodology │
│ • Decision frameworks │ │ • Examples & templates │
│ • Principles │ │ • Patterns │
│ • Constraints │ │ • Definitions │
│ • Output specifications │ │ • Troubleshooting │
│ │ │ │
│ Voice: Imperative │ │ Voice: Descriptive │
│ ("Do X, then Y") │ │ ("X works by...") │
└─────────────────────────┘ └─────────────────────────┘
| Attribute | Layer 1: Instructions | Layer 2: Operational | Layer 3: Reference |
|---|---|---|---|
| Purpose | Route to content | Define processes | Provide knowledge |
| Question Answered | "Where do I look?" | "How do I do this?" | "What do I apply?" |
| Voice | Directive | Imperative | Descriptive |
| Retrieval | Always present | On-demand | On-demand |
| Size Target | 4,000–8,000 chars total | 2,000–5,000 chars per file | 2,000–5,000 chars per file |
The project contains 20 files organised into four functional categories:
┌─────────────────────────────────────────────────────────────┐
│ COMPONENT TAXONOMY │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ ORCHESTRATION (2 files) │ │
│ │ Define system purpose and execution sequence │ │
│ │ │ │
│ │ • README.md (overview, version history) │ │
│ │ • INSTRUCTIONS (routing layer - in system prompt) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ PHASES (7) │ │ MECHANICS │ │ TEMPLATES │ │
│ │ │ │ (5) │ │ (4) │ │
│ │ Sequential │ │ Technical │ │ Blueprints │ │
│ │ workflow │ │ knowledge │ │ for output │ │
│ │ │ │ │ │ │ │
│ │ • phase-0 │ │ • retrieval │ │ • routing │ │
│ │ • phase-1 │ │ • chunks │ │ • operation │ │
│ │ • phase-2 │ │ • semantic │ │ • reference │ │
│ │ • phase-3 │ │ • bm25 │ │ • test │ │
│ │ • phase-4 │ │ • fusion │ │ │ │
│ │ • phase-5 │ │ │ │ │ │
│ │ • phase-6 │ │ │ │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────┐ │
│ │ QUALITY ASSURANCE (2) │ │
│ │ │ │
│ │ • anti-patterns.md │ │
│ │ • quality-gates.md │ │
│ └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
| Category | File | Purpose | Size | Type |
|---|---|---|---|---|
| Orchestration | README.md | System overview, version history | 7.5K | Reference |
| Instructions (prompt) | Routing layer, execution sequence | ~5K | Operational | |
| Phases | phase-0-classification.md | Classify source content | 11K | Operational |
| phase-1-ingestion.md | Extract content by layer | 9.5K | Operational | |
| phase-2-architecture.md | Design domain structure | 8K | Operational | |
| phase-3-instructions.md | Create routing instructions | 8K | Operational | |
| phase-4-domain-files.md | Write reference files | 9.5K | Operational | |
| phase-5-hybrid-optimisation.md | Balance retrieval signals | 6.5K | Operational | |
| phase-6-verification.md | Test retrieval accuracy | 8.5K | Operational | |
| Mechanics | retrieval-mechanics.md | How Contextual Retrieval works | 5K | Reference |
| chunk-resilience.md | Writing for arbitrary splits | 7K | Reference | |
| semantic-hedging.md | Embedding optimisation | 6K | Reference | |
| bm25-optimisation.md | Keyword matching optimisation | 6K | Reference | |
| rank-fusion.md | Balancing retrieval signals | 5.5K | Reference | |
| Templates | routing-instruction-template.md | Blueprint for Instructions | 11K | Reference |
| operational-domain-template.md | Blueprint for operational files | 11K | Reference | |
| domain-file-template.md | Blueprint for reference files | 6.5K | Reference | |
| test-matrix-template.md | Blueprint for verification | 7K | Reference | |
| Quality | anti-patterns.md | 16 common failures to avoid | 9K | Reference |
| quality-gates.md | Final checklist before delivery | 6.5K | Reference |
The system executes through 8 sequential phases with one mandatory checkpoint:
┌──────────────────────────────────────────────────────────────────────────┐
│ EXECUTION SEQUENCE │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────────────────┐ │
│ │ PHASE 0 │───▶│ PHASE 1 │───▶│ PHASE 2 │───▶│ ⚠️ USER CHECKPOINT │ │
│ │ Classify│ │ Ingest │ │ Architect│ │ Approval Required │ │
│ └─────────┘ └─────────┘ └─────────┘ └──────────┬──────────┘ │
│ │ │
│ ▼ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────────────────┐ │
│ │ PHASE 6 │◀───│ PHASE 5 │◀───│ PHASE 4 │◀───│ PHASE 3 │ │
│ │ Verify │ │ Optimise│ │ Domain │ │ Instructions │ │
│ └────┬────┘ └─────────┘ │ Files │ └─────────────────────┘ │
│ │ └─────────┘ │
│ ▼ │
│ ┌─────────┐ ┌─────────┐ │
│ │ PHASE 7 │───▶│ PHASE 8 │───▶ DELIVERY │
│ │ Test │ │ Package │ │
│ │ Matrix │ │ │ │
│ └─────────┘ └─────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────┘
| Phase | Name | Input | Output | Key Activities |
|---|---|---|---|---|
| 0 | Classify | Source prompt + attachments | Classification document | Measure size, tag content types, design architecture |
| 1 | Ingest | Classification + source | Knowledge inventory | Extract operational content, extract reference content, build terminology index |
| 2 | Architect | Knowledge inventory | Domain architecture table | Group domains, define triggers, map connections |
| Checkpoint | Architecture table | User approval | MANDATORY PAUSE — Do not proceed without confirmation | |
| 3 | Instructions | Approved architecture | INSTRUCTIONS.md | Create routing layer with query examples |
| 4 | Domain Files | Architecture + inventory | Operational + Reference files | Write chunk-resilient content |
| 5 | Optimise | All files | Optimised files | Balance BM25 and embedding signals |
| 6 | Verify | Complete system | Test results | Execute test matrix, diagnose failures |
| 7 | Test Matrix | Verified system | TEST-MATRIX.md | Document 25+ verification queries |
| 8 | Package | All deliverables | Final system | Organise structure, create README |
┌──────────────────────────────────────────────────────────────────────────┐
│ DATA FLOW │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ SOURCE MATERIAL │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ • Primary system prompt │ │
│ │ • Attached methodology documents │ │
│ │ • Examples and templates │ │
│ │ • Supporting files │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ PHASE 0: CLASSIFICATION ───────────────────────────────────────────── │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ CLASSIFICATION DOCUMENT │ │
│ │ • Source size: X characters │ │
│ │ • Operational content: ~Y% │ │
│ │ • Reference content: ~Z% │ │
│ │ • Proposed file architecture │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ PHASE 1: INGESTION ────────────────────────────────────────────────── │
│ │ │
│ ┌───────────────┴───────────────┐ │
│ ▼ ▼ │
│ ┌─────────────────────────────┐ ┌─────────────────────────────┐ │
│ │ OPERATIONAL EXTRACTION │ │ REFERENCE EXTRACTION │ │
│ │ • Phases & workflows │ │ • Methodology │ │
│ │ • Decision frameworks │ │ • Examples │ │
│ │ • Principles │ │ • Patterns │ │
│ │ • Constraints │ │ • Definitions │ │
│ └─────────────────────────────┘ └─────────────────────────────┘ │
│ │ │ │
│ └───────────────┬───────────────┘ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ KNOWLEDGE INVENTORY │ │
│ │ • Tagged operational content │ │
│ │ • Tagged reference content │ │
│ │ • Terminology index │ │
│ │ • Cross-layer connections │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ PHASE 2: ARCHITECTURE ─────────────────────────────────────────────── │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ DOMAIN ARCHITECTURE TABLE │ │
│ │ • 8–15 domains defined │ │
│ │ • Semantic triggers per domain │ │
│ │ • Query examples per domain │ │
│ │ • Cross-domain relationships │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ⚠️ CHECKPOINT │
│ │ │
│ ▼ │
│ PHASES 3–6: CREATION & OPTIMISATION ───────────────────────────────── │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ FINAL DELIVERABLE │ │
│ │ │ │
│ │ {system-name}/ │ │
│ │ ├── INSTRUCTIONS.md (4,000–8,000 chars) │ │
│ │ ├── operational/ │ │
│ │ │ └── [workflow files] (2,000–5,000 chars each) │ │
│ │ ├── reference/ │ │
│ │ │ └── [knowledge files] (2,000–5,000 chars each) │ │
│ │ ├── supporting/ │ │
│ │ │ ├── quick-reference.md │ │
│ │ │ └── terminology-index.md │ │
│ │ ├── TEST-MATRIX.md │ │
│ │ └── README.md │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────┘
Claude Projects uses Contextual Retrieval, which combines two search mechanisms:
┌──────────────────────────────────────────────────────────────────────────┐
│ HYBRID RETRIEVAL SYSTEM │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ USER QUERY: "How do I configure OAuth?" │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ CLAUDE GENERATES SEARCH QUERY │ │
│ │ (Influenced by Instructions) │ │
│ │ │ │
│ │ Generated: "OAuth configuration setup identity provider" │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────┴───────────────┐ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ EMBEDDING SEARCH │ │ BM25 KEYWORD SEARCH │ │
│ │ │ │ │ │
│ │ Converts query to │ │ Matches exact terms │ │
│ │ vector, finds │ │ like "OAuth" and │ │
│ │ semantically │ │ "configuration" │ │
│ │ similar chunks │ │ │ │
│ │ │ │ │ │
│ │ Ranking: │ │ Ranking: │ │
│ │ 1. Chunk A (#2) │ │ 1. Chunk B (#3) │ │
│ │ 2. Chunk B (#5) │ │ 2. Chunk A (#7) │ │
│ │ 3. Chunk C (#8) │ │ 3. Chunk D (#1) │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │ │ │
│ └───────────────┬───────────────┘ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ RANK FUSION (RRF) │ │
│ │ │ │
│ │ Combines rankings — chunks scoring well on BOTH signals win: │ │
│ │ │ │
│ │ Final Ranking: │ │
│ │ 1. Chunk B (Embed #5 + BM25 #3 = balanced) ✓ RETRIEVED │ │
│ │ 2. Chunk A (Embed #2 + BM25 #7 = balanced) ✓ RETRIEVED │ │
│ │ 3. Chunk D (Embed #15 + BM25 #1 = imbalanced) ✗ Lower rank │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────┘
The system employs four complementary optimisation strategies:
| Strategy | Target Signal | Technique | Implementation |
|---|---|---|---|
| Semantic Hedging | Embeddings | Include synonyms, variations, question echoes | Primary term + 2–3 variations in first 500 chars |
| BM25 Optimisation | Keywords | Distribute exact terms throughout | Key term every 500–800 characters |
| Chunk Resilience | Both | Write paragraphs that survive arbitrary splits | Paragraph autonomy, inverted pyramid structure |
| Rank Fusion Balance | Combined | Score well on both signals, not just one | Natural prose with distributed technical terms |
┌──────────────────────────────────────────────────────────────────────────┐
│ CHUNK RESILIENCE PATTERN │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ ❌ BAD: Context-dependent paragraph │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ "To do this, follow these steps. First, go to the settings. │ │
│ │ Then find the authentication option. Click on it." │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ Problem: If retrieved alone, "this" has no referent │
│ │
│ ✓ GOOD: Self-contained paragraph │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ "To configure OAuth authentication in the Admin Console, │ │
│ │ navigate to Settings > Authentication > OAuth. OAuth │ │
│ │ configuration requires credentials from your identity provider." │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ Solution: Context in every paragraph, keywords distributed │
│ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ INVERTED PYRAMID STRUCTURE │ │
│ │ │ │
│ │ Sentence 1: THE ANSWER (most important) │ │
│ │ Sentence 2: Essential context │ │
│ │ Sentence 3+: Supporting details │ │
│ │ │ │
│ │ If chunk gets truncated, key information was already delivered │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────┐
│ COMPONENT DEPENDENCY GRAPH │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ │
│ │ INSTRUCTIONS │ │
│ │ (System Prompt) │ │
│ └────────┬────────┘ │
│ │ routes to │
│ ┌─────────────┼─────────────┐ │
│ ▼ ▼ ▼ │
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
│ │ PHASES │ │ MECHANICS │ │ TEMPLATES │ │
│ │ (7 files) │ │ (5 files) │ │ (5 files) │ │
│ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ │
│ │ │ │ │
│ │ uses │ informs │ structures │
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────┐ │
│ │ OUTPUT SYSTEM │ │
│ │ ┌───────────────────────────────────┐ │ │
│ │ │ {system-name}/ │ │ │
│ │ │ ├── INSTRUCTIONS.md │ │ │
│ │ │ ├── operational/ │ │ │
│ │ │ ├── reference/ │ │ │
│ │ │ ├── supporting/ │ │ │
│ │ │ └── TEST-MATRIX.md │ │ │
│ │ └───────────────────────────────────┘ │ │
│ └─────────────────────────────────────────┘ │
│ │ │
│ │ verified by │
│ ▼ │
│ ┌───────────────┐ │
│ │ QUALITY FILES │ │
│ │ (2 files) │ │
│ └───────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────┘
| Component | Depends On | Used By | Purpose |
|---|---|---|---|
| Instructions | — | All phases | Orchestrates execution, routes queries |
| phase-0 | Instructions | phase-1 | Classification methodology |
| phase-1 | phase-0 output | phase-2 | Extraction methodology |
| phase-2 | phase-1 output | phase-3, phase-4 | Architecture design |
| phase-3 | phase-2 output, routing-template | Output system | Instructions creation |
| phase-4 | phase-2 output, domain-templates | Output system | Domain file creation |
| phase-5 | phase-4 output, mechanics files | Output system | Optimisation pass |
| phase-6 | All outputs, test-template | Final delivery | Verification |
| retrieval-mechanics | — | All phases | Explains retrieval system |
| chunk-resilience | retrieval-mechanics | phase-4, phase-5 | Writing guidelines |
| semantic-hedging | retrieval-mechanics | phase-4, phase-5 | Embedding optimisation |
| bm25-optimisation | retrieval-mechanics | phase-4, phase-5 | Keyword optimisation |
| rank-fusion | semantic + bm25 | phase-5 | Balance strategy |
| anti-patterns | — | All phases | Failure prevention |
| quality-gates | All deliverables | phase-6, delivery | Final checklist |
| # | Principle | Description | Implementation |
|---|---|---|---|
| 1 | Conductor, Not Container | Instructions route to content, don't hold it | Target 4,000–8,000 char Instructions |
| 2 | Universal Decomposition | Both operational AND reference content become files | Operational/ and Reference/ directories |
| 3 | Chunk Resilience | Content survives arbitrary splits | Paragraph autonomy, distributed keywords |
| 4 | Hybrid Optimisation | Balance embedding and BM25 signals | Natural prose with technical terms |
| 5 | Concrete Over Abstract | Examples beat patterns | 10–15 specific query examples in Instructions |
| 6 | Checkpoint Discipline | Get user approval before major work | Mandatory pause after Phase 2 |
| 7 | Observable Verification | Test retrieval, don't assume it works | 25+ query test matrix |
The system documents 16 anti-patterns across four categories:
Instruction Anti-Patterns (3):
- Abstract query instructions (no concrete examples)
- No vague query handling
- No knowledge gap handling
Conductor Anti-Patterns (3): 4. Conductor contains sheet music (Instructions too long) 5. Orphaned operational file (not routed to) 6. Orphaned reference file (not routed to)
Operational File Anti-Patterns (4): 7. Context-free operational steps 8. Broken cross-layer connection 9. Descriptive operational content (passive voice) 10. Vague decision criteria
Reference File Anti-Patterns (6): 11. Context-free paragraphs 12. Front-loaded keywords 13. Keyword deserts 14. Giant sections (>4,000 chars) 15. Cross-referential content 16. Paraphrased error messages
The system defines 7 quality gates that must pass before delivery:
┌──────────────────────────────────────────────────────────────────────────┐
│ QUALITY GATE CHECKLIST │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ Gate 1: Instructions Quality │
│ ├── □ System overview present (2–3 sentences) │
│ ├── □ Query example table (10–15 concrete mappings) │
│ ├── □ Vague query handling section │
│ ├── □ Knowledge gap handling section │
│ ├── □ Domain map covering all domains │
│ └── □ Size: 4,000–8,000 characters │
│ │
│ Gate 2: Domain File Quality │
│ ├── □ Opening has question echoes + key terms │
│ ├── □ Every paragraph opens with context │
│ ├── □ Keywords distributed every 500–800 chars │
│ ├── □ Inverted pyramid structure │
│ ├── □ No cross-references │
│ └── □ File size: 2,000–6,000 characters │
│ │
│ Gate 3: Supporting Files Quality │
│ ├── □ quick-reference.md present │
│ └── □ terminology-index.md present │
│ │
│ Gate 4: Test Matrix Quality │
│ ├── □ 25+ test queries │
│ ├── □ All 5 query types represented │
│ └── □ Expected behaviour documented │
│ │
│ Gate 5: Coverage │
│ ├── □ 100% source topics represented │
│ └── □ Knowledge gaps documented │
│ │
│ Gate 6: Anti-Pattern Verification │
│ ├── □ No instruction anti-patterns │
│ └── □ No domain file anti-patterns │
│ │
│ Gate 7: Deployment Package │
│ ├── □ Correct directory structure │
│ └── □ README with deployment instructions │
│ │
│ ═══════════════════════════════════════════════════════════════════ │
│ ALL GATES MUST PASS BEFORE DELIVERY │
└──────────────────────────────────────────────────────────────────────────┘
| Type | Purpose | Example | Pass Criteria |
|---|---|---|---|
| Direct Keyword | Test BM25 matching | "Configure OAuth" | Exact term chunks retrieved |
| Semantic Variation | Test embeddings | "Let users sign in with Google" | Semantically related chunks retrieved |
| Troubleshooting | Test verbatim errors | "error: invalid_client" | Error-specific chunks retrieved |
| Vague | Test clarification handling | "It's not working" | Claude asks for specifics |
| Out-of-Scope | Test gap handling | "[uncovered topic]" | Claude acknowledges gap |
{system-name}/
├── INSTRUCTIONS.md # Routing layer
│ Target: 4,000–8,000 characters
│ Contains: Purpose, index, routing, edge cases
│
├── operational/ # HOW the system works
│ ├── [phase-1].md # 2,000–5,000 chars each
│ ├── [phase-2].md
│ ├── [decision-framework].md
│ └── [constraints].md
│
├── reference/ # WHAT the system knows
│ ├── [methodology].md # 2,000–5,000 chars each
│ ├── [examples].md
│ ├── [patterns].md
│ └── [troubleshooting].md
│
├── supporting/
│ ├── quick-reference.md # Most common lookups
│ └── terminology-index.md # Terms → domains
│
├── TEST-MATRIX.md # 25+ verification queries
│
└── README.md # Deployment instructions
┌──────────────────────────────────────────────────────────────────────────┐
│ DEPLOYMENT WORKFLOW │
├──────────────────────────────────────────────────────────────────────────┤
│ │
│ Step 1: Create Claude Project │
│ ├── Navigate to Claude Projects │
│ ├── Create new project │
│ └── Name appropriately │
│ │
│ Step 2: Set Instructions │
│ ├── Copy INSTRUCTIONS.md content │
│ ├── Paste into Project Instructions field │
│ └── Save │
│ │
│ Step 3: Upload Knowledge Files │
│ ├── Upload all files from operational/ │
│ ├── Upload all files from reference/ │
│ └── Upload all files from supporting/ │
│ │
│ Step 4: Verify │
│ ├── Run TEST-MATRIX.md queries │
│ ├── Expand project_knowledge_search output │
│ ├── Verify correct files retrieved │
│ └── Iterate if failures detected │
│ │
└──────────────────────────────────────────────────────────────────────────┘
| Version | Key Changes |
|---|---|
| v3.4 | YAML frontmatter, tiered source authority, cross-cutting domains, prompt templates, supporting file templates |
| v3.3 | Three-layer architecture, conductor pattern, universal decomposition |
| v3.2 | Prompt classification, hybrid templates |
| v3.1 | Chunk resilience, concrete query examples |
| v3.0 | Initial chunk-aware architecture |
Full changelog with migration guides: See CHANGELOG.md Detailed release notes: See docs/release-notes/
| Term | Definition |
|---|---|
| BM25 | Best Matching 25 — keyword-based ranking algorithm |
| Chunk | ~400–800 token segment of content indexed for retrieval |
| Conductor Pattern | Design where Instructions route to content rather than containing it |
| Contextual Retrieval | Anthropic's hybrid retrieval system combining embeddings and BM25 |
| Embedding | Vector representation of text capturing semantic meaning |
| Inverted Pyramid | Writing structure with most important information first |
| Operational Knowledge | HOW content — processes, workflows, decisions |
| Paragraph Autonomy | Writing paragraphs that make sense if retrieved alone |
| Rank Fusion | Combining multiple ranking signals into a final score |
| Reference Knowledge | WHAT content — methodology, examples, patterns |
| Semantic Hedging | Including synonyms and variations to improve embedding matches |
| Three-Layer Architecture | Instructions + Operational + Reference decomposition |
Document generated by Progressive Disclosure Knowledge System Generator v3.4