CognitiveRAG

CognitiveRAG is a multi-layer memory and context-construction backend for OpenClaw. It helps an agent remember what matters, choose better evidence for the current turn, reuse prior reasoning and promoted knowledge, incorporate local files and web evidence, and explain why specific context was selected.

Why this matters

Most agent systems fail in the same places:

• they forget important earlier work
• they stuff the prompt with too much weak context
• they fail to reuse good prior reasoning
• they blur together temporary evidence and durable knowledge
• they cannot clearly explain why a given piece of context was included

CognitiveRAG is built to fix those problems.

What this does for an OpenClaw agent

CognitiveRAG improves the things that matter in real use:

• Memory continuity: important conclusions, workflows, and prior reasoning stay reusable instead of disappearing into long chats.
• Better evidence selection: the system tries to assemble the best context under budget, not just the nearest or newest text.
• Reasoning reuse: useful prior reasoning and promoted knowledge can be brought back when relevant.
• First-class evidence lanes: local corpus data, promoted memory, episodic memory, reasoning memory, and web evidence can all contribute in controlled ways.
• Explainable context construction: the system can expose what was selected, what was dropped, and why.
• Bounded discovery: it can explore adjacent evidence without turning into uncontrolled search.

Why this is different from ordinary RAG

CognitiveRAG is not just:

• vector search over chunks
• chat-memory summarization
• top-k prompt stuffing
• or a bigger prompt wrapped around a chat interface

It is a backend intelligence layer for OpenClaw that combines multi-layer memory, retrieval lanes, typed candidates, budget-aware selection, promoted knowledge, reasoning reuse, and explainable context assembly.

Core benefits

• Better turn quality under token limits.
• Better long-session reliability and recoverability.
• Better reuse of high-value prior knowledge.
• Better grounding across local and web evidence.
• Better auditability of context decisions.

Current architecture

Primary backend surfaces:

• API and routes: CognitiveRAG/main_server.py, CognitiveRAG/api/routes/*
• Retrieval lanes and routing: CognitiveRAG/crag/retrieval/*
• Context selection and explanation outputs: CognitiveRAG/crag/context_selection/*
• Session memory compaction/recovery: CognitiveRAG/session_memory/*
• Promoted/reasoning/web/skill memory layers: CognitiveRAG/memory/*, CognitiveRAG/web_memory/*, CognitiveRAG/skill_memory/*

Memory taxonomy

Current layers include:

• session and episodic memory
• promoted memory
• reasoning memory
• skill execution and evaluation memory
• web evidence and promoted web memory
• corpus lexical and semantic retrieval lanes

Markdown mirrors are integration artifacts, not the full memory system.

What is already real

• retrieval lanes
• context-selection foundation
• promoted memory durability
• reasoning memory and reuse
• web evidence and promoted web memory
• explanation artifacts
• bounded discovery foundations
• skill-memory execution/evaluation foundations
• structured session export with message parts plus compaction lineage/raw-snapshot/quarantine sections

What is partial

Epic B parity status:

• B1 typed candidate coverage: FULLY_BUILT
• B2 scoring and token-budget rules: FULLY_BUILT
• B3 contradiction/compatibility filtering: FULLY_BUILT
• B4 reorder and explanation output: FULLY_BUILT

Current B3 behavior includes deterministic contradiction threshold drops, heuristic pairwise compatibility gating, a runtime-configured transformers-backed NLI path, and deterministic fallback when the backend is unavailable. The backend now includes explicit optional dependency wiring for real NLI mode (pip install .[nli], including transformers + torch), machine-readable runtime fallback diagnostics, and an environment-gated real transformers compatibility test path that runs only when local model assets are available. Use tools/check_b3_nli_env.py for a machine-readable environment feasibility probe before attempting real-NLI runtime proof. Epic B parity is closed in repo code/tests; Epic C is now in progress. Current Epic C implementation in this repo includes:

• selector/lane metrics surfacing (including latency and discovery counters)
• C2/C3 benchmark suite outputs with stable JSON + Markdown reporting
• fast retrieval benchmark integration into a single Epic C runner (tools/run_epic_c_suite.py)

What is later

• graph layer and graph-assisted retrieval
• deeper trust/promotion lifecycle expansion
• federation/hive-style extensions

Graph is planned, not current implementation.

Setup / run / test

cd /home/ictin_claw/.openclaw/workspace/CognitiveRAG
python3 -m pip install -r CognitiveRAG/requirements.txt
python3 -m pip install ".[nli]"   # optional, enables real transformers-backed compatibility path when local model assets exist
python3 -m pip install "./CognitiveRAG[nli]"   # equivalent optional install path when using nested package entrypoint
python3 -m uvicorn CognitiveRAG.main_server:app --reload --host 127.0.0.1 --port 8080
python3 -m pytest CognitiveRAG/tests/context_selection -q
python3 -m pytest CognitiveRAG/tests/retrieval -q
python3 -m pytest CognitiveRAG/tests -q
python3 tools/check_b3_nli_env.py
./tools/run_b3_validation_seq.sh
./tools/run_epic_c_bootstrap_seq.sh
./tools/run_epic_c_suite.py
python3 tools/run_phase3_live_slice.py --model gpt-5-mini
python3 tools/run_phase3_closure_battery.py --model gpt-5-mini
python3 tools/run_phase3_supplemental_readback.py

Phase 3 runtime note: closure authority is defined in docs/phase3_acceptance_matrix.md. Use tools/run_phase3_closure_battery.py for Phase-3 closure-grade runtime checks and treat the plugin-heavy benchmark path as stress telemetry unless the matrix explicitly promotes it. Latest closure-grade artifacts (2026-04-12):

• forensics/20260412T_phase3_closure_battery_pkg3/summary.json (authoritative battery pass)
• forensics/20260412T_phase3_readback_pkg3/summary.json (supplemental runtime readback pass)

Phase 4 runtime note: durable promoted-memory closure authority is defined in docs/phase4_acceptance_matrix.md. Use tools/run_phase4_promoted_closure.py as the authoritative runtime battery for promotion bridge/readback/truthfulness checks. Latest Phase-4 artifact in this package:

• forensics/20260412T_phase4_promoted_closure_pkg8/summary.json