Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ All notable changes to RecallForge will be documented in this file.
## [Unreleased]

- Replaced the tiny UAT video clips with compact episodic-memory fixtures, richer transcript sidecars, related artifact metadata, and regression coverage for the video corpus.
- Added `memory_add_conversation` so conversation threads ingest as canonical parent memories with turn-level child memories and standard memory rollups.

## [0.2.1] — 2026-05-17

Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ One query. Any modality. All local.
| Audio transcript ingest | ✅ | ❌ | ❌ | ❌ | ❌ |
| Document ingest (PDF/DOCX/PPTX) | ✅ | ❌ | ❌ | ❌ | ❌ |
| Built-in reranking | ✅ Multimodal | ❌ | ❌ | ✅ ColBERT | ✅ Modules |
| MCP-native | ✅ 21 tools | ❌ | ❌ | ❌ | ❌ |
| MCP-native | ✅ 24 tools | ❌ | ❌ | ❌ | ❌ |
| 100% local | ✅ | ✅ | ⚠️ Cloud default | ✅ | ✅ Docker |
| Apple Silicon optimized | ✅ MLX 4-bit | ❌ | ❌ | ❌ | ❌ |
| Cloud option | ❌ | ✅ | ✅ | ✅ | ✅ |
Expand Down Expand Up @@ -146,7 +146,7 @@ Run over HTTP/SSE:
recallforge serve --http --host 127.0.0.1 --port 7433 --mode embed
```

RecallForge now exposes **21 MCP tools** across search, ingest, memory, collection admin, and runtime config. HTTP/SSE mode also exposes `/health`, `/sse`, and `/messages/`.
RecallForge now exposes **24 MCP tools** across search, ingest, memory, collection admin, and runtime config. HTTP/SSE mode also exposes `/health`, `/sse`, and `/messages/`.

See [docs/mcp-tools.md](docs/mcp-tools.md) for the full tool reference.

Expand All @@ -161,7 +161,7 @@ See [docs/mcp-tools.md](docs/mcp-tools.md) for the full tool reference.

## How it works

RecallForge encodes text, images, video frames, documents, and audio transcripts into the same 2048-dimensional vector space using Qwen3-VL. This means "find notes about this diagram" works whether the diagram is text, an image, or a frame from a video. A 3-stage pipeline handles the rest:
RecallForge encodes text, images, video frames, documents, conversation turns, and audio transcripts into the same 2048-dimensional vector space using Qwen3-VL. This means "find notes about this diagram" works whether the diagram is text, an image, a conversation thread, or a frame from a video. A 3-stage pipeline handles the rest:

```mermaid
graph TD
Expand All @@ -170,13 +170,15 @@ graph TD
Imgs[🖼️ Images]
Vids[🎬 Video]
Aud[🎙️ Audio + Transcript]
Conv[Conversation Turns]
end

subgraph RecallForge Ingest
Docs --> TxtExt[Text Extractor]
Imgs --> VLM[Qwen3-VL Encoder]
Vids --> Frame[Frame & Audio Extractor]
Aud --> TxtExt
Conv --> TxtExt
Frame --> VLM
TxtExt --> VLM
end
Expand Down Expand Up @@ -274,7 +276,7 @@ src/recallforge/
│ └── lancedb_backend.py # LanceDB + Tantivy FTS
├── cache.py # LRU embedding cache
├── search.py # Hybrid search pipeline (BM25 + vector + RRF)
├── server.py # MCP server (21 tools, stdio + HTTP/SSE)
├── server.py # MCP server (24 tools, stdio + HTTP/SSE)
├── documents.py # PDF/DOCX/PPTX extraction
├── video.py # Frame/transcript extraction
├── audio.py # Transcript-first audio ingest
Expand Down
12 changes: 11 additions & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,16 @@ id | collection | file_path | title | content_hash | content_type | active
created_at | updated_at
```

Conversation memories use the same parent/child layout as media-derived memories:

```
conversation root path
├──> root text memory with summary, participants, and excerpts
└──> turn child memories (`path::turn:0001`, `path::turn:0002`, ...)
```

All turn children share the root `memory_id` and `memory_root_path`, so matching turns strengthen the parent conversation result through the standard memory rollup path.

**content** (bodies, content-addressed)
```
hash | doc | content_type | created_at
Expand Down Expand Up @@ -183,7 +193,7 @@ backend = recallforge.get_backend()
### 5. MCP Server (`src/recallforge/server.py`)

```
Tools: 21 MCP tools across search, ingest, memory, collection admin, batch, and runtime config
Tools: 24 MCP tools across search, ingest, memory, collection admin, batch, and runtime config
Transport: stdio (default) or HTTP/SSE (`/health`, `/sse`, `/messages/`)
Startup: backend.warm_up() for predictable latency
Signals: SIGTERM/SIGINT graceful shutdown
Expand Down
5 changes: 5 additions & 0 deletions docs/MEMORY_POLICY.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ RecallForge stores complex files as a root memory plus child assets:
- Video memory: one root video memory, with frame children and transcript children when available.
- Document memory: one root document memory, with section/page children and OCR siblings when available.
- Audio memory: one root audio memory, with timestamped transcript children from sidecar transcripts.
- Conversation memory: one root text memory with a summary/participant overview, plus one child memory per turn.

The root memory uses `memory_role="root"` and child assets use `memory_role="child"` with `memory_root_path` pointing back to the root.

Expand All @@ -42,3 +43,7 @@ The boost is intentionally modest: it can break ties and surface important recen
Audio support is transcript-first. Put a `.srt`, `.vtt`, `.txt`, or `.transcript.json` sidecar next to the audio file. RecallForge indexes the audio as a root `content_type="audio"` memory and indexes transcript segments as text child memories.

Raw audio transcription and dedicated audio encoders are not part of the shipped v0.3.0 policy.

## Conversations

Use `memory_add_conversation` when an agent or app wants to persist a thread. RecallForge stores the parent at the supplied `path` and stores each turn at `path::turn:0001`, `path::turn:0002`, and so on. All turns share the parent `memory_id`, so if several turns match a query, search and explanation output roll them up into the parent conversation with evidence paths.
185 changes: 182 additions & 3 deletions docs/mcp-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,11 @@ Example MCP client config (Claude Desktop):

### Memory
- `memory_add`
- `memory_add_conversation`
- `memory_update`
- `memory_delete`
- `memory_get`
- `list_memories`

### Admin / Introspection
- `status`
Expand Down Expand Up @@ -117,7 +120,13 @@ Example MCP client config (Claude Desktop):
"user_id": null,
"session_id": null,
"project_id": null,
"profile": null
"profile": null,
"memory_id": "b4f7...",
"memory_role": "root",
"memory_root_path": "notes/meeting.md",
"memory_hit_count": 2,
"memory_primary_evidence_path": "recallforge://default/notes/meeting.md::turn:0002",
"memory_supporting_paths": []
}
]
}
Expand Down Expand Up @@ -632,6 +641,77 @@ Example MCP client config (Claude Desktop):

---

## memory_add_conversation

**Description:** Add or replace a conversation as a canonical parent memory with turn-level child memories. Matching turns roll up into the parent conversation in `search` and `explain_results`.

**Parameters:**
| Name | Type | Required | Default | Description |
|------|------|----------|---------|-------------|
| path | string | Yes | — | Conversation root path |
| turns | array[object] | Yes | — | Chronological turns or message groups |
| title | string | No | derived from `path` | Conversation title |
| summary | string | No | — | Optional parent summary |
| collection | string | No | server default collection | Collection name |
| user_id | string | No | — | User namespace |
| session_id | string | No | — | Session/thread namespace |
| project_id | string | No | — | Project namespace |
| profile | string | No | — | Profile namespace |
| importance | number | No | — | 0.0 to 1.0 importance score |
| ttl_seconds | integer | No | — | TTL in seconds; `0`/`null` means no expiration |
| tags | array[string] | No | — | Extra tags |

Turn objects accept:

| Name | Type | Required | Description |
|------|------|----------|-------------|
| role | string | No | Role such as `user`, `assistant`, `agent`, `tool`, or `system` |
| speaker | string | No | Persona/person label |
| content | string | Conditionally* | Turn content |
| text | string | Conditionally* | Alias for `content` |
| timestamp | string | No | Timestamp string, ideally ISO 8601 |

\* Each turn must include non-empty `content` or `text`.

**Example Request:**
```json
{
"path": "threads/customer-renewal",
"title": "Customer Renewal Thread",
"summary": "Pricing approval and legal review are the open blockers.",
"turns": [
{ "role": "user", "content": "Can we renew the customer contract before Q3?" },
{ "role": "assistant", "content": "Yes. The renewal plan depends on pricing approval." },
{ "role": "user", "content": "Please remember the pricing risk and legal review." }
],
"tags": ["sales"]
}
```

**Example Response:**
```json
{
"success": true,
"path": "threads/customer-renewal",
"collection": "default",
"hash": "abc123...",
"memory_id": "b4f7...",
"title": "Customer Renewal Thread",
"indexed_turns": 3,
"tags": ["conversation", "turns:3", "role:user", "role:assistant", "sales"],
"operation": "add_conversation"
}
```

**Errors:**
- `INVALID_INPUT`: when `path` is missing, `turns` is empty, or a turn lacks text.
- `BACKEND_ERROR`: when the storage backend does not support conversation memories.
- `INTERNAL_ERROR`: uncaught exceptions.

**Notes:** The root path becomes the stable `memory_id` identity seed. Child turns are stored at `path::turn:0001`, `path::turn:0002`, and so on with the same `memory_id` and `memory_root_path`.

---

## memory_update

**Description:** Update an existing memory entry (same upsert backend path as add).
Expand Down Expand Up @@ -723,6 +803,100 @@ Example MCP client config (Claude Desktop):

---

## memory_get

**Description:** Fetch one canonical memory object by stable `memory_id` or by root `path`.

**Parameters:**
| Name | Type | Required | Default | Description |
|------|------|----------|---------|-------------|
| memory_id | string | Conditionally* | — | Stable memory identifier |
| path | string | Conditionally* | — | Root memory path |
| collection | string | No | server default collection | Collection name |
| user_id | string | No | — | User namespace |
| session_id | string | No | — | Session namespace |
| project_id | string | No | — | Project namespace |
| profile | string | No | — | Profile namespace |

\* Provide either `memory_id` or `path`.

**Example Request:**
```json
{
"path": "threads/customer-renewal",
"collection": "default"
}
```

**Example Response:**
```json
{
"memory_id": "b4f7...",
"collection": "default",
"title": "Customer Renewal Thread",
"path": "threads/customer-renewal",
"content_type": "text",
"summary": "Discussion about renewal timing...",
"children": [
{ "path": "threads/customer-renewal::turn:0001", "content_type": "text", "memory_role": "child" }
],
"snippets": [
{ "path": "threads/customer-renewal::turn:0001", "text": "..." }
]
}
```

**Errors:**
- `INVALID_INPUT`: when neither `memory_id` nor `path` is provided.
- `NOT_FOUND`: when the memory cannot be found.
- `INTERNAL_ERROR`: uncaught exceptions.

---

## list_memories

**Description:** List canonical root memories for a collection or namespace.

**Parameters:**
| Name | Type | Required | Default | Description |
|------|------|----------|---------|-------------|
| collection | string | No | all collections | Collection filter |
| limit | integer | No | 50 | Max memories |
| user_id | string | No | — | User namespace |
| session_id | string | No | — | Session namespace |
| project_id | string | No | — | Project namespace |
| profile | string | No | — | Profile namespace |

**Example Request:**
```json
{
"collection": "default",
"limit": 20
}
```

**Example Response:**
```json
{
"success": true,
"collection": "default",
"count": 1,
"memories": [
{
"memory_id": "b4f7...",
"path": "threads/customer-renewal",
"title": "Customer Renewal Thread",
"summary": "Discussion about renewal timing..."
}
]
}
```

**Errors:**
- `INTERNAL_ERROR`: uncaught exceptions.

---

## status

**Description:** Return server/model/database status.
Expand Down Expand Up @@ -1006,7 +1180,12 @@ Operation object schema:
2. Update with `memory_update` as facts evolve.
3. Retrieve later with `search` + matching namespace filters.

### 3) Configure mode, then ingest
### 3) Persist a conversation thread
1. Call `memory_add_conversation` with a stable thread `path`, optional `summary`, and chronological `turns`.
2. Query with `search` or `explain_results`; matching turns roll up to the parent `memory_id`.
3. Inspect the full memory with `memory_get` when the agent needs turn evidence.

### 4) Configure mode, then ingest
1. Inspect config using `get_config`.
2. Set desired runtime defaults with `set_config` (for mode, default collection, max file size).
3. Run `ingest` without repeating shared defaults.
Expand All @@ -1019,7 +1198,7 @@ Structured errors returned via `_error_response(code, message, details)`:
- `NOT_FOUND`
- Resource is missing (for example, image path does not exist) or capability is unavailable (e.g., unsupported raw video query backend).
- `BACKEND_ERROR`
- Backend/storage operation failed in handler-managed exception path (currently used in `rebuild_fts`).
- Backend/storage capability is unavailable or failed in a handler-managed path.
- `INTERNAL_ERROR`
- Unhandled exception at top-level tool dispatch.

Expand Down
Loading