Skip to content

feat(mcp/ocs): add file_path mode to ocs_upload_collection_files (close Phase 5 b64 wedge)#360

Open
jjackson wants to merge 1 commit into
mainfrom
emdash/ocs-upload-file-path
Open

feat(mcp/ocs): add file_path mode to ocs_upload_collection_files (close Phase 5 b64 wedge)#360
jjackson wants to merge 1 commit into
mainfrom
emdash/ocs-upload-file-path

Conversation

@jjackson
Copy link
Copy Markdown
Owner

Summary

Two consecutive ace:ocs-setup dispatches on leep-paint-collection/20260517-1515 hit stream-idle timeouts (~30 min / 49 calls + ~114 min / 44 calls) without writing any Drive artifacts. Session-log bisect pinned the cause: agent base64-encoded its ~67 KB RAG content pack on disk, then Read the .b64 chunks back into its own context to emit them as the content field of the ocs_upload_collection_files tool_use input. Generating 100s of KB of b64 as output tokens stalls model generation either mid-emit or on the next turn. Pure output-token budget exhaustion — no OCS slowness, no auth churn, no QA loop, no indexing wedge.

Fix

mcp/ocs-server.tsocs_upload_collection_files extended to accept file_path as an alternative source per file. The MCP reads the file server-side via fs/promises and b64-encodes internally. The agent never holds the encoded form in its context. Exactly-one-source-per-file invariant enforced server-side: each file MUST supply EXACTLY ONE of content (legacy inline b64) or file_path (absolute filesystem path); mixed or missing sources fail fast with a named error citing the offending file.

Refactor: file-decoding logic moved into decodeUploadCollectionFileSource, exported from mcp/ocs-server.ts for unit-testability.

Tests

7 new vitest cases in test/mcp/ocs/unit/upload-collection-files-decoder.test.ts:

  • file_path reads UTF-8 text verbatim
  • file_path reads arbitrary binary verbatim
  • content decodes inline b64 (legacy mode preserved for tiny payloads + back-compat)
  • missing source → named error
  • both sources → named error
  • error message names offending file (debuggability)
  • ENOENT propagates cleanly

All 12 tests in test/mcp/ocs pass. tsc --noEmit clean.

Skill-side guidance

Phase 5 ocs-content-pack and any future skill calling this atom SHOULD use file_path for any payload > ~1KB. Pattern:

// Write content to a tmp file via Bash. Never Read it back.
await Bash('echo "$content" > /tmp/leep-rag/pdd-summary.md');
// Or: drive_download_binary into a tmp path for files already on Drive.
await Bash('drive_download_binary ... | base64 -d > /tmp/leep-rag/pdd.md');

// Then upload by reference:
await ocs_upload_collection_files({
  collection_id: 123,
  files: [{
    name: 'pdd.md',
    file_path: '/tmp/leep-rag/pdd.md',  // absolute path
    mime_type: 'text/markdown',
  }],
});

DO NOT Read the .b64 files. DO NOT Read the original markdown files into context if all you're going to do is re-emit them through the upload tool — that's the wedge.

Docs

  • docs/learnings/2026-05-19-ocs-upload-b64-context-wedge.md — full bug doc with prior-framing refutations (RAG indexing wedge, per-prompt QA loop, auth churn, OCS slowness — all wrong) and the session-log proof from both subagent transcripts.
  • docs/learnings/2026-05-12-boundary-probe-registry.md — new Shipped probe row + a generalized pending probe ("every MCP atom whose input schema takes a string carrying > ~10KB of payload should have a _path companion"). Existing examples of the pattern done correctly: commcare_upload_multimedia.file_bytes_path, commcare_patch_xform.new_xform_xml_path. Still missing: drive_upload_binary.content, drive_create_file.content (high-value follow-ups).

Recovery for leep-paint-collection/20260517-1515

After this PR merges + /ace:update:

  1. Delete the partial chatbot from the prior wedge attempts (experiment_id=12193, name "ACE - leep-paint-collection (run 20260517-1515)") via ocs_delete_chatbot — clean restart.
  2. Re-dispatch Phase 5 with explicit instruction to use file_path mode (the new SKILL.md guidance ships in this PR; existing ocs-content-pack SKILL.md should be updated in a follow-up PR to bake the pattern in unconditionally).
  3. The full Phase 5 should complete in ~5-10 min (the expected wall time) instead of timing out.

Test plan

  • npx vitest run test/mcp/ocs/ — 12 pass
  • npx tsc --noEmit — clean
  • After merge + /ace:update, re-run Phase 5 of leep-paint-collection/20260517-1515 using file_path mode and confirm completion in expected wall time.
  • Follow-up PR: update skills/ocs-content-pack/SKILL.md to mandate file_path mode for any payload > 1KB.

🤖 Generated with Claude Code

Two consecutive Phase 5 (ocs-setup) dispatches on leep-paint-collection
hit stream-idle timeouts (~30 min / 49 calls + ~114 min / 44 calls)
without writing any Drive artifacts. Session-log bisect pinned the
cause: the agent base64-encoded its ~67 KB RAG content pack on disk,
then `Read` the .b64 chunks back into context to emit them as the
`content` field of the tool_use input. Generating 100s of KB of b64 as
output tokens stalled model generation either mid-emit or on the next
turn. Pure output-token budget exhaustion — no OCS slowness, no auth
churn, no QA loop, no indexing wedge.

Fix: extend `ocs_upload_collection_files` to accept `file_path` per
file. MCP reads the bytes server-side via fs/promises and b64-encodes
internally — agent never holds the encoded form. Each file MUST supply
EXACTLY ONE of `content` (legacy inline b64) or `file_path` (absolute
filesystem path); mixed or missing sources fail fast with a named
error citing the offending file.

Refactor: file-decoding logic moved into `decodeUploadCollectionFileSource`,
exported from mcp/ocs-server.ts for unit-testability.

7 new vitest cases in test/mcp/ocs/unit/upload-collection-files-decoder.test.ts:
- file_path reads UTF-8 text verbatim
- file_path reads arbitrary binary verbatim
- content decodes inline b64 (legacy preserved)
- missing source → named error
- both sources → named error
- error message names offending file (debuggability)
- ENOENT propagates cleanly

docs/learnings/2026-05-19-ocs-upload-b64-context-wedge.md documents the
bug, the four prior-framing refutations, the session-log proof, the
generalized class ("MCP atom accepts large payload as string param")
and the audit candidates that still need the same fix
(drive_upload_binary, drive_create_file, drive_update_file).

docs/learnings/2026-05-12-boundary-probe-registry.md updated with the
new Shipped probe + a generalized pending probe row.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jjackson jjackson enabled auto-merge May 19, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant