Skip to content

feat: platform-brokered MCP tools (keep the gateway key out of the agent)#320

Open
ishaan-berri wants to merge 7 commits into
mainfrom
litellm_platform-mcp-broker
Open

feat: platform-brokered MCP tools (keep the gateway key out of the agent)#320
ishaan-berri wants to merge 7 commits into
mainfrom
litellm_platform-mcp-broker

Conversation

@ishaan-berri
Copy link
Copy Markdown
Contributor

What

Make external MCP tools (Linear, GitHub, Slack) platform-brokered so the gateway key (LITELLM_API_KEY) is never injected into the agent. Today both harnesses set Authorization: Bearer ${LITELLM_API_KEY} on each external MCP and talk straight to the gateway — i.e. a powerful key lives in the agent's MCP config.

How

One platform MCP surface, key held server-side:

  • New broker route …/sessions/{session_id}/mcp/{server} — authenticates with a scoped per-agent token (new "mcp" scope) or master key, resolves session→agent, enforces that {server} is in the agent's mcp_servers allow-list, then proxies the MCP JSON-RPC to ${LITELLM_API_BASE}/mcp/{server} with the gateway key server-side, streaming the response back.
  • resolveAgentMcpServers now points specs at that broker URL and attaches a minted mcp-scoped token (auth_token) instead of the gateway key.
  • claude-agent-sdk harness uses the per-server auth_token, falling back to LITELLM_API_KEY only for legacy specs.

The agent only ever holds a scoped LAP token that works through the platform for its servers — never the gateway key.

Evidence (local, claude-agent-sdk harness)

# allow-list: server NOT attached to the agent
github(not attached) = 403

# agent's Linear MCP -> broker URL, auth = LAP token (not sk-WA):
TOOL: mcp__linear__linear-list_teams  completed
OUT:  {"teams":[{"id":"c30b931c…","name":"LiteLLM-prod",…}],"hasNextPage":false}

# broker proxied to the gateway with the key held server-side (fired on each call):
[mcp-broker] session=… agent=… server=linear -> gateway (key held server-side)

# direct broker curl (tools/list) -> 41 linear tools

Scope / follow-ups

  • Covers the claude-agent-sdk harness (per-session). The opencode-inline harness is process-global (gen-mcp-config.mjs bakes opencode.json at boot), so it needs the session-scoped variant (session_id at call time) — follow-up, same shape as the session-scoped memory work.
  • Tools only: the harness still uses a key for the LLM itself (ANTHROPIC_*); brokering model calls is a separate change.
  • No DB/schema changes.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 25, 2026

Greptile Summary

This PR introduces a platform-brokered MCP layer that keeps the LiteLLM gateway key entirely server-side: agents now receive a scoped 24h mcp-scope token pointing at the new /sessions/{id}/mcp/{server} broker, which authenticates first, enforces the agent's server allow-list, then proxies to the gateway with the real key.

  • New broker route (sessions/[session_id]/mcp/[server]/route.ts): auth runs before any DB lookup, allow-list is enforced via a 60s-TTL cache of gateway server IDs (fails closed on empty cache), and only content-type / mcp-session-id headers are forwarded from the upstream response.
  • Session creation (agents/[agent_id]/session/route.ts): resolveAgentMcpServers now mints a single mcp-scoped access token per session (24h TTL to match sandbox lifetime) and rewrites all MCP specs to point at the broker URL; mcpWarning is now logged in both the pod and inline session paths.
  • Harness (harnesses/claude-agent-sdk/src/server.ts): uses the per-server auth_token when present, falling back to LITELLM_API_KEY for legacy specs.

Confidence Score: 5/5

The change is safe to merge; the gateway key isolation holds end-to-end and the broker's auth/allow-list checks are correctly ordered.

The core security invariant — gateway key never reaches the agent — is correctly implemented. Auth runs before DB access, the allow-list check uses signed server IDs (not spoofable names), and upstream response headers are filtered. The two observations are design-level notes: the ttl_sec override uses a field documented as test-only, and the minted token is agent-scoped rather than session-scoped, which is within the intended threat model but slightly broader than necessary.

The broker route and session route.ts are the highest-impact new code; both work correctly but have the design notes above worth considering before the feature is extended further.

Important Files Changed

Filename Overview
src/app/api/v1/managed_agents/sessions/[session_id]/mcp/[server]/route.ts New broker route: auth-first, allow-list enforcement, gateway-key proxy, response streaming. Module-level cache works correctly. Token is agent-scoped but not session-scoped (P2).
src/app/api/v1/managed_agents/agents/[agent_id]/session/route.ts resolveAgentMcpServers now mints a 24h MCP-scoped token and redirects specs to the broker URL. Uses ttl_sec override that the interface documents as test-only (P2). mcpWarning is now logged in both call sites.
src/server/auth/agent-token.ts Adds "mcp" to AgentScope union. No other changes; existing sign/verify/parseClaims logic is unchanged.
src/server/types.ts Adds optional auth_token field to HarnessMcpServerSpec with clear documentation. No breaking changes.
harnesses/claude-agent-sdk/src/server.ts Harness now prefers per-server auth_token over LITELLM_API_KEY when present, with correct fallback for legacy specs. Type updated to include auth_token.

Reviews (3): Last reviewed commit: "fix(mcp-broker): session-lifetime MCP to..." | Re-trigger Greptile

Comment thread src/app/api/v1/managed_agents/sessions/[session_id]/mcp/[server]/route.ts Outdated
Comment thread src/app/api/v1/managed_agents/agents/[agent_id]/session/route.ts Outdated
Comment thread src/app/api/v1/managed_agents/sessions/[session_id]/mcp/[server]/route.ts Outdated
Comment thread src/app/api/v1/managed_agents/agents/[agent_id]/session/route.ts
@ishaan-berri
Copy link
Copy Markdown
Contributor Author

@greptile review

@ishaan-berri
Copy link
Copy Markdown
Contributor Author

@greptile review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant