AiPipe

A high-performance, multi-provider LLM proxy with per-tenant isolation, quality-adjusted routing, and runtime cost tracking.

Used as the smart routing layer for Mission Control.

What it does

• Multi-provider routing: OpenAI, Anthropic, xAI, OpenRouter, MiniMax, Kimi (Moonshot), Google Gemini
• Quality-adjusted cost routing: scores each request on complexity, routes to the cheapest model that can handle it at sufficient quality — not just the cheapest model period
• Per-tenant isolation: each tenant's keys, stats, and cost data are stored separately in SQLite; no cross-tenant data access
• Admin API: manage tenant keys and query stats via a header-authenticated admin endpoint
• In-memory caching: non-streaming responses are cached by provider + model + normalised prompt (5-minute TTL, LRU eviction)
• Reliability: per-model success rate tracking; failing providers are penalised and traffic shifts away automatically
• Metrics: p50/p95/p99 latency, TTFT tracking, per-provider cost and token usage, queue depth

Routing formula

AiPipe scores each request on five signals: prompt length, code presence, keywords, structural complexity, and conversation depth. The combined score drives model selection via a quality-adjusted cost formula:

adjusted_cost = raw_cost / quality_score ^ max(0, complexity - 0.25) * 6

Below complexity 0.25: pure cheapest-model routing. Above 0.25: quality-weighted — higher-quality models earn a cost discount proportional to how complex the task is.

Benchmark result: simple tasks route to gpt-4o-mini (94% cheaper than GPT-4o). Complex reasoning routes to claude-sonnet-4-5. Typical user saves ~50% on LLM costs vs. using a single frontier model for everything.

Full benchmark with cost tables →

Repository layout

Server/          Go server implementation
  cmd/aipipe/    Entry point
  internal/      app, model, provider, util packages
PRDs/            Product requirement docs
Architecture/    Technical design references
Prompts/         Workflow prompt assets

Quick start

Requires Go 1.22+.

# Export at least one provider key
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...

# Run
cd Server
go run ./cmd/aipipe

# Health check
curl http://127.0.0.1:8082/healthz

For production use, copy the env file template and run as a systemd user service:

cp Server/.env.example ~/.config/aipipe/env
# Edit env file with your keys + AIPIPE_ADMIN_SECRET
systemctl --user enable --now aipipe

Admin endpoints

All admin endpoints require X-Admin-Secret header matching AIPIPE_ADMIN_SECRET.

POST /admin/tenants/:id/keys      Upsert provider keys for a tenant
GET  /admin/tenants/:id/stats     Get cost/token stats for a tenant

Mission Control injects X-Tenant-ID on every proxied request to scope routing and stats to the calling tenant.

Test status

cd Server && go test ./... -race -cover

As of February 2026: all tests pass, race-clean.

License

Apache 2.0