Worker LLM concurrency refactor: universal gate + retry + tok/s observability (CA-169)#43

Merged

jstuart0 merged 10 commits into

mainfrom

feat/worker-llm-concurrency

May 7, 2026

Commits on May 6, 2026

feat(worker): Phase 1 — concurrency gate foundations + shared fake provider

jstuart0
and
claude
committed
feat(worker): Phase 2 — wire ProviderGateRegistry through factories, servicers, CLI, and capability endpoint

jstuart0
and
claude
committed
feat(worker): Phase 3 — activate gate + retry; delete hand-rolled retries; load real defaults; raise local caps (atomic per Decision 3)

jstuart0
and
claude
committed
feat(worker): Phase 4 — parallelize spec extraction group loop
jstuart0
committed
feat(worker): Phase 5 — embedding fan-out (OpenAI-compat only)
jstuart0
committed
feat(phase6/tok-s): thread current_tokens_per_second end-to-end (CA-169)

jstuart0
and
claude
committed
feat(observability): Phase 7 — extend admin /api/v1/admin/llm/activity with gate_snapshot

jstuart0
and
claude
committed
docs: Phase 8 — operator concurrency tuning runbook + CLAUDE.md recent-refactors entry

jstuart0
and
claude
committed
fix(worker): emit warning on unknown provider token in concurrency env vars

jstuart0
and
claude
committed

Commits on May 7, 2026

fix(ci): unblock PR #43 — lint cleanups + wrap_provider fallback + slow-test marker

jstuart0
and
claude
committed