From ca45715f0f5bd9843d35cd71449fcebac3332ba5 Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Fri, 22 May 2026 19:16:31 -0400 Subject: [PATCH 01/52] Add browser spy plan --- PLAN.md | 617 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 617 insertions(+) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..b0038c8 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,617 @@ +# Rootcell Browser Spy Plan + +## Goal + +Replace the current `rootcell spy --tui` / terminal spy workflow with an opt-in, +firewall-hosted browser system for inspecting LLM provider traffic. + +The system should make it easy to answer: + +- What did the harness send to the model? +- How much visible context was included with a short request? +- Which sections are system context, tools, history, current user input, + assistant output, thinking/reasoning, cache markers, and provider metadata? +- What did the provider report for input, output, cache read, and cache write + token usage? +- How did this request differ from the previous comparable request? +- What happened on the exact network request/response when deeper inspection is + needed? + +The first implementation targets Pi.dev using Amazon Bedrock. The architecture +must make future direct Anthropic, OpenAI, Claude Code, Codex, Cursor, and +multi-conversation support straightforward, but those are not v1 scope. + +## Decisions + +- `./rootcell spy` launches the browser UI. It no longer tails the terminal. +- Remove the old Textual TUI and old NDJSON spy format. There is no migration or + compatibility requirement. +- The spy system is opt-in. `ROOTCELL_SPY_ENABLED=false` is the default. +- When enabled, capture is always-on even when the browser is closed. +- Capture and the web service run on the firewall VM. +- The browser reaches the service through a rootcell-managed SSH local port + forward. No public web port is exposed. +- The browser UI is a local operator tool, not a hardened multi-user web app. + No auth system, no collaboration, and no public exposure in v1. +- Only LLM-provider traffic is captured. Non-provider HTTPS traffic must never be + spooled or persisted. +- Spy does not broaden network access. Bedrock/provider endpoints still must be + allowed through the normal DNS/HTTPS allowlists for the harness to use them. +- Python remains only as a minimal mitmproxy shim. The main spy system is + TypeScript under `src/spy`. +- The Python shim provider-gates and redacts auth headers/query credentials, + then writes bounded raw provider events to a spool. +- TypeScript owns validation, provider normalization, persistence, retention, + API, SSE, and UI serving. +- SQLite is the persistent store. +- V1 stores normalized semantic content by default. Exact raw payload storage is + optional via `ROOTCELL_SPY_STORE_RAW=false`. +- Request/response bodies that pass provider gating are sensitive and may + contain secrets. Do not attempt body secret redaction beyond binary/media + summarization. +- Desktop-only UI. Do not spend v1 scope on mobile support. +- No keyboard shortcut requirement. Design the browser UX on its own terms, not + as a TUI clone. +- Token counting for highlighted text and per-block token estimates are v1.5, + not v1. +- Automated compaction detection is v1.5, not v1. +- Broader charts/visual regression screenshots are v1.5 or later. + +## Architecture + +### Firewall Components + +Provision these components on every firewall VM, even when spy is disabled: + +- Minimal Python mitmproxy shim, replacing `proxy/agent_spy.py` and + `proxy/agent_spy_tui.py`. +- TypeScript spy service under `src/spy`. +- Built static React app served by the TypeScript service. +- Persistent directories: + - SQLite store: `/var/lib/rootcell-spy/spy.sqlite` + - transient spool: `/var/spool/rootcell-spy/` + - generated config: `/etc/agent-vm/spy.env` + +When `ROOTCELL_SPY_ENABLED=false`, the service should be stopped/disabled and +the shim must not write spool files. Existing spy data is preserved until the +user enables spy and retention runs, or explicitly clears it. + +### Python Shim + +The shim must be tiny and traffic-safe: + +- Check generated spy config/marker before doing any capture work. +- Detect only registered LLM provider candidates, starting with Bedrock Runtime + host/path patterns. +- Redact auth headers and credential query parameters. +- Emit separate spool events for request, response metadata/body, error, and + stream/chunk observations when available. +- Do not reassemble streams in Python. If mitmproxy exposes safe chunk hooks, + write one sanitized spool event per observed chunk. Otherwise write the + provider response body and let TypeScript decode logical stream frames. +- Enforce `ROOTCELL_SPY_SPOOL_MAX_BYTES` before appending. +- Never open SQLite. +- Never perform deep provider normalization. +- Never alter allow/deny decisions or block agent traffic if capture fails. + +If the spool is full, the shim should stop writing new capture payloads and, if +there is room, write a small rate-limited dropped-capture marker. Agent traffic +must continue under the existing firewall policy. + +### TypeScript Service + +Use Bun and TypeScript. Prefer Bun built-ins and pure JS/TS dependencies. Avoid +native npm packages unless they are explicitly target-built by Nix for the +firewall architecture. + +The service responsibilities: + +- Validate spool events with Zod. +- Ingest and delete spool files after successful commit. +- Decode Bedrock payloads and AWS event-stream frames. +- Pair request/response events by flow id. +- Persist provider calls, normalized blocks, stream events, raw payloads when + enabled, health counters, and service metadata. +- Run SQLite migrations on startup. +- Enforce retention by age and size while running. +- Serve a same-origin JSON API, SSE endpoint, and built React assets. +- Expose health/status data for capture and service state. + +Use Bun's native HTTP server for v1 unless routing becomes painful. Use Bun's +SQLite support if available in the pinned guest Bun; otherwise choose a +Nix-provisioned, target-native SQLite option. + +### Build And Delivery + +Static frontend assets may be built on the host because HTML, CSS, and browser +JavaScript bundles are architecture-neutral. + +The TypeScript service runtime and dependencies must be target-native on the +firewall VM: + +- Do not copy host `node_modules` into the firewall VM. +- Do not rely on macOS-built native npm artifacts. +- Avoid native npm dependencies in `src/spy` where feasible. +- If a native dependency becomes necessary, build/provision it through Nix for + the firewall target architecture. +- The firewall should not download npm packages or CDN assets at runtime. + +### Browser UI + +Use React + TypeScript + Vite, Tailwind, and local shadcn/ui-style components. +Vendor only the needed shadcn components. Do not depend on CDN assets, remote +fonts, or runtime package downloads. + +The first screen should be a live conversation-analysis surface: + +- Default load mode is "live from now"; do not auto-load historical events. +- `./rootcell spy` passes a viewer launch timestamp so the UI starts clean. +- Historical loading is explicit through time range controls such as last + 10 minutes, last hour, today, and custom range. +- Timeline rows are provider calls styled as conversation events. +- Each request/response pair is directly selectable. +- The right-side inspector is call-native: it shows exactly one provider call. +- The inspector includes request details, response details, network metadata, + headers, usage, cache markers, stream events on demand, and diff against the + previous comparable request. + +Performance requirements: + +- Virtualize the live timeline. +- Fetch summaries first and details on demand. +- Paginate historical queries. +- Keep stream events and raw payload details collapsed and loaded only on + request. +- Use semantic highlighting instead of editor-style highlighting as the primary + visual language. +- Avoid rendering giant JSON/code blocks into the DOM. + +Semantic highlighting should distinguish: + +- provider/request envelope +- harness/system context +- user-visible messages +- prior conversation history +- current user input +- assistant output +- thinking/reasoning +- tool definitions +- tool calls and tool results +- cache markers +- media summaries +- unknown/unclassified content + +JSON/code highlighting inside raw detail panels is secondary and should only be +used when cheap and bounded. + +## Configuration + +Seed these settings into `.env.defaults` as explicit defaults/comments: + +```sh +ROOTCELL_SPY_ENABLED=false +# ROOTCELL_SPY_RETENTION_DAYS=7 +# ROOTCELL_SPY_MAX_BYTES=6442450944 +# ROOTCELL_SPY_SPOOL_MAX_BYTES=1073741824 +# ROOTCELL_SPY_STORE_RAW=false +# ROOTCELL_SPY_BIND=127.0.0.1 +# ROOTCELL_SPY_PORT=6174 +``` + +Defaults: + +- Spy disabled unless `ROOTCELL_SPY_ENABLED=true`. +- Retain for 7 days. +- Total spy store budget: 6 GiB. +- Spool budget: 1 GiB. +- Raw exact payload storage disabled. +- Firewall service binds `127.0.0.1:6174`. + +`./rootcell spy` should choose host-local port `6174` when available and fall +back to another available local port if needed. The SSH tunnel forwards the +host-local port to `127.0.0.1:6174` on the firewall VM. + +## CLI And Provisioning + +### `rootcell provision` + +Always provision the spy service files, UI assets, directories, config template, +and systemd units. + +When enabled: + +- Render `/etc/agent-vm/spy.env`. +- Enable/start the TypeScript spy service. +- Enable shim writes through generated config/marker. +- Preserve existing spy data. + +When disabled: + +- Render config with spy disabled. +- Stop/disable the TypeScript spy service. +- Ensure the shim returns without spooling. +- Preserve existing spy data. + +Do not implement migration/remediation for existing small disks. There are no +existing users to support. + +### `rootcell spy` + +Required behavior: + +- If spy is disabled, print clear instructions to set + `ROOTCELL_SPY_ENABLED=true` in the selected instance `.env` and run + `./rootcell provision`. +- If service files/assets are missing or stale, tell the user to run + `./rootcell provision`; do not auto-provision. +- Ensure the firewall VM and service are reachable. +- Start an SSH local port forward through the provider abstraction. +- Print the local URL. +- Open the browser by default, with `--no-open` available. +- Stay in the foreground to keep the tunnel alive. +- Exit on Ctrl-C, closing only the tunnel. + +Remove `--tui`, `--raw`, and `--no-dedupe` from the user-facing CLI. + +Implement host-side launcher and tunnel lifecycle in TypeScript using provider +and transport abstractions. Avoid POSIX shell assumptions because Windows host +support is a future goal. + +## Data Model + +Use Zod schemas and SQLite migrations checked into `src/spy`. + +V1 durable unit: + +- `provider_call` + +Attached records: + +- request event metadata +- response event metadata +- normalized request blocks +- normalized response blocks +- stream events decoded from provider payloads +- usage records reported by the provider +- optional raw sanitized payload records +- content hashes for repeated-context and diffing +- health/drop/error counters + +Reserve future grouping concepts such as `turn_id` or conversation grouping, but +do not build turn grouping behavior in v1. + +Normalized blocks should preserve: + +- original order +- role/type +- source/provenance +- provider payload location when possible +- character and byte size +- content hash +- cache marker metadata +- media summaries instead of full binary/media bytes + +Do not compute or display per-block token estimates in v1. + +## API Shape + +Endpoint names are provisional, but v1 should expose these boundaries: + +- `GET /api/health` +- `GET /api/calls?since=&cursor=&limit=` +- `GET /api/calls/:id` +- `GET /api/calls/:id/diff` +- `GET /api/calls/:id/stream-events` +- `GET /api/search` +- `POST /api/clear` +- `GET /api/events` for SSE + +Use SSE for small live notifications such as new/updated call summaries and +health changes. Use normal paginated/detail endpoints for content. + +Do not enable broad CORS. V1 does not need a per-launch access token, auth +system, CSP/security-header hardening, or public web exposure. + +`POST /api/clear` should: + +- Take an ingestion lock. +- Stop ingestion briefly. +- Delete captured call data and pending spool files. +- Reset relevant capture counters. +- Store a clear baseline timestamp/generation. +- Resume ingestion. +- Keep schema/migration metadata. + +## Persistence And Retention + +SQLite is the source of truth. Spool files are transient and sensitive. + +Retention runs inside the TypeScript service only: + +- Run on startup and periodically while the service is running. +- Enforce age and size caps. +- Delete oldest `provider_call` rows first. +- Cascade delete related normalized blocks, stream events, and raw payloads. +- Enforce spool cleanup after successful ingestion. + +No separate systemd retention timer. + +If the TypeScript service is stopped, the Python shim can only fill the bounded +spool, then it must stop writing. + +## Provider And Harness Layers + +Keep two independent extension layers: + +- Provider adapters answer "what happened on the wire?" +- Harness analyzers answer "what does this mean for Pi/Codex/Claude Code/etc?" + +V1 provider: + +- Amazon Bedrock only. +- Decode Bedrock Runtime request/response shapes. +- Decode AWS event-stream frames in TypeScript. +- Extract provider-reported usage, status, metrics, stop reasons, text, + thinking/reasoning, tool deltas, and cache markers. + +V1 harness: + +- Pi.dev only. +- Use empirical fixtures to classify obvious Pi-added context and provenance. +- Fall back to generic Bedrock roles and `unknown` when unsure. + +Do not introduce LiteLLM or any translation proxy. The point of spy is to +observe real harness/provider behavior, not normalize traffic through another +provider abstraction. + +## UI V1 Scope + +V1 includes: + +- Desktop-only browser UI. +- Live-from-now default timeline. +- Explicit historical time range loading. +- Search and filtering by time, provider/model, event type, and normalized text. +- Provider call summaries with status, duration, model, operation, and + provider-reported usage totals. +- Request composition summary using exact structural measures: + - section presence + - message count + - character/byte size by section + - tool count and tool schema size + - cache markers + - media summaries + - provider-reported total input/output/cache usage when available +- Clear visual distinction between short current user input and large repeated + system/tool/history context. +- Repeated/new/changed cues based on block content hashes compared to the + previous comparable request. +- Cache markers clearly visible in timeline summaries and inspector details. +- Call-native right inspector. +- Diff against previous comparable request. +- On-demand stream event section. +- Network metadata and headers. +- Raw payload panels only when raw storage is enabled; otherwise show that raw + storage was disabled. +- Health/settings area showing enabled state, DB size, spool size, caps, + retention days, dropped capture count, last ingest time, and service version. +- Manual "clear spy data" action with confirmation. + +V1 excludes: + +- Automated compaction detection. +- Highlighted text token counting. +- Local token estimates. +- Exact provider token-count calls. +- Broad charts/dashboards. +- Mobile support. +- Keyboard shortcut requirements. +- Annotations/bookmarks/labels. +- Import/export. +- Multi-instance aggregation. +- Multi-conversation grouping. +- Public access/auth hardening. + +## Roadmap + +### V1 + +Build the Bedrock/Pi browser spy: + +1. Define spool event schema, normalized provider schema, and SQLite schema. +2. Replace Python spy with minimal provider-gated spool shim. +3. Implement TypeScript Bedrock adapter and AWS event-stream decoder. +4. Implement SQLite persistence, migrations, retention, and clear-data. +5. Implement TS web service, API, SSE, and static asset serving. +6. Implement React desktop UI with virtualized timeline and call inspector. +7. Wire `rootcell provision`, systemd service config, and `rootcell spy` + launcher/tunnel. +8. Raise firewall disk/root volume defaults to 64 GiB. +9. Remove old TUI/terminal spy flags, tests, and docs. +10. Add `src/spy/README.md` and brief links from main/proxy docs. + +### V1.5 + +Add analysis depth: + +- Exact/estimated token counting for highlighted text, blocks, sections, and + whole requests. +- Provider-routed token-count backend; browser never calls LLM providers + directly. +- Per-block token provenance: `provider_reported`, `provider_counted`, + `estimated`, or `unavailable`. +- Automated compaction candidate detection: + - Pi-specific request patterns from fixtures. + - Generic fallback heuristics. + - Labels that distinguish Pi-specific candidates from heuristic candidates. +- Dedicated compaction investigation view. +- Visual regression/screenshot checks. + +### V2 + +Broaden scope: + +- Direct Anthropic provider adapter. +- OpenAI provider adapter. +- Additional harness analyzers for Claude Code, Codex, Cursor, and others. +- Multiple simultaneous conversation grouping. +- Rich token/time/cache charts and dashboards. +- Export/archive workflows if real use shows demand. +- Stronger auth/security model only if any non-local exposure is introduced. + +## Capacity Defaults + +Change firewall disk defaults: + +- Lima firewall disk: 64 GiB. +- AWS firewall root volume: 64 GiB. +- Keep agent disk default unchanged. + +Keep current CPU/RAM defaults as starting points, but validate with fixtures and +raise them if the service, SQLite ingestion, or UI serving needs more headroom. +Disk is cheap; do not optimize the service around artificially tiny capacity. + +## Security And Privacy + +This feature persists decrypted LLM prompts and responses. + +Security model: + +- Disabled by default. +- SSH tunnel only. +- No public web listener. +- Service binds firewall-local `127.0.0.1`. +- Header/query credential redaction is mandatory. +- No body secret redaction. +- Binary/media payloads summarized by default. +- Raw exact payload storage disabled by default. +- Spool is sensitive even when raw storage is disabled. +- Retention and manual clear are required. +- Capture failure must not affect agent traffic. +- EBS encryption / disk-at-rest posture should be reviewed for AWS, but v1 does + not add a separate application-level encryption layer. + +The docs must clearly state that if a prompt contains a secret, the spy store may +contain that secret until retention or manual clear removes it. + +## Failure Modes + +Define user-visible behavior and recovery for: + +- Spy disabled. +- Service not provisioned or stale. +- SSH tunnel failure. +- TypeScript service down. +- SQLite locked/corrupt. +- Spool full. +- Store retention limit reached. +- Retention cleanup failure. +- Mitmproxy shim error. +- Bedrock adapter parse failure. +- SSE disconnect. +- Dropped capture events. + +Hard invariant: spy failures must not block, slow significantly, or change +network traffic allow/deny decisions. + +## Testing + +V1 tests: + +- Python shim unit tests: + - enabled/disabled gating + - Bedrock provider candidate detection + - auth header/query redaction + - spool cap behavior + - no-write behavior when disabled/full +- TypeScript unit tests: + - Zod schema validation + - Bedrock request/response normalization + - AWS event-stream decoding + - usage/cache marker extraction + - repeated/new/changed hash classification + - Pi provenance classification from fixtures +- SQLite tests: + - migrations + - ingest idempotence/retry behavior + - request/response pairing + - retention by age and size + - clear-data baseline + - cascade deletes +- API tests: + - health + - pagination + - detail loading + - diff endpoint + - stream events endpoint + - search + - clear + - SSE notifications +- Playwright functional tests without screenshot baselines: + - app loads in local fixture mode + - receives SSE update + - selects a call + - opens inspector sections + - loads historical range + - searches + - clears data + +Fixture strategy: + +- Start with handcrafted minimal Bedrock/Pi fixtures. +- Add sanitized real Pi/Bedrock captures once the new spool schema exists. +- Cover normal calls, streaming, tool calls/results, cache markers, large + history, error responses, disabled capture, raw disabled, and raw enabled. + +## Documentation + +Add `src/spy/README.md` as the detailed operator/developer doc. + +Briefly reference it from `README.md` and `proxy/README.md`. + +Docs should cover: + +- Enabling spy in the instance `.env`. +- Running `./rootcell provision`. +- Launching `./rootcell spy`. +- Data locations. +- Retention settings. +- Disk sizing defaults. +- Privacy/security implications. +- Clear-data behavior. +- Service health and troubleshooting. +- Removed TUI/terminal flags. +- How provider and harness adapters are organized. + +## Non-Goals + +- No LiteLLM or request translation proxy. +- No old TUI or old NDJSON compatibility. +- No public web exposure. +- No auth system in v1. +- No multi-provider support in v1. +- No multi-instance UI in v1. +- No multi-conversation grouping in v1. +- No automated compaction detection in v1. +- No highlighted-text token counting in v1. +- No local token estimates in v1. +- No mobile optimization. +- No keyboard shortcut parity with the TUI. +- No annotations/bookmarks/labels. +- No import/export. +- No in-UI settings editing. +- No body secret redaction beyond auth headers/query credentials. + +## Open Technical Validations + +- Confirm Bun's SQLite support is available and suitable in the pinned Nixpkgs + firewall runtime. +- Validate whether mitmproxy can safely expose true streaming chunk timing. If + not, persist logical stream events decoded from completed AWS event-stream + bodies and label real arrival timing unavailable. +- Measure firewall CPU/RAM under representative Bedrock/Pi streaming and + large-history fixtures, then decide whether to raise defaults. +- Confirm systemd `DynamicUser` plus persistent `StateDirectory`/spool + permissions cleanly support TS service ownership and mitmproxy append access. +- Finalize the exact provider-neutral spool schema after the first Bedrock/Pi + fixture pass. From 33bde8d05a47425de5efadc5ff30fd4d1d061d53 Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Fri, 22 May 2026 20:50:35 -0400 Subject: [PATCH 02/52] Add spy fixtures and schema foundation --- PLAN.md | 47 +++-- src/spy/eventstream.ts | 138 +++++++++++++ src/spy/fixtures/README.md | 19 ++ .../fixtures/bedrock-pi-us-sonnet-4-6.ndjson | 10 + src/spy/migrations.ts | 175 ++++++++++++++++ src/spy/schemas.test.ts | 169 ++++++++++++++++ src/spy/schemas.ts | 189 ++++++++++++++++++ 7 files changed, 735 insertions(+), 12 deletions(-) create mode 100644 src/spy/eventstream.ts create mode 100644 src/spy/fixtures/README.md create mode 100644 src/spy/fixtures/bedrock-pi-us-sonnet-4-6.ndjson create mode 100644 src/spy/migrations.ts create mode 100644 src/spy/schemas.test.ts create mode 100644 src/spy/schemas.ts diff --git a/PLAN.md b/PLAN.md index b0038c8..6d34d82 100644 --- a/PLAN.md +++ b/PLAN.md @@ -414,21 +414,41 @@ V1 excludes: ## Roadmap +### Completed + +- Captured sanitized real Pi/Bedrock traffic from the existing running + `default` instance using Pi provider `amazon-bedrock` and model + `us.anthropic.claude-sonnet-4-6`. +- Added `src/spy/fixtures/bedrock-pi-us-sonnet-4-6.ndjson` with real + request/response pairs for simple streaming, two-turn history, cache markers, + toolUse, toolResult, and provider-reported usage. +- Added initial `src/spy` TypeScript contract: + - Zod spool event, provider call, normalized block, usage, stream event, raw + payload, and diff schemas. + - AWS event-stream decoder with CRC validation. + - V1 SQLite migration helper and initial schema. + - Fixture validation, event-stream decoding, and migration tests. +- Verified `bun run typecheck`, `bun run lint`, `bun run test`, direct + `bun:sqlite` migration execution, and a fixture credential audit. + ### V1 Build the Bedrock/Pi browser spy: -1. Define spool event schema, normalized provider schema, and SQLite schema. -2. Replace Python spy with minimal provider-gated spool shim. -3. Implement TypeScript Bedrock adapter and AWS event-stream decoder. -4. Implement SQLite persistence, migrations, retention, and clear-data. -5. Implement TS web service, API, SSE, and static asset serving. -6. Implement React desktop UI with virtualized timeline and call inspector. -7. Wire `rootcell provision`, systemd service config, and `rootcell spy` +- [x] Define spool event schema, normalized provider schema, and SQLite schema. +- [x] Capture sanitized real Pi/Bedrock fixtures to ground the schema and + adapter work. +- [x] Add initial AWS event-stream decoder. +- [ ] Replace Python spy with minimal provider-gated spool shim. +- [ ] Implement TypeScript Bedrock adapter on top of the captured fixtures. +- [ ] Implement SQLite persistence, migrations, retention, and clear-data. +- [ ] Implement TS web service, API, SSE, and static asset serving. +- [ ] Implement React desktop UI with virtualized timeline and call inspector. +- [ ] Wire `rootcell provision`, systemd service config, and `rootcell spy` launcher/tunnel. -8. Raise firewall disk/root volume defaults to 64 GiB. -9. Remove old TUI/terminal spy flags, tests, and docs. -10. Add `src/spy/README.md` and brief links from main/proxy docs. +- [ ] Raise firewall disk/root volume defaults to 64 GiB. +- [ ] Remove old TUI/terminal spy flags, tests, and docs. +- [ ] Add `src/spy/README.md` and brief links from main/proxy docs. ### V1.5 @@ -558,8 +578,11 @@ V1 tests: Fixture strategy: -- Start with handcrafted minimal Bedrock/Pi fixtures. -- Add sanitized real Pi/Bedrock captures once the new spool schema exists. +- Initial sanitized real Pi/Bedrock fixture capture is complete for + `us.anthropic.claude-sonnet-4-6`. +- Add handcrafted minimal fixtures only as supplements for targeted edge cases. +- Add more sanitized real captures as the Bedrock adapter, shim, and UI expose + concrete gaps. - Cover normal calls, streaming, tool calls/results, cache markers, large history, error responses, disabled capture, raw disabled, and raw enabled. diff --git a/src/spy/eventstream.ts b/src/spy/eventstream.ts new file mode 100644 index 0000000..520b366 --- /dev/null +++ b/src/spy/eventstream.ts @@ -0,0 +1,138 @@ +export interface AwsEventStreamMessage { + readonly headers: Readonly>; + readonly payload: Uint8Array; +} + +export class AwsEventStreamDecodeError extends Error { + constructor(message: string) { + super(message); + this.name = "AwsEventStreamDecodeError"; + } +} + +export function decodeAwsEventStream(input: Uint8Array | string): AwsEventStreamMessage[] { + const data = typeof input === "string" ? Buffer.from(input, "base64") : Buffer.from(input); + const messages: AwsEventStreamMessage[] = []; + let offset = 0; + + while (offset < data.length) { + if (data.length - offset < 16) { + throw new AwsEventStreamDecodeError("truncated prelude"); + } + + const totalLength = data.readUInt32BE(offset); + const headersLength = data.readUInt32BE(offset + 4); + if (totalLength < 16) { + throw new AwsEventStreamDecodeError(`invalid total length ${String(totalLength)}`); + } + const end = offset + totalLength; + if (end > data.length) { + throw new AwsEventStreamDecodeError("truncated message"); + } + + const preludeCrc = data.readUInt32BE(offset + 8); + if (crc32(data.subarray(offset, offset + 8)) !== preludeCrc) { + throw new AwsEventStreamDecodeError("prelude CRC mismatch"); + } + + const messageCrc = data.readUInt32BE(end - 4); + if (crc32(data.subarray(offset, end - 4)) !== messageCrc) { + throw new AwsEventStreamDecodeError("message CRC mismatch"); + } + + const headersStart = offset + 12; + const headersEnd = headersStart + headersLength; + if (headersEnd > end - 4) { + throw new AwsEventStreamDecodeError("headers exceed message"); + } + + messages.push({ + headers: decodeHeaders(data.subarray(headersStart, headersEnd)), + payload: data.subarray(headersEnd, end - 4), + }); + offset = end; + } + + return messages; +} + +export function decodeAwsEventStreamJson(input: Uint8Array | string): { + readonly headers: Readonly>; + readonly payload: unknown; +}[] { + const decoder = new TextDecoder(); + return decodeAwsEventStream(input).map((message) => ({ + headers: message.headers, + payload: JSON.parse(decoder.decode(message.payload)) as unknown, + })); +} + +function decodeHeaders(data: Buffer): Record { + const headers: Record = {}; + let offset = 0; + while (offset < data.length) { + const nameLength = data[offset]; + offset += 1; + if (nameLength === undefined || offset + nameLength + 1 > data.length) { + throw new AwsEventStreamDecodeError("truncated header"); + } + + const name = data.subarray(offset, offset + nameLength).toString("utf8"); + offset += nameLength; + const valueType = data[offset]; + offset += 1; + + if (valueType === 0) { + headers[name] = true; + } else if (valueType === 1) { + headers[name] = false; + } else if (valueType === 2) { + headers[name] = data.readInt8(offset); + offset += 1; + } else if (valueType === 3) { + headers[name] = data.readInt16BE(offset); + offset += 2; + } else if (valueType === 4) { + headers[name] = data.readInt32BE(offset); + offset += 4; + } else if (valueType === 5) { + headers[name] = Number(data.readBigInt64BE(offset)); + offset += 8; + } else if (valueType === 6 || valueType === 7) { + const length = data.readUInt16BE(offset); + offset += 2; + const raw = data.subarray(offset, offset + length); + offset += length; + headers[name] = valueType === 6 ? new Uint8Array(raw) : raw.toString("utf8"); + } else if (valueType === 8) { + const millis = Number(data.readBigInt64BE(offset)); + offset += 8; + headers[name] = new Date(millis).toISOString(); + } else if (valueType === 9) { + headers[name] = data.subarray(offset, offset + 16).toString("hex"); + offset += 16; + } else { + throw new AwsEventStreamDecodeError(`unknown header type ${String(valueType)}`); + } + } + return headers; +} + +// Small table-driven CRC32 implementation to avoid adding a dependency for AWS +// event-stream validation. +const CRC32_TABLE = new Uint32Array(256); +for (let index = 0; index < CRC32_TABLE.length; index += 1) { + let value = index; + for (let bit = 0; bit < 8; bit += 1) { + value = (value & 1) === 1 ? 0xEDB88320 ^ (value >>> 1) : value >>> 1; + } + CRC32_TABLE[index] = value >>> 0; +} + +function crc32(data: Uint8Array): number { + let crc = 0xFFFFFFFF; + for (const byte of data) { + crc = (CRC32_TABLE[(crc ^ byte) & 0xFF] ?? 0) ^ (crc >>> 8); + } + return (crc ^ 0xFFFFFFFF) >>> 0; +} diff --git a/src/spy/fixtures/README.md b/src/spy/fixtures/README.md new file mode 100644 index 0000000..9546d97 --- /dev/null +++ b/src/spy/fixtures/README.md @@ -0,0 +1,19 @@ +# Spy Fixtures + +`bedrock-pi-us-sonnet-4-6.ndjson` is a credential-redacted capture from the +existing `default` rootcell instance using Pi with Amazon Bedrock model +`us.anthropic.claude-sonnet-4-6`. + +The fixture preserves real Bedrock request bodies and AWS event-stream response +bytes. Header/query credential redaction stayed enabled during capture; fixture +generation also stabilized flow ids, timestamps, request ids, and SDK invocation +ids. + +Captured cases: + +- simple streaming prompt/response +- two-turn session history +- cache markers emitted by Pi +- toolUse stream response +- follow-up request containing toolResult +- provider-reported usage metadata diff --git a/src/spy/fixtures/bedrock-pi-us-sonnet-4-6.ndjson b/src/spy/fixtures/bedrock-pi-us-sonnet-4-6.ndjson new file mode 100644 index 0000000..316365c --- /dev/null +++ b/src/spy/fixtures/bedrock-pi-us-sonnet-4-6.ndjson @@ -0,0 +1,10 @@ +{"version":1,"ts":1779496800,"direction":"request","flow_id":"fixture-flow-simple","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","headers":[["content-type","application/json"],["content-length","5687"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-1"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_text":"{\"messages\":[{\"role\":\"user\",\"content\":[{\"text\":\"Fixture capture simple prompt. Reply with exactly: fixture-simple-ok\"},{\"cachePoint\":{\"type\":\"default\"}}]}],\"system\":[{\"text\":\"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.\\n\\nAvailable tools:\\n- read: Read file contents\\n- bash: Execute bash commands (ls, grep, find, etc.)\\n- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call\\n- write: Create or overwrite files\\n\\nIn addition to the tools above, you may have access to other custom tools depending on the project.\\n\\nGuidelines:\\n- Use bash for file operations like ls, rg, find\\n- Use read to examine files instead of cat or sed.\\n- Use edit for precise changes (edits[].oldText must match exactly)\\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\\n- Use write only for new files or complete rewrites.\\n- Be concise in your responses\\n- Show file paths clearly when working with files\\n\\nPi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):\\n- Main documentation: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/README.md\\n- Additional docs: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/docs\\n- Examples: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/examples (extensions, custom tools, SDK)\\n- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)\\n- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing\\n- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)\\nCurrent date: 2026-05-23\\nCurrent working directory: /home/luser\"},{\"cachePoint\":{\"type\":\"default\"}}],\"inferenceConfig\":{\"maxTokens\":32000},\"toolConfig\":{\"tools\":[{\"toolSpec\":{\"name\":\"read\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to read (relative or absolute)\"},\"offset\":{\"type\":\"number\",\"description\":\"Line number to start reading from (1-indexed)\"},\"limit\":{\"type\":\"number\",\"description\":\"Maximum number of lines to read\"}}}},\"description\":\"Read the contents of a file. Supports text files and images (jpg, png, gif, webp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.\"}},{\"toolSpec\":{\"name\":\"bash\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"command\"],\"properties\":{\"command\":{\"type\":\"string\",\"description\":\"Bash command to execute\"},\"timeout\":{\"type\":\"number\",\"description\":\"Timeout in seconds (optional, no default timeout)\"}}}},\"description\":\"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.\"}},{\"toolSpec\":{\"name\":\"edit\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\",\"edits\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to edit (relative or absolute)\"},\"edits\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"oldText\",\"newText\"],\"properties\":{\"oldText\":{\"type\":\"string\",\"description\":\"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.\"},\"newText\":{\"type\":\"string\",\"description\":\"Replacement text for this targeted edit.\"}},\"additionalProperties\":false},\"description\":\"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.\"}},\"additionalProperties\":false}},\"description\":\"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.\"}},{\"toolSpec\":{\"name\":\"write\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\",\"content\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to write (relative or absolute)\"},\"content\":{\"type\":\"string\",\"description\":\"Content to write to the file\"}}}},\"description\":\"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.\"}}]},\"additionalModelRequestFields\":{\"thinking\":{\"type\":\"adaptive\",\"display\":\"summarized\"},\"output_config\":{\"effort\":\"low\"}}}"} +{"version":1,"ts":1779496801,"direction":"response","flow_id":"fixture-flow-simple","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","status_code":200,"reason":"","headers":[["date","Sat, 23 May 2026 00:00:00 GMT"],["content-type","application/vnd.amazon.eventstream"],["x-amzn-requestid","fixture-request-2"]],"request_headers":[["content-type","application/json"],["content-length","5687"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-2"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_b64":"AAAAnAAAAFL0sSWgCzpldmVudC10eXBlBwAMbWVzc2FnZVN0YXJ0DTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREUiLCJyb2xlIjoiYXNzaXN0YW50In1Q/B+YAAAAyAAAAFdJSLgkCzpldmVudC10eXBlBwARY29udGVudEJsb2NrRGVsdGENOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJjb250ZW50QmxvY2tJbmRleCI6MCwiZGVsdGEiOnsidGV4dCI6ImZpeHR1cmUifSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU4ifeqSqesAAADVAAAAV9E46xcLOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tEZWx0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJkZWx0YSI6eyJ0ZXh0IjoiLXNpbXBsZS1vayJ9LCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTk9QUVJTVFVWV1gifQdmnOAAAACQAAAAVjYsDLgLOmV2ZW50LXR5cGUHABBjb250ZW50QmxvY2tTdG9wDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsInAiOiJhYmNkZWZnaGlqa2wifcLZpl4AAACfAAAAUSoYDsoLOmV2ZW50LXR5cGUHAAttZXNzYWdlU3RvcA06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7InAiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0QiLCJzdG9wUmVhc29uIjoiZW5kX3R1cm4ifas1whsAAADxAAAAToESyhMLOmV2ZW50LXR5cGUHAAhtZXRhZGF0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7Im1ldHJpY3MiOnsibGF0ZW5jeU1zIjoxMTE2fSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eCIsInVzYWdlIjp7ImlucHV0VG9rZW5zIjoxOTM2LCJvdXRwdXRUb2tlbnMiOjgsInNlcnZlclRvb2xVc2FnZSI6e30sInRvdGFsVG9rZW5zIjoxOTQ0fX0HNwmw","body_sha256":"7c9449cbd76af329391a2c7aec86cf4357263ce700971a9bdd6747d80b6287ba","body_encoding":"aws-eventstream"} +{"version":1,"ts":1779496802,"direction":"request","flow_id":"fixture-flow-session-turn-one","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","headers":[["content-type","application/json"],["content-length","5716"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-3"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_text":"{\"messages\":[{\"role\":\"user\",\"content\":[{\"text\":\"Fixture capture session turn one. Remember marker RCSPY-ALPHA and reply with exactly: turn-one-ok\"},{\"cachePoint\":{\"type\":\"default\"}}]}],\"system\":[{\"text\":\"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.\\n\\nAvailable tools:\\n- read: Read file contents\\n- bash: Execute bash commands (ls, grep, find, etc.)\\n- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call\\n- write: Create or overwrite files\\n\\nIn addition to the tools above, you may have access to other custom tools depending on the project.\\n\\nGuidelines:\\n- Use bash for file operations like ls, rg, find\\n- Use read to examine files instead of cat or sed.\\n- Use edit for precise changes (edits[].oldText must match exactly)\\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\\n- Use write only for new files or complete rewrites.\\n- Be concise in your responses\\n- Show file paths clearly when working with files\\n\\nPi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):\\n- Main documentation: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/README.md\\n- Additional docs: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/docs\\n- Examples: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/examples (extensions, custom tools, SDK)\\n- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)\\n- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing\\n- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)\\nCurrent date: 2026-05-23\\nCurrent working directory: /home/luser\"},{\"cachePoint\":{\"type\":\"default\"}}],\"inferenceConfig\":{\"maxTokens\":32000},\"toolConfig\":{\"tools\":[{\"toolSpec\":{\"name\":\"read\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to read (relative or absolute)\"},\"offset\":{\"type\":\"number\",\"description\":\"Line number to start reading from (1-indexed)\"},\"limit\":{\"type\":\"number\",\"description\":\"Maximum number of lines to read\"}}}},\"description\":\"Read the contents of a file. Supports text files and images (jpg, png, gif, webp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.\"}},{\"toolSpec\":{\"name\":\"bash\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"command\"],\"properties\":{\"command\":{\"type\":\"string\",\"description\":\"Bash command to execute\"},\"timeout\":{\"type\":\"number\",\"description\":\"Timeout in seconds (optional, no default timeout)\"}}}},\"description\":\"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.\"}},{\"toolSpec\":{\"name\":\"edit\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\",\"edits\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to edit (relative or absolute)\"},\"edits\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"oldText\",\"newText\"],\"properties\":{\"oldText\":{\"type\":\"string\",\"description\":\"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.\"},\"newText\":{\"type\":\"string\",\"description\":\"Replacement text for this targeted edit.\"}},\"additionalProperties\":false},\"description\":\"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.\"}},\"additionalProperties\":false}},\"description\":\"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.\"}},{\"toolSpec\":{\"name\":\"write\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\",\"content\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to write (relative or absolute)\"},\"content\":{\"type\":\"string\",\"description\":\"Content to write to the file\"}}}},\"description\":\"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.\"}}]},\"additionalModelRequestFields\":{\"thinking\":{\"type\":\"adaptive\",\"display\":\"summarized\"},\"output_config\":{\"effort\":\"low\"}}}"} +{"version":1,"ts":1779496803,"direction":"response","flow_id":"fixture-flow-session-turn-one","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","status_code":200,"reason":"","headers":[["date","Sat, 23 May 2026 00:00:00 GMT"],["content-type","application/vnd.amazon.eventstream"],["x-amzn-requestid","fixture-request-4"]],"request_headers":[["content-type","application/json"],["content-length","5716"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-4"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_b64":"AAAAmQAAAFI8UarQCzpldmVudC10eXBlBwAMbWVzc2FnZVN0YXJ0DTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUIiLCJyb2xlIjoiYXNzaXN0YW50In3SL1jNAAAAuwAAAFf3OiI7CzpldmVudC10eXBlBwARY29udGVudEJsb2NrRGVsdGENOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJjb250ZW50QmxvY2tJbmRleCI6MCwiZGVsdGEiOnsidGV4dCI6InR1cm4ifSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDRCJ9HMfm2gAAAMAAAABXeTjz5Qs6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRleHQiOiItb25lLW9rIn0sInAiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRiJ98dO+5gAAAJkAAABWOzxuyQs6ZXZlbnQtdHlwZQcAEGNvbnRlbnRCbG9ja1N0b3ANOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJjb250ZW50QmxvY2tJbmRleCI6MCwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dSJ9d2II1AAAALIAAABRE0nlfws6ZXZlbnQtdHlwZQcAC21lc3NhZ2VTdG9wDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU5PUFFSU1RVVlciLCJzdG9wUmVhc29uIjoiZW5kX3R1cm4ifUx81a0AAAEBAAAATgJ6SesLOmV2ZW50LXR5cGUHAAhtZXRhZGF0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7Im1ldHJpY3MiOnsibGF0ZW5jeU1zIjoxMTcxfSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU4iLCJ1c2FnZSI6eyJpbnB1dFRva2VucyI6MTk0Niwib3V0cHV0VG9rZW5zIjo4LCJzZXJ2ZXJUb29sVXNhZ2UiOnt9LCJ0b3RhbFRva2VucyI6MTk1NH19j/RbVw==","body_sha256":"7f36c588d092f12062bb3ef7e36b62b69629a0aad9c23b1477f2accdd1149d36","body_encoding":"aws-eventstream"} +{"version":1,"ts":1779496804,"direction":"request","flow_id":"fixture-flow-session-turn-two","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","headers":[["content-type","application/json"],["content-length","5918"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-5"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_text":"{\"messages\":[{\"role\":\"user\",\"content\":[{\"text\":\"Fixture capture session turn one. Remember marker RCSPY-ALPHA and reply with exactly: turn-one-ok\"}]},{\"role\":\"assistant\",\"content\":[{\"text\":\"turn-one-ok\"}]},{\"role\":\"user\",\"content\":[{\"text\":\"Fixture capture session turn two. State the marker from the previous turn and reply in one short sentence.\"},{\"cachePoint\":{\"type\":\"default\"}}]}],\"system\":[{\"text\":\"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.\\n\\nAvailable tools:\\n- read: Read file contents\\n- bash: Execute bash commands (ls, grep, find, etc.)\\n- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call\\n- write: Create or overwrite files\\n\\nIn addition to the tools above, you may have access to other custom tools depending on the project.\\n\\nGuidelines:\\n- Use bash for file operations like ls, rg, find\\n- Use read to examine files instead of cat or sed.\\n- Use edit for precise changes (edits[].oldText must match exactly)\\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\\n- Use write only for new files or complete rewrites.\\n- Be concise in your responses\\n- Show file paths clearly when working with files\\n\\nPi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):\\n- Main documentation: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/README.md\\n- Additional docs: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/docs\\n- Examples: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/examples (extensions, custom tools, SDK)\\n- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)\\n- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing\\n- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)\\nCurrent date: 2026-05-23\\nCurrent working directory: /home/luser\"},{\"cachePoint\":{\"type\":\"default\"}}],\"inferenceConfig\":{\"maxTokens\":32000},\"toolConfig\":{\"tools\":[{\"toolSpec\":{\"name\":\"read\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to read (relative or absolute)\"},\"offset\":{\"type\":\"number\",\"description\":\"Line number to start reading from (1-indexed)\"},\"limit\":{\"type\":\"number\",\"description\":\"Maximum number of lines to read\"}}}},\"description\":\"Read the contents of a file. Supports text files and images (jpg, png, gif, webp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.\"}},{\"toolSpec\":{\"name\":\"bash\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"command\"],\"properties\":{\"command\":{\"type\":\"string\",\"description\":\"Bash command to execute\"},\"timeout\":{\"type\":\"number\",\"description\":\"Timeout in seconds (optional, no default timeout)\"}}}},\"description\":\"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.\"}},{\"toolSpec\":{\"name\":\"edit\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\",\"edits\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to edit (relative or absolute)\"},\"edits\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"oldText\",\"newText\"],\"properties\":{\"oldText\":{\"type\":\"string\",\"description\":\"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.\"},\"newText\":{\"type\":\"string\",\"description\":\"Replacement text for this targeted edit.\"}},\"additionalProperties\":false},\"description\":\"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.\"}},\"additionalProperties\":false}},\"description\":\"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.\"}},{\"toolSpec\":{\"name\":\"write\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"path\",\"content\"],\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Path to the file to write (relative or absolute)\"},\"content\":{\"type\":\"string\",\"description\":\"Content to write to the file\"}}}},\"description\":\"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.\"}}]},\"additionalModelRequestFields\":{\"thinking\":{\"type\":\"adaptive\",\"display\":\"summarized\"},\"output_config\":{\"effort\":\"low\"}}}"} +{"version":1,"ts":1779496805,"direction":"response","flow_id":"fixture-flow-session-turn-two","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","status_code":200,"reason":"","headers":[["date","Sat, 23 May 2026 00:00:00 GMT"],["content-type","application/vnd.amazon.eventstream"],["x-amzn-requestid","fixture-request-6"]],"request_headers":[["content-type","application/json"],["content-length","5918"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-6"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_b64":"AAAAugAAAFK6MP8ECzpldmVudC10eXBlBwAMbWVzc2FnZVN0YXJ0DTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVowMTIzNDU2NzgiLCJyb2xlIjoiYXNzaXN0YW50In00xKBOAAAAoAAAAFfgCoSoCzpldmVudC10eXBlBwARY29udGVudEJsb2NrRGVsdGENOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJjb250ZW50QmxvY2tJbmRleCI6MCwiZGVsdGEiOnsidGV4dCI6IlRoZSJ9LCJwIjoiYWJjZCJ93/VteQAAAN4AAABXpujaBgs6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRleHQiOiIgbWFya2VyIGZyb20gdGhlIHByZXZpb3VzIHR1cm4gaXMgKioifSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGRyJ9IRJeYwAAAMUAAABXsdh8lQs6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRleHQiOiJSQ1NQWS1BTFBIQSoqLiJ9LCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNEIn2ODa0vAAAAlgAAAFa5bPkYCzpldmVudC10eXBlBwAQY29udGVudEJsb2NrU3RvcA06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyIn3rI9fcAAAAnwAAAFEqGA7KCzpldmVudC10eXBlBwALbWVzc2FnZVN0b3ANOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNEIiwic3RvcFJlYXNvbiI6ImVuZF90dXJuIn2rNcIbAAAA6gAAAE6WImyACzpldmVudC10eXBlBwAIbWV0YWRhdGENOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJtZXRyaWNzIjp7ImxhdGVuY3lNcyI6MTg2M30sInAiOiJhYmNkZWZnaGlqa2xtbm9wIiwidXNhZ2UiOnsiaW5wdXRUb2tlbnMiOjE5NzgsIm91dHB1dFRva2VucyI6MTgsInNlcnZlclRvb2xVc2FnZSI6e30sInRvdGFsVG9rZW5zIjoxOTk2fX2vGjDI","body_sha256":"ed8d60f13bcf49f91d867d5fc6d1c7fc2185f0890379a3e5ba6ffafd7e33b798","body_encoding":"aws-eventstream"} +{"version":1,"ts":1779496806,"direction":"request","flow_id":"fixture-flow-tool-use","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","headers":[["content-type","application/json"],["content-length","2652"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-7"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_text":"{\"messages\":[{\"role\":\"user\",\"content\":[{\"text\":\"Fixture capture tool cycle. Use the bash tool to run: printf \\\"tool-fixture-ok\\\\n\\\". Then reply with exactly the command output and nothing else.\"},{\"cachePoint\":{\"type\":\"default\"}}]}],\"system\":[{\"text\":\"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.\\n\\nAvailable tools:\\n- bash: Execute bash commands (ls, grep, find, etc.)\\n\\nIn addition to the tools above, you may have access to other custom tools depending on the project.\\n\\nGuidelines:\\n- Use bash for file operations like ls, rg, find\\n- Be concise in your responses\\n- Show file paths clearly when working with files\\n\\nPi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):\\n- Main documentation: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/README.md\\n- Additional docs: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/docs\\n- Examples: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/examples (extensions, custom tools, SDK)\\n- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)\\n- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing\\n- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)\\nCurrent date: 2026-05-23\\nCurrent working directory: /home/luser\"},{\"cachePoint\":{\"type\":\"default\"}}],\"inferenceConfig\":{\"maxTokens\":32000},\"toolConfig\":{\"tools\":[{\"toolSpec\":{\"name\":\"bash\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"command\"],\"properties\":{\"command\":{\"type\":\"string\",\"description\":\"Bash command to execute\"},\"timeout\":{\"type\":\"number\",\"description\":\"Timeout in seconds (optional, no default timeout)\"}}}},\"description\":\"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.\"}}]},\"additionalModelRequestFields\":{\"thinking\":{\"type\":\"adaptive\",\"display\":\"summarized\"},\"output_config\":{\"effort\":\"low\"}}}"} +{"version":1,"ts":1779496807,"direction":"response","flow_id":"fixture-flow-tool-use","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","status_code":200,"reason":"","headers":[["date","Sat, 23 May 2026 00:00:00 GMT"],["content-type","application/vnd.amazon.eventstream"],["x-amzn-requestid","fixture-request-8"]],"request_headers":[["content-type","application/json"],["content-length","2652"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-8"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_b64":"AAAAkgAAAFJLgZvBCzpldmVudC10eXBlBwAMbWVzc2FnZVN0YXJ0DTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dSIsInJvbGUiOiJhc3Npc3RhbnQifVMJx7gAAAEhAAAAV6fQzi8LOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tTdGFydA06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0Iiwic3RhcnQiOnsidG9vbFVzZSI6eyJuYW1lIjoiYmFzaCIsInRvb2xVc2VJZCI6InRvb2x1c2VfU0p5cG5CbG0xQ25laENkeENvMTU2SSIsInR5cGUiOiJ0b29sX3VzZSJ9fX1I986VAAAAtgAAAFcPquaKCzpldmVudC10eXBlBwARY29udGVudEJsb2NrRGVsdGENOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJjb250ZW50QmxvY2tJbmRleCI6MCwiZGVsdGEiOnsidG9vbFVzZSI6eyJpbnB1dCI6IiJ9fSwicCI6ImFiY2RlZmdoaWprbG1ub3AifSoZPRsAAADRAAAAVyS4TdcLOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tEZWx0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJkZWx0YSI6eyJ0b29sVXNlIjp7ImlucHV0Ijoie1wiIn19LCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTiJ9j46xsgAAAPAAAABX2BlLYws6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRvb2xVc2UiOnsiaW5wdXQiOiJjb21tYW5kXCI6IFwicCJ9fSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVowMTIzNDU2NyJ95f4stgAAANMAAABXXngetws6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRvb2xVc2UiOnsiaW5wdXQiOiJyaW4ifX0sInAiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRkdISUpLTE1OT1Aifbc0m/UAAAC6AAAAV8paC4sLOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tEZWx0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJkZWx0YSI6eyJ0b29sVXNlIjp7ImlucHV0IjoidGYgXFwifX0sInAiOiJhYmNkZWZnaGlqa2xtbm8ifbD/SpEAAAC3AAAAVzLKzzoLOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tEZWx0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJkZWx0YSI6eyJ0b29sVXNlIjp7ImlucHV0IjoiXCJ0b29sLWYifX0sInAiOiJhYmNkZWZnaGkifeFk6VoAAADdAAAAV+FIoNYLOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tEZWx0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJkZWx0YSI6eyJ0b29sVXNlIjp7ImlucHV0IjoiaXh0dXJlLW9rXFxcXG4ifX0sInAiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRkdISUpLTE1OTyJ9u96EjwAAAOcAAABXCtkA8Qs6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRvb2xVc2UiOnsiaW5wdXQiOiJcXFwiXCJ9In19LCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NSJ9n6/oyQAAAMAAAABWDj/Dcws6ZXZlbnQtdHlwZQcAEGNvbnRlbnRCbG9ja1N0b3ANOmNvbnRlbnQtdHlwZQcAEGFwcGxpY2F0aW9uL2pzb24NOm1lc3NhZ2UtdHlwZQcABWV2ZW50eyJjb250ZW50QmxvY2tJbmRleCI6MCwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVowMTIzNDU2NyJ9EPTSwQAAALgAAABRWfn93gs6ZXZlbnQtdHlwZQcAC21lc3NhZ2VTdG9wDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVowMTIiLCJzdG9wUmVhc29uIjoidG9vbF91c2UifWRpgAMAAAETAAAAThhajQkLOmV2ZW50LXR5cGUHAAhtZXRhZGF0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7Im1ldHJpY3MiOnsibGF0ZW5jeU1zIjoxNzM3fSwicCI6ImFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVowMTIzNCIsInVzYWdlIjp7ImlucHV0VG9rZW5zIjoxMTc0LCJvdXRwdXRUb2tlbnMiOjYwLCJzZXJ2ZXJUb29sVXNhZ2UiOnt9LCJ0b3RhbFRva2VucyI6MTIzNH19VtfPhw==","body_sha256":"4184ca2da5e8f547a6e550dbd0d3b4573a6560dd40aec44227ed10231799e96f","body_encoding":"aws-eventstream"} +{"version":1,"ts":1779496808,"direction":"request","flow_id":"fixture-flow-tool-result","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","headers":[["content-type","application/json"],["content-length","2960"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-9"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_text":"{\"messages\":[{\"role\":\"user\",\"content\":[{\"text\":\"Fixture capture tool cycle. Use the bash tool to run: printf \\\"tool-fixture-ok\\\\n\\\". Then reply with exactly the command output and nothing else.\"}]},{\"role\":\"assistant\",\"content\":[{\"toolUse\":{\"toolUseId\":\"tooluse_SJypnBlm1CnehCdxCo156I\",\"name\":\"bash\",\"input\":{\"command\":\"printf \\\"tool-fixture-ok\\\\n\\\"\"}}}]},{\"role\":\"user\",\"content\":[{\"toolResult\":{\"toolUseId\":\"tooluse_SJypnBlm1CnehCdxCo156I\",\"content\":[{\"text\":\"tool-fixture-ok\\n\"}],\"status\":\"success\"}},{\"cachePoint\":{\"type\":\"default\"}}]}],\"system\":[{\"text\":\"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.\\n\\nAvailable tools:\\n- bash: Execute bash commands (ls, grep, find, etc.)\\n\\nIn addition to the tools above, you may have access to other custom tools depending on the project.\\n\\nGuidelines:\\n- Use bash for file operations like ls, rg, find\\n- Be concise in your responses\\n- Show file paths clearly when working with files\\n\\nPi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):\\n- Main documentation: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/README.md\\n- Additional docs: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/docs\\n- Examples: /nix/store/igyx27gfzvgvi2b27vrlk4fcadjixsw7-pi-coding-agent-0.74.0/share/pi-coding-agent/examples (extensions, custom tools, SDK)\\n- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)\\n- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing\\n- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)\\nCurrent date: 2026-05-23\\nCurrent working directory: /home/luser\"},{\"cachePoint\":{\"type\":\"default\"}}],\"inferenceConfig\":{\"maxTokens\":32000},\"toolConfig\":{\"tools\":[{\"toolSpec\":{\"name\":\"bash\",\"inputSchema\":{\"json\":{\"type\":\"object\",\"required\":[\"command\"],\"properties\":{\"command\":{\"type\":\"string\",\"description\":\"Bash command to execute\"},\"timeout\":{\"type\":\"number\",\"description\":\"Timeout in seconds (optional, no default timeout)\"}}}},\"description\":\"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.\"}}]},\"additionalModelRequestFields\":{\"thinking\":{\"type\":\"adaptive\",\"display\":\"summarized\"},\"output_config\":{\"effort\":\"low\"}}}"} +{"version":1,"ts":1779496809,"direction":"response","flow_id":"fixture-flow-tool-result","provider":"bedrock","operation":"converse-stream","model_id":"us.anthropic.claude-sonnet-4-6","host":"bedrock-runtime.us-east-1.amazonaws.com","method":"POST","path":"/model/us.anthropic.claude-sonnet-4-6/converse-stream","status_code":200,"reason":"","headers":[["date","Sat, 23 May 2026 00:00:00 GMT"],["content-type","application/vnd.amazon.eventstream"],["x-amzn-requestid","fixture-request-10"]],"request_headers":[["content-type","application/json"],["content-length","2960"],["x-amz-user-agent","aws-sdk-js/3.1044.0"],["user-agent","aws-sdk-js/3.1044.0 ua/2.1 os/linux#6.19.11 lang/js md/bun#1.2.20 api/bedrock-runtime#3.1044.0 m/E"],["amz-sdk-invocation-id","fixture-invocation-10"],["amz-sdk-request","attempt=1; max=3"],["authorization","[redacted]"]],"body_b64":"AAAAhQAAAFKZQdBTCzpldmVudC10eXBlBwAMbWVzc2FnZVN0YXJ0DTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsicCI6ImFiY2RlZmdoIiwicm9sZSI6ImFzc2lzdGFudCJ9PVk6xgAAANMAAABXXngetws6ZXZlbnQtdHlwZQcAEWNvbnRlbnRCbG9ja0RlbHRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsImRlbHRhIjp7InRleHQiOiJ0b29sIn0sInAiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRkdISUpLTE1OT1BRUlNUVVZXWFlaMDEifXIMwOsAAAC0AAAAV3VqteoLOmV2ZW50LXR5cGUHABFjb250ZW50QmxvY2tEZWx0YQ06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7ImNvbnRlbnRCbG9ja0luZGV4IjowLCJkZWx0YSI6eyJ0ZXh0IjoiLWZpeHR1cmUtb2sifSwicCI6ImFiY2RlZmdoaWprbG1ub3Aifc8klXIAAACcAAAAVvPc4bkLOmV2ZW50LXR5cGUHABBjb250ZW50QmxvY2tTdG9wDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsiY29udGVudEJsb2NrSW5kZXgiOjAsInAiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3gifWORzUUAAACSAAAAUdKIynsLOmV2ZW50LXR5cGUHAAttZXNzYWdlU3RvcA06Y29udGVudC10eXBlBwAQYXBwbGljYXRpb24vanNvbg06bWVzc2FnZS10eXBlBwAFZXZlbnR7InAiOiJhYmNkZWZnaGlqa2xtbm9wcSIsInN0b3BSZWFzb24iOiJlbmRfdHVybiJ9lR4ZiwAAAPcAAABODlI/sws6ZXZlbnQtdHlwZQcACG1ldGFkYXRhDTpjb250ZW50LXR5cGUHABBhcHBsaWNhdGlvbi9qc29uDTptZXNzYWdlLXR5cGUHAAVldmVudHsibWV0cmljcyI6eyJsYXRlbmN5TXMiOjEwODZ9LCJwIjoiYWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNEIiwidXNhZ2UiOnsiaW5wdXRUb2tlbnMiOjEyNTMsIm91dHB1dFRva2VucyI6OCwic2VydmVyVG9vbFVzYWdlIjp7fSwidG90YWxUb2tlbnMiOjEyNjF9fRoOceY=","body_sha256":"97c162eebfeb1979ea6c448ec21c2d2c6cf1e3e097f38354f0897a7a8f044064","body_encoding":"aws-eventstream"} diff --git a/src/spy/migrations.ts b/src/spy/migrations.ts new file mode 100644 index 0000000..a16f1ea --- /dev/null +++ b/src/spy/migrations.ts @@ -0,0 +1,175 @@ +import type { Database } from "bun:sqlite"; + +export interface SpyMigration { + readonly version: number; + readonly name: string; + readonly sql: string; +} + +export const SPY_SQLITE_MIGRATIONS: readonly SpyMigration[] = [ + { + version: 1, + name: "initial spy capture schema", + sql: ` +CREATE TABLE IF NOT EXISTS provider_call ( + id TEXT PRIMARY KEY, + provider TEXT NOT NULL, + operation TEXT NOT NULL, + model_id TEXT NOT NULL, + status TEXT NOT NULL CHECK (status IN ('pending', 'complete', 'error', 'dropped')), + started_at REAL NOT NULL, + completed_at REAL, + status_code INTEGER, + request_flow_id TEXT NOT NULL, + response_flow_id TEXT, + request_content_hash TEXT, + response_content_hash TEXT, + created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +CREATE UNIQUE INDEX IF NOT EXISTS provider_call_request_flow_id_idx + ON provider_call(request_flow_id); +CREATE INDEX IF NOT EXISTS provider_call_started_at_idx + ON provider_call(started_at); +CREATE INDEX IF NOT EXISTS provider_call_model_started_idx + ON provider_call(provider, model_id, operation, started_at); + +CREATE TABLE IF NOT EXISTS http_event ( + id TEXT PRIMARY KEY, + call_id TEXT NOT NULL REFERENCES provider_call(id) ON DELETE CASCADE, + direction TEXT NOT NULL CHECK (direction IN ('request', 'response')), + observed_at REAL NOT NULL, + host TEXT NOT NULL, + method TEXT NOT NULL, + path TEXT NOT NULL, + status_code INTEGER, + reason TEXT, + headers_json TEXT NOT NULL, + request_headers_json TEXT, + body_text TEXT, + body_b64 TEXT, + body_sha256 TEXT, + body_encoding TEXT, + content_type TEXT +); + +CREATE INDEX IF NOT EXISTS http_event_call_direction_idx + ON http_event(call_id, direction); + +CREATE TABLE IF NOT EXISTS normalized_block ( + id TEXT PRIMARY KEY, + call_id TEXT NOT NULL REFERENCES provider_call(id) ON DELETE CASCADE, + direction TEXT NOT NULL CHECK (direction IN ('request', 'response')), + ordinal INTEGER NOT NULL, + role TEXT, + kind TEXT NOT NULL, + source TEXT NOT NULL, + provider_path TEXT, + text TEXT, + json TEXT, + char_size INTEGER NOT NULL, + byte_size INTEGER NOT NULL, + content_hash TEXT NOT NULL, + cache_marker INTEGER NOT NULL DEFAULT 0 +); + +CREATE INDEX IF NOT EXISTS normalized_block_call_ordinal_idx + ON normalized_block(call_id, direction, ordinal); +CREATE INDEX IF NOT EXISTS normalized_block_hash_idx + ON normalized_block(content_hash); +CREATE INDEX IF NOT EXISTS normalized_block_kind_idx + ON normalized_block(kind); +CREATE VIRTUAL TABLE IF NOT EXISTS normalized_block_fts + USING fts5(block_id UNINDEXED, text); + +CREATE TABLE IF NOT EXISTS usage_record ( + id TEXT PRIMARY KEY, + call_id TEXT NOT NULL REFERENCES provider_call(id) ON DELETE CASCADE, + source TEXT NOT NULL, + input_tokens INTEGER, + output_tokens INTEGER, + cache_read_tokens INTEGER, + cache_write_tokens INTEGER, + total_tokens INTEGER, + raw_json TEXT +); + +CREATE INDEX IF NOT EXISTS usage_record_call_idx + ON usage_record(call_id); + +CREATE TABLE IF NOT EXISTS stream_event ( + id TEXT PRIMARY KEY, + call_id TEXT NOT NULL REFERENCES provider_call(id) ON DELETE CASCADE, + ordinal INTEGER NOT NULL, + event_type TEXT NOT NULL, + headers_json TEXT NOT NULL, + payload_json TEXT, + payload_text TEXT, + payload_sha256 TEXT, + observed_at REAL +); + +CREATE INDEX IF NOT EXISTS stream_event_call_ordinal_idx + ON stream_event(call_id, ordinal); +CREATE INDEX IF NOT EXISTS stream_event_type_idx + ON stream_event(event_type); + +CREATE TABLE IF NOT EXISTS raw_payload ( + id TEXT PRIMARY KEY, + call_id TEXT NOT NULL REFERENCES provider_call(id) ON DELETE CASCADE, + direction TEXT NOT NULL CHECK (direction IN ('request', 'response')), + content_type TEXT, + body_text TEXT, + body_b64 TEXT, + body_sha256 TEXT, + body_encoding TEXT +); + +CREATE INDEX IF NOT EXISTS raw_payload_call_idx + ON raw_payload(call_id); + +CREATE TABLE IF NOT EXISTS health_counter ( + name TEXT PRIMARY KEY, + value INTEGER NOT NULL, + updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +CREATE TABLE IF NOT EXISTS service_metadata ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL, + updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP +); +`, + }, +]; + +export function applySpyMigrations(db: Database): void { + db.run("PRAGMA foreign_keys = ON"); + db.run(` +CREATE TABLE IF NOT EXISTS schema_migration ( + version INTEGER PRIMARY KEY, + name TEXT NOT NULL, + applied_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP +); +`); + + const applied = new Set( + db.query("SELECT version FROM schema_migration").all() + .map((row) => (row as { version: number }).version), + ); + + for (const migration of SPY_SQLITE_MIGRATIONS) { + if (applied.has(migration.version)) { + continue; + } + db.transaction(() => { + db.run(migration.sql); + db.query("INSERT INTO schema_migration (version, name) VALUES (?, ?)") + .run(migration.version, migration.name); + })(); + } +} + +export function currentSpySchemaVersion(): number { + return SPY_SQLITE_MIGRATIONS.at(-1)?.version ?? 0; +} diff --git a/src/spy/schemas.test.ts b/src/spy/schemas.test.ts new file mode 100644 index 0000000..f3b5b3f --- /dev/null +++ b/src/spy/schemas.test.ts @@ -0,0 +1,169 @@ +import { readFileSync } from "node:fs"; +import { describe, expect, test } from "vitest"; +import { decodeAwsEventStreamJson } from "./eventstream.ts"; +import { applySpyMigrations, currentSpySchemaVersion } from "./migrations.ts"; +import { + SpoolEventSchema, + type SpoolEvent, + type SpoolRequestEvent, + type SpoolResponseEvent, +} from "./schemas.ts"; + +const FIXTURE_PATH = new URL("./fixtures/bedrock-pi-us-sonnet-4-6.ndjson", import.meta.url); +const bunSqlite = await import("bun:sqlite").catch(() => null); + +function fixtureEvents(): SpoolEvent[] { + return readFileSync(FIXTURE_PATH, "utf8") + .trim() + .split("\n") + .filter(Boolean) + .map((line) => SpoolEventSchema.parse(JSON.parse(line) as unknown)); +} + +function requestBody(event: SpoolRequestEvent): Record { + if (event.body_text === undefined) { + throw new Error(`request ${event.flow_id} does not include body_text`); + } + return JSON.parse(event.body_text) as Record; +} + +function responseEvents(events: readonly SpoolEvent[]): SpoolResponseEvent[] { + return events.filter((event): event is SpoolResponseEvent => event.direction === "response"); +} + +function httpEvents(events: readonly SpoolEvent[]): (SpoolRequestEvent | SpoolResponseEvent)[] { + return events.filter((event): event is SpoolRequestEvent | SpoolResponseEvent => ( + event.direction === "request" || event.direction === "response" + )); +} + +function requestFor(events: readonly SpoolEvent[], flowId: string): SpoolRequestEvent { + const event = events.find((candidate): candidate is SpoolRequestEvent => candidate.direction === "request" && candidate.flow_id === flowId); + if (event === undefined) { + throw new Error(`missing request fixture ${flowId}`); + } + return event; +} + +describe("spy fixture capture", () => { + test("validates the real Pi/Bedrock capture under the spool schema", () => { + const events = fixtureEvents(); + expect(events).toHaveLength(10); + const captured = httpEvents(events); + expect(new Set(captured.map((event) => event.flow_id))).toEqual(new Set([ + "fixture-flow-simple", + "fixture-flow-session-turn-one", + "fixture-flow-session-turn-two", + "fixture-flow-tool-use", + "fixture-flow-tool-result", + ])); + expect(events.every((event) => event.provider === "bedrock")).toBe(true); + expect(captured.every((event) => event.model_id === "us.anthropic.claude-sonnet-4-6")).toBe(true); + }); + + test("keeps credential metadata redacted while preserving body structure", () => { + const events = fixtureEvents(); + const headerPairs = events.flatMap((event) => [ + ...("headers" in event ? event.headers : []), + ...("request_headers" in event ? event.request_headers : []), + ]); + expect(headerPairs.filter(([name]) => name.toLowerCase() === "authorization")) + .toEqual(headerPairs.filter(([name]) => name.toLowerCase() === "authorization").map(([name]) => [name, "[redacted]"])); + + const simple = requestFor(events, "fixture-flow-simple"); + const body = requestBody(simple); + expect(body).toHaveProperty("messages"); + expect(JSON.stringify(body)).toContain("cachePoint"); + expect(JSON.stringify(body)).toContain("toolConfig"); + }); + + test("captures prior conversation history and tool result request shapes", () => { + const events = fixtureEvents(); + const history = requestFor(events, "fixture-flow-session-turn-two"); + const toolResult = requestFor(events, "fixture-flow-tool-result"); + + const historyBody = requestBody(history); + expect(JSON.stringify(historyBody)).toContain("turn-one-ok"); + expect(JSON.stringify(historyBody)).toContain("RCSPY-ALPHA"); + + const toolResultBody = requestBody(toolResult); + expect(JSON.stringify(toolResultBody)).toContain("toolUse"); + expect(JSON.stringify(toolResultBody)).toContain("toolResult"); + expect(JSON.stringify(toolResultBody)).toContain("tool-fixture-ok"); + }); + + test("decodes response event streams, usage, and tool use deltas", () => { + const responses = responseEvents(fixtureEvents()); + expect(responses).toHaveLength(5); + + const decoded = responses.map((event) => decodeAwsEventStreamJson(event.body_b64 ?? "")); + expect(decoded.every((messages) => messages.some((message) => message.headers[":event-type"] === "metadata"))).toBe(true); + expect(JSON.stringify(decoded)).toContain("inputTokens"); + expect(JSON.stringify(decoded)).toContain("outputTokens"); + + const toolUse = decoded[3]; + const toolUseJson = JSON.stringify(toolUse); + expect(toolUseJson).toContain("tool_use"); + expect(toolUseJson).toContain("toolUse"); + expect(toolUseJson).toContain("tooluse_"); + }); +}); + +describe("spy sqlite migrations", () => { + const testWithBunSqlite = bunSqlite === null ? test.skip : test; + + testWithBunSqlite("creates the v1 schema and supports core provider call inserts", () => { + if (bunSqlite === null) { + throw new Error("bun:sqlite unavailable"); + } + const db = new bunSqlite.Database(":memory:"); + try { + applySpyMigrations(db); + expect(currentSpySchemaVersion()).toBe(1); + expect(db.query("SELECT version FROM schema_migration").get()).toEqual({ version: 1 }); + + db.query(` +INSERT INTO provider_call ( + id, provider, operation, model_id, status, started_at, completed_at, + status_code, request_flow_id, response_flow_id +) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) +`).run( + "call-fixture-flow-simple", + "bedrock", + "converse-stream", + "us.anthropic.claude-sonnet-4-6", + "complete", + 1779496800, + 1779496801, + 200, + "fixture-flow-simple", + "fixture-flow-simple", + ); + + db.query(` +INSERT INTO normalized_block ( + id, call_id, direction, ordinal, kind, source, text, + char_size, byte_size, content_hash, cache_marker +) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) +`).run( + "block-simple-user", + "call-fixture-flow-simple", + "request", + 0, + "current-user-input", + "bedrock-converse", + "Fixture capture simple prompt.", + 30, + 30, + "hash-simple-user", + 0, + ); + + expect(db.query("SELECT COUNT(*) AS count FROM provider_call").get()).toEqual({ count: 1 }); + db.query("DELETE FROM provider_call WHERE id = ?").run("call-fixture-flow-simple"); + expect(db.query("SELECT COUNT(*) AS count FROM normalized_block").get()).toEqual({ count: 0 }); + } finally { + db.close(); + } + }); +}); diff --git a/src/spy/schemas.ts b/src/spy/schemas.ts new file mode 100644 index 0000000..96803b2 --- /dev/null +++ b/src/spy/schemas.ts @@ -0,0 +1,189 @@ +import { z } from "zod"; + +export const SpyHeaderPairSchema = z.tuple([z.string(), z.string()]); + +const CapturedBodyShape = { + body_text: z.string().optional(), + body_b64: z.string().optional(), + body_sha256: z.string().optional(), + body_encoding: z.enum(["aws-eventstream"]).optional(), +} as const; + +const HttpCaptureBaseSchema = z.object({ + version: z.literal(1), + ts: z.number(), + flow_id: z.string().min(1), + provider: z.literal("bedrock"), + operation: z.string().min(1), + model_id: z.string().min(1), + host: z.string().min(1), + method: z.string().min(1), + path: z.string().min(1), + headers: z.array(SpyHeaderPairSchema), +}).strict(); + +export const SpoolRequestEventSchema = HttpCaptureBaseSchema.extend({ + direction: z.literal("request"), + ...CapturedBodyShape, +}).refine( + (value) => value.body_text !== undefined || value.body_b64 !== undefined, + { message: "capture event must include body_text or body_b64" }, +); + +export const SpoolResponseEventSchema = HttpCaptureBaseSchema.extend({ + direction: z.literal("response"), + status_code: z.number().int().nonnegative(), + reason: z.string(), + request_headers: z.array(SpyHeaderPairSchema), + ...CapturedBodyShape, +}).refine( + (value) => value.body_text !== undefined || value.body_b64 !== undefined, + { message: "capture event must include body_text or body_b64" }, +); + +export const SpoolStreamChunkEventSchema = HttpCaptureBaseSchema.extend({ + direction: z.literal("stream-chunk"), + chunk_index: z.number().int().nonnegative(), + body_b64: z.string().min(1), + body_sha256: z.string().optional(), + body_encoding: z.enum(["aws-eventstream"]).optional(), +}); + +export const SpoolErrorEventSchema = z.object({ + version: z.literal(1), + ts: z.number(), + direction: z.literal("error"), + flow_id: z.string().optional(), + provider: z.literal("bedrock").optional(), + error: z.string().min(1), +}).strict(); + +export const SpoolDroppedEventSchema = z.object({ + version: z.literal(1), + ts: z.number(), + direction: z.literal("dropped"), + provider: z.literal("bedrock").optional(), + reason: z.string().min(1), + dropped_count: z.number().int().positive(), +}).strict(); + +export const SpoolEventSchema = z.discriminatedUnion("direction", [ + SpoolRequestEventSchema, + SpoolResponseEventSchema, + SpoolStreamChunkEventSchema, + SpoolErrorEventSchema, + SpoolDroppedEventSchema, +]); + +export type SpoolEvent = Readonly>; +export type SpoolRequestEvent = Readonly>; +export type SpoolResponseEvent = Readonly>; + +export const ProviderCallStatusSchema = z.enum([ + "pending", + "complete", + "error", + "dropped", +]); + +export const ProviderCallSchema = z.object({ + id: z.string().min(1), + provider: z.literal("bedrock"), + operation: z.string().min(1), + model_id: z.string().min(1), + status: ProviderCallStatusSchema, + started_at: z.number(), + completed_at: z.number().optional(), + status_code: z.number().int().nonnegative().optional(), + request_flow_id: z.string().min(1), + response_flow_id: z.string().min(1).optional(), + request_content_hash: z.string().optional(), + response_content_hash: z.string().optional(), +}).strict(); + +export type ProviderCall = Readonly>; + +export const NormalizedBlockKindSchema = z.enum([ + "provider-envelope", + "harness-system-context", + "user-visible-message", + "prior-conversation-history", + "current-user-input", + "assistant-output", + "thinking", + "tool-definition", + "tool-call", + "tool-result", + "cache-marker", + "media-summary", + "unknown", +]); + +export const NormalizedBlockSchema = z.object({ + id: z.string().min(1), + call_id: z.string().min(1), + direction: z.enum(["request", "response"]), + ordinal: z.number().int().nonnegative(), + role: z.string().optional(), + kind: NormalizedBlockKindSchema, + source: z.string().min(1), + provider_path: z.string().optional(), + text: z.string().optional(), + json: z.unknown().optional(), + char_size: z.number().int().nonnegative(), + byte_size: z.number().int().nonnegative(), + content_hash: z.string().min(1), + cache_marker: z.boolean().default(false), +}).strict(); + +export type NormalizedBlock = Readonly>; + +export const UsageRecordSchema = z.object({ + id: z.string().min(1), + call_id: z.string().min(1), + source: z.literal("provider-reported"), + input_tokens: z.number().int().nonnegative().optional(), + output_tokens: z.number().int().nonnegative().optional(), + cache_read_tokens: z.number().int().nonnegative().optional(), + cache_write_tokens: z.number().int().nonnegative().optional(), + total_tokens: z.number().int().nonnegative().optional(), + raw: z.unknown().optional(), +}).strict(); + +export type UsageRecord = Readonly>; + +export const StreamEventSchema = z.object({ + id: z.string().min(1), + call_id: z.string().min(1), + ordinal: z.number().int().nonnegative(), + event_type: z.string().min(1), + headers: z.record(z.string(), z.unknown()), + payload: z.unknown().optional(), + payload_text: z.string().optional(), + payload_sha256: z.string().optional(), + observed_at: z.number().optional(), +}).strict(); + +export type StreamEvent = Readonly>; + +export const RawPayloadRecordSchema = z.object({ + id: z.string().min(1), + call_id: z.string().min(1), + direction: z.enum(["request", "response"]), + content_type: z.string().optional(), + body_text: z.string().optional(), + body_b64: z.string().optional(), + body_sha256: z.string().optional(), + body_encoding: z.enum(["aws-eventstream"]).optional(), +}).strict(); + +export type RawPayloadRecord = Readonly>; + +export const DiffClassificationSchema = z.enum([ + "new", + "repeated", + "changed", + "unknown", +]); + +export type DiffClassification = z.infer; From 1895f9ae548a43070e12982b253f37ce081342aa Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Fri, 22 May 2026 21:06:05 -0400 Subject: [PATCH 03/52] Add Bedrock spy spool shim --- PLAN.md | 19 ++- firewall-vm.nix | 8 +- proxy/agent_spy.py | 304 ++++++++++++++++++++++++++++++++------ proxy/mitmproxy_addon.py | 5 + proxy/test_agent_spy.py | 307 ++++++++++++++++++++++++++------------- 5 files changed, 495 insertions(+), 148 deletions(-) diff --git a/PLAN.md b/PLAN.md index 6d34d82..aa67dfe 100644 --- a/PLAN.md +++ b/PLAN.md @@ -430,6 +430,23 @@ V1 excludes: - Fixture validation, event-stream decoding, and migration tests. - Verified `bun run typecheck`, `bun run lint`, `bun run test`, direct `bun:sqlite` migration execution, and a fixture credential audit. +- Replaced the mitmproxy-facing Python capture path with a minimal Bedrock + spool shim: + - Reads `/etc/agent-vm/spy.env` and captures only when + `ROOTCELL_SPY_ENABLED=true`. + - Provider-gates Bedrock Runtime request, response, and error events. + - Redacts auth headers and credential query parameters before spooling. + - Writes one atomic schema-shaped JSON file per event under + `/var/spool/rootcell-spy`. + - Enforces `ROOTCELL_SPY_SPOOL_MAX_BYTES` and emits rate-limited dropped + markers when the spool is full. + - Stores AWS event-stream responses as base64 with + `body_encoding=aws-eventstream` for TypeScript decoding. + - Added firewall group/tmpfiles/systemd sandbox permissions so mitmproxy can + write the sensitive spool path. + - Added Python unit coverage for disabled/default behavior, config parsing, + Bedrock detection, redaction, event-stream response encoding, provider-gated + errors, spool cap behavior, dropped markers, and failure swallowing. ### V1 @@ -439,7 +456,7 @@ Build the Bedrock/Pi browser spy: - [x] Capture sanitized real Pi/Bedrock fixtures to ground the schema and adapter work. - [x] Add initial AWS event-stream decoder. -- [ ] Replace Python spy with minimal provider-gated spool shim. +- [x] Replace Python spy with minimal provider-gated spool shim. - [ ] Implement TypeScript Bedrock adapter on top of the captured fixtures. - [ ] Implement SQLite persistence, migrations, retention, and clear-data. - [ ] Implement TS web service, API, SSE, and static asset serving. diff --git a/firewall-vm.nix b/firewall-vm.nix index 0e2794d..a13c11b 100644 --- a/firewall-vm.nix +++ b/firewall-vm.nix @@ -74,6 +74,7 @@ in pkgs.bun (pkgs.python3.withPackages (ps: [ ps.textual ])) ]; + users.groups.rootcell-spy = {}; # ── Networking ──────────────────────────────────────────────────────── networking.useDHCP = false; @@ -191,6 +192,7 @@ in "d /etc/agent-vm 0755 ${username} users -" "f /etc/agent-vm/dnsmasq-allowlist.conf 0644 root root -" "d /run/agent-vm-spy 1777 root root -" + "d /var/spool/rootcell-spy 2770 root rootcell-spy -" ]; # ── mitmproxy ───────────────────────────────────────────────────────── @@ -276,7 +278,8 @@ in ProtectHome = true; NoNewPrivileges = true; ReadOnlyPaths = "/etc/agent-vm"; - ReadWritePaths = "/run/agent-vm-spy"; + ReadWritePaths = [ "/run/agent-vm-spy" "/var/spool/rootcell-spy" ]; + SupplementaryGroups = [ "rootcell-spy" ]; Restart = "on-failure"; RestartSec = "2s"; }; @@ -308,7 +311,8 @@ in ProtectSystem = "strict"; ProtectHome = true; ReadOnlyPaths = "/etc/agent-vm"; - ReadWritePaths = "/run/agent-vm-spy"; + ReadWritePaths = [ "/run/agent-vm-spy" "/var/spool/rootcell-spy" ]; + SupplementaryGroups = [ "rootcell-spy" ]; Restart = "on-failure"; RestartSec = "2s"; # Transparent mode binds the listening socket with IP_TRANSPARENT diff --git a/proxy/agent_spy.py b/proxy/agent_spy.py index 6197ab7..91998b8 100644 --- a/proxy/agent_spy.py +++ b/proxy/agent_spy.py @@ -1,9 +1,10 @@ #!/usr/bin/env python3 -"""Bedrock traffic capture and formatting helpers for the firewall VM. +"""Bedrock traffic capture helpers for the firewall VM. This module is intentionally stdlib-only. mitmproxy imports the capture -helpers from its own Python environment, while `./rootcell spy` runs the same -file as a CLI inside the firewall VM to tail and format captured events. +helpers from its own Python environment. The mitmproxy-facing path is a minimal +Bedrock-gated spool shim; the older formatter helpers remain below for tests +and for the later CLI/TUI cleanup task. """ from __future__ import annotations @@ -14,6 +15,7 @@ import datetime as _dt import fnmatch import hashlib +import itertools import json import os import re @@ -24,9 +26,17 @@ from typing import Any, Iterable -SPY_DIR = os.environ.get("AGENT_SPY_DIR", "/run/agent-vm-spy") -SPY_ENABLED = os.path.join(SPY_DIR, "enabled") -SPY_EVENTS = os.path.join(SPY_DIR, "events.ndjson") +LEGACY_SPY_DIR = os.environ.get("AGENT_SPY_DIR", "/run/agent-vm-spy") +SPY_EVENTS = os.path.join(LEGACY_SPY_DIR, "events.ndjson") + +SPY_ENV = os.environ.get("ROOTCELL_SPY_ENV", "/etc/agent-vm/spy.env") +DEFAULT_SPY_SPOOL_DIR = "/var/spool/rootcell-spy" +DEFAULT_SPOOL_MAX_BYTES = 1_073_741_824 +DROPPED_MARKER_INTERVAL_SECONDS = 30.0 + +_SPOOL_COUNTER = itertools.count() +_DROPPED_SINCE_MARKER = 0 +_LAST_DROPPED_MARKER_AT = 0.0 EVENTSTREAM_CONTENT_TYPE = "application/vnd.amazon.eventstream" JSON_CONTENT_TYPES = { @@ -266,17 +276,216 @@ def _response_body_bytes(flow: Any) -> bytes: return body -def _write_event(event: dict[str, Any]) -> None: - if not os.path.exists(SPY_ENABLED): - return - line = json.dumps(event, ensure_ascii=False, separators=(",", ":")) + "\n" +def load_spy_config(path: str | None = None) -> dict[str, str]: + config_path = path or SPY_ENV + try: + with open(config_path, "r", encoding="utf-8") as f: + lines = f.readlines() + except OSError: + return {} + + config: dict[str, str] = {} + for raw_line in lines: + line = raw_line.strip() + if not line or line.startswith("#") or "=" not in line: + continue + key, value = line.split("=", 1) + key = key.strip() + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: + value = value[1:-1] + if key: + config[key] = value + return config + + +def _positive_int(value: str | None, default: int) -> int: + if value is None: + return default + try: + parsed = int(value, 10) + except ValueError: + return default + return parsed if parsed > 0 else default + + +def _capture_config() -> dict[str, Any] | None: + config = load_spy_config() + if config.get("ROOTCELL_SPY_ENABLED", "").strip().lower() != "true": + return None + return { + "spool_dir": DEFAULT_SPY_SPOOL_DIR, + "spool_max_bytes": _positive_int(config.get("ROOTCELL_SPY_SPOOL_MAX_BYTES"), DEFAULT_SPOOL_MAX_BYTES), + } + + +def _spool_size_bytes(path: str) -> int: + total = 0 + try: + with os.scandir(path) as entries: + for entry in entries: + try: + if entry.is_file(follow_symlinks=False): + total += entry.stat(follow_symlinks=False).st_size + except OSError: + continue + except OSError: + return 0 + return total + + +def _filename_part(value: Any) -> str: + cleaned = re.sub(r"[^A-Za-z0-9_.-]+", "_", str(value))[:80].strip("._-") + return cleaned or "none" + + +def _spool_file_name(event: dict[str, Any]) -> str: + return "-".join( + [ + str(time.time_ns()), + str(os.getpid()), + _filename_part(event.get("flow_id", "no-flow")), + _filename_part(event.get("direction", "event")), + str(next(_SPOOL_COUNTER)), + ], + ) + ".json" + + +def _write_all(fd: int, payload: bytes) -> None: + view = memoryview(payload) + while view: + written = os.write(fd, view) + if written <= 0: + raise OSError("short spool write") + view = view[written:] + + +def _atomic_write_spool_file(spool_dir: str, name: str, payload: bytes) -> bool: + tmp_name = f".{name}.tmp" + tmp_path = os.path.join(spool_dir, tmp_name) + final_path = os.path.join(spool_dir, name) + fd: int | None = None try: - fd = os.open(SPY_EVENTS, os.O_APPEND | os.O_CREAT | os.O_WRONLY, 0o666) + fd = os.open(tmp_path, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o640) try: - os.write(fd, line.encode("utf-8")) + _write_all(fd, payload) finally: os.close(fd) + fd = None + os.replace(tmp_path, final_path) + try: + os.chmod(final_path, 0o640) + except OSError: + pass + return True + except OSError: + if fd is not None: + try: + os.close(fd) + except OSError: + pass + try: + os.unlink(tmp_path) + except OSError: + pass + return False + + +def _write_dropped_marker(config: dict[str, Any], provider: str | None, reason: str) -> None: + global _DROPPED_SINCE_MARKER, _LAST_DROPPED_MARKER_AT + _DROPPED_SINCE_MARKER += 1 + now = time.time() + if _LAST_DROPPED_MARKER_AT != 0 and now - _LAST_DROPPED_MARKER_AT < DROPPED_MARKER_INTERVAL_SECONDS: + return + + event: dict[str, Any] = { + "version": 1, + "ts": now, + "direction": "dropped", + "reason": reason, + "dropped_count": _DROPPED_SINCE_MARKER, + } + if provider is not None: + event["provider"] = provider + if _write_spool_event(event, config, write_drop_marker=False): + _DROPPED_SINCE_MARKER = 0 + _LAST_DROPPED_MARKER_AT = now + + +def _write_spool_event(event: dict[str, Any], config: dict[str, Any], write_drop_marker: bool = True) -> bool: + payload = json.dumps(event, ensure_ascii=False, separators=(",", ":")).encode("utf-8") + spool_dir = str(config["spool_dir"]) + max_bytes = int(config["spool_max_bytes"]) + try: + os.makedirs(spool_dir, mode=0o770, exist_ok=True) except OSError: + return False + + if _spool_size_bytes(spool_dir) + len(payload) > max_bytes: + if write_drop_marker: + provider = event.get("provider") + _write_dropped_marker(config, str(provider) if provider is not None else None, "spool_full") + return False + + return _atomic_write_spool_file(spool_dir, _spool_file_name(event), payload) + + +def _bedrock_info_for_flow(flow: Any) -> dict[str, str] | None: + metadata = getattr(flow, "metadata", None) + info = metadata.get("agent_spy") if isinstance(metadata, dict) else None + if isinstance(info, dict) and info.get("provider") == "bedrock": + return {str(key): str(value) for key, value in info.items()} + + request = getattr(flow, "request", None) + if request is None: + return None + return detect_bedrock_request( + getattr(request, "pretty_host", None) or getattr(request, "host", None), + str(getattr(request, "path", "")), + getattr(request, "headers", None), + ) + + +def _flow_id(flow: Any) -> str | None: + value = getattr(flow, "id", None) + return str(value) if value is not None else None + + +def _attach_body(event: dict[str, Any], body: bytes, *, force_encoding: str | None = None) -> None: + if force_encoding == "aws-eventstream": + event["body_b64"] = base64.b64encode(body).decode("ascii") + event["body_sha256"] = _sha256_bytes(body) + event["body_encoding"] = "aws-eventstream" + return + + text = _decode_utf8(body) + if text is None: + event["body_b64"] = base64.b64encode(body).decode("ascii") + event["body_sha256"] = _sha256_bytes(body) + else: + event["body_text"] = text + + +def _write_shim_error(flow: Any, message: str) -> None: + try: + config = _capture_config() + if config is None: + return + event: dict[str, Any] = { + "version": 1, + "ts": time.time(), + "direction": "error", + "error": message, + } + flow_id = _flow_id(flow) + if flow_id is not None: + event["flow_id"] = flow_id + info = _bedrock_info_for_flow(flow) + if info is None: + return + event["provider"] = "bedrock" + _write_spool_event(event, config) + except Exception: # The spy tap must never interfere with user traffic. return @@ -285,7 +494,8 @@ def capture_request(flow: Any) -> None: """mitmproxy hook helper. Capture a validated Bedrock request if enabled.""" try: - if not os.path.exists(SPY_ENABLED): + config = _capture_config() + if config is None: return request = flow.request info = detect_bedrock_request( @@ -295,8 +505,6 @@ def capture_request(flow: Any) -> None: ) if not info: return - if not _is_json_content_type(_header_value(getattr(request, "headers", None), "content-type")): - return metadata = getattr(flow, "metadata", None) if isinstance(metadata, dict): @@ -305,32 +513,20 @@ def capture_request(flow: Any) -> None: body = _request_body_bytes(flow) event = _event_base(flow, "request", info) event["headers"] = redact_headers(getattr(request, "headers", None)) - text = _decode_utf8(body) - if text is None: - event["body_b64"] = base64.b64encode(body).decode("ascii") - event["body_sha256"] = _sha256_bytes(body) - else: - event["body_text"] = text - _write_event(event) + _attach_body(event, body) + _write_spool_event(event, config) except Exception as exc: # pragma: no cover - defensive for live traffic. - _write_event({"version": 1, "ts": time.time(), "direction": "error", "error": str(exc)}) + _write_shim_error(flow, str(exc)) def capture_response(flow: Any) -> None: """mitmproxy hook helper. Capture a Bedrock response if its request matched.""" try: - if not os.path.exists(SPY_ENABLED): + config = _capture_config() + if config is None: return - metadata = getattr(flow, "metadata", None) - info = metadata.get("agent_spy") if isinstance(metadata, dict) else None - if not info: - request = flow.request - info = detect_bedrock_request( - getattr(request, "pretty_host", None) or getattr(request, "host", None), - str(getattr(request, "path", "")), - getattr(request, "headers", None), - ) + info = _bedrock_info_for_flow(flow) if not info or getattr(flow, "response", None) is None: return @@ -344,19 +540,39 @@ def capture_response(flow: Any) -> None: body = _response_body_bytes(flow) content_type = _header_value(getattr(response, "headers", None), "content-type") if _is_eventstream_content_type(content_type): - event["body_b64"] = base64.b64encode(body).decode("ascii") - event["body_sha256"] = _sha256_bytes(body) - event["body_encoding"] = "aws-eventstream" + _attach_body(event, body, force_encoding="aws-eventstream") else: - text = _decode_utf8(body) - if text is None: - event["body_b64"] = base64.b64encode(body).decode("ascii") - event["body_sha256"] = _sha256_bytes(body) - else: - event["body_text"] = text - _write_event(event) + _attach_body(event, body) + _write_spool_event(event, config) except Exception as exc: # pragma: no cover - defensive for live traffic. - _write_event({"version": 1, "ts": time.time(), "direction": "error", "error": str(exc)}) + _write_shim_error(flow, str(exc)) + + +def capture_error(flow: Any) -> None: + """mitmproxy hook helper. Capture Bedrock flow errors if spy is enabled.""" + + try: + config = _capture_config() + if config is None: + return + info = _bedrock_info_for_flow(flow) + if info is None: + return + flow_error = getattr(flow, "error", None) + message = getattr(flow_error, "msg", None) or str(flow_error or "mitmproxy flow error") + event: dict[str, Any] = { + "version": 1, + "ts": time.time(), + "direction": "error", + "provider": "bedrock", + "error": str(message), + } + flow_id = _flow_id(flow) + if flow_id is not None: + event["flow_id"] = flow_id + _write_spool_event(event, config) + except Exception: + return def _looks_base64(value: str) -> bool: diff --git a/proxy/mitmproxy_addon.py b/proxy/mitmproxy_addon.py index 36e94ff..43fd560 100644 --- a/proxy/mitmproxy_addon.py +++ b/proxy/mitmproxy_addon.py @@ -336,3 +336,8 @@ def request(flow: http.HTTPFlow) -> None: def response(flow: http.HTTPFlow) -> None: if agent_spy is not None: agent_spy.capture_response(flow) + + +def error(flow: http.HTTPFlow) -> None: + if agent_spy is not None: + agent_spy.capture_error(flow) diff --git a/proxy/test_agent_spy.py b/proxy/test_agent_spy.py index 67ac044..a744342 100644 --- a/proxy/test_agent_spy.py +++ b/proxy/test_agent_spy.py @@ -2,8 +2,10 @@ import binascii import json import os -import sys import struct +import sys +import tempfile +import types import unittest sys.path.insert(0, os.path.dirname(__file__)) @@ -27,7 +29,43 @@ def eventstream_message(headers, payload): return without_message_crc + message_crc -class AgentSpyTests(unittest.TestCase): +def make_flow( + flow_id="flow-1", + host="bedrock-runtime.us-east-1.amazonaws.com", + path="/model/anthropic.claude/converse-stream", + request_headers=None, + request_body=b'{"messages":[]}', + response_headers=None, + response_body=b'{"output":{}}', + status_code=200, +): + request = types.SimpleNamespace( + pretty_host=host, + host=host, + method="POST", + path=path, + headers=request_headers + or [ + ("Content-Type", "application/json"), + ("Authorization", "AWS4-HMAC-SHA256 Credential=AKIA/..., Signature=abc"), + ], + raw_content=request_body, + ) + response = types.SimpleNamespace( + status_code=status_code, + reason="OK", + headers=response_headers or [("Content-Type", "application/json")], + raw_content=response_body, + ) + return types.SimpleNamespace( + id=flow_id, + request=request, + response=response, + metadata={}, + ) + + +class AgentSpyDetectionTests(unittest.TestCase): def test_detects_bedrock_runtime_paths(self): info = agent_spy.detect_bedrock_request( "bedrock-runtime.us-west-2.amazonaws.com", @@ -75,84 +113,10 @@ def test_binary_fields_are_summarized(self): self.assertIn("image/png base64", summarized["source"]["data"]) self.assertIn("sha256:", summarized["source"]["data"]) - def test_converse_formatting_and_cache_dedupe(self): - formatter = agent_spy.SpyFormatter() - event = { - "ts": 1, - "direction": "request", - "provider": "bedrock", - "operation": "converse", - "model_id": "anthropic.claude", - "host": "bedrock-runtime.us-east-1.amazonaws.com", - "method": "POST", - "path": "/model/anthropic.claude/converse", - "headers": [["Content-Type", "application/json"]], - "body_text": json.dumps( - { - "system": [{"text": "system prompt"}, {"cachePoint": {"type": "default"}}], - "messages": [{"role": "user", "content": [{"text": "dynamic question"}]}], - "toolConfig": { - "tools": [ - { - "toolSpec": { - "name": "shell", - "description": "run commands", - "inputSchema": {"json": {"type": "object"}}, - } - } - ] - }, - } - ), - } - - first = formatter.format_event(event) - second = formatter.format_event(event) - self.assertIn("system prompt", first) - self.assertIn("dynamic question", first) - self.assertIn("tools:", first) - self.assertIn("cached prefix", second) - self.assertNotIn("system prompt", second) - self.assertIn("dynamic question", second) - - def test_claude_invoke_formatting(self): - formatter = agent_spy.SpyFormatter(raw=True) - event = { - "ts": 1, - "direction": "request", - "provider": "bedrock", - "operation": "invoke", - "model_id": "anthropic.claude", - "host": "bedrock-runtime.us-east-1.amazonaws.com", - "method": "POST", - "path": "/model/anthropic.claude/invoke", - "headers": [["Content-Type", "application/json"]], - "body_text": json.dumps( - { - "anthropic_version": "bedrock-2023-05-31", - "system": [{"type": "text", "text": "be useful", "cache_control": {"type": "ephemeral"}}], - "messages": [ - { - "role": "user", - "content": [{"type": "text", "text": "hello"}], - } - ], - "tools": [{"name": "lookup", "input_schema": {"type": "object"}}], - } - ), - } - out = formatter.format_event(event) - self.assertIn("anthropic_version", out) - self.assertIn("be useful", out) - self.assertIn("lookup", out) - self.assertIn("raw request body", out) - - def test_eventstream_decoding_and_formatting(self): + def test_eventstream_decoding(self): payloads = [ {"messageStart": {"role": "assistant"}}, {"contentBlockDelta": {"delta": {"text": "Hello"}}}, - {"contentBlockDelta": {"delta": {"text": " world"}}}, - {"messageStop": {"stopReason": "end_turn"}}, {"metadata": {"usage": {"inputTokens": 4, "outputTokens": 2}}}, ] stream = b"".join( @@ -165,31 +129,172 @@ def test_eventstream_decoding_and_formatting(self): decoded = agent_spy.decode_event_stream(stream) self.assertEqual(len(decoded), len(payloads)) - formatter = agent_spy.SpyFormatter() - out = formatter.format_event( - { - "ts": 1, - "direction": "response", - "provider": "bedrock", - "operation": "converse-stream", - "model_id": "anthropic.claude", - "status_code": 200, - "headers": [["Content-Type", "application/vnd.amazon.eventstream"]], - "body_encoding": "aws-eventstream", - "body_b64": base64.b64encode(stream).decode("ascii"), - } + +class AgentSpySpoolShimTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.old_spy_env = agent_spy.SPY_ENV + self.old_spool_dir = agent_spy.DEFAULT_SPY_SPOOL_DIR + self.old_max = agent_spy.DEFAULT_SPOOL_MAX_BYTES + self.old_drop_interval = agent_spy.DROPPED_MARKER_INTERVAL_SECONDS + self.spy_env = os.path.join(self.tmp.name, "spy.env") + self.spool_dir = os.path.join(self.tmp.name, "spool") + agent_spy.SPY_ENV = self.spy_env + agent_spy.DEFAULT_SPY_SPOOL_DIR = self.spool_dir + agent_spy.DEFAULT_SPOOL_MAX_BYTES = 1_073_741_824 + agent_spy.DROPPED_MARKER_INTERVAL_SECONDS = 0 + agent_spy._DROPPED_SINCE_MARKER = 0 + agent_spy._LAST_DROPPED_MARKER_AT = 0.0 + + def tearDown(self): + agent_spy.SPY_ENV = self.old_spy_env + agent_spy.DEFAULT_SPY_SPOOL_DIR = self.old_spool_dir + agent_spy.DEFAULT_SPOOL_MAX_BYTES = self.old_max + agent_spy.DROPPED_MARKER_INTERVAL_SECONDS = self.old_drop_interval + agent_spy._DROPPED_SINCE_MARKER = 0 + agent_spy._LAST_DROPPED_MARKER_AT = 0.0 + self.tmp.cleanup() + + def write_config(self, enabled=True, max_bytes=None): + lines = [f"ROOTCELL_SPY_ENABLED={'true' if enabled else 'false'}"] + if max_bytes is not None: + lines.append(f"ROOTCELL_SPY_SPOOL_MAX_BYTES={max_bytes}") + with open(self.spy_env, "w", encoding="utf-8") as f: + f.write("\n".join(lines) + "\n") + + def read_events(self): + if not os.path.exists(self.spool_dir): + return [] + events = [] + for name in sorted(os.listdir(self.spool_dir)): + if not name.endswith(".json"): + continue + with open(os.path.join(self.spool_dir, name), "r", encoding="utf-8") as f: + events.append(json.load(f)) + return events + + def test_disabled_by_default_writes_nothing(self): + agent_spy.capture_request(make_flow()) + self.assertFalse(os.path.exists(self.spool_dir)) + + def test_enabled_config_is_parsed(self): + self.write_config(enabled=True, max_bytes=12345) + config = agent_spy._capture_config() + self.assertIsNotNone(config) + self.assertEqual(config["spool_dir"], self.spool_dir) + self.assertEqual(config["spool_max_bytes"], 12345) + + def test_request_spool_event_shape_and_redaction(self): + self.write_config(enabled=True) + flow = make_flow( + path="/model/anthropic.claude/converse?X-Amz-Signature=secret&ok=1", + request_headers=[ + ("Content-Type", "application/json"), + ("Authorization", "Bearer secret-token"), + ("X-Amz-Security-Token", "session-secret"), + ], + request_body=b'{"messages":[{"role":"user","content":[{"text":"hello"}]}]}', ) - self.assertIn("Hello world", out) - self.assertIn("usage: input=4, output=2", out) - self.assertIn("stop: end_turn", out) - - def test_tail_keyboard_interrupt_exits_cleanly(self): - original = agent_spy._tail_events - try: - agent_spy._tail_events = lambda path, formatter: (_ for _ in ()).throw(KeyboardInterrupt()) - self.assertEqual(agent_spy.main(["tail", "--events", "/tmp/missing"]), 130) - finally: - agent_spy._tail_events = original + + agent_spy.capture_request(flow) + + events = self.read_events() + self.assertEqual(len(events), 1) + event = events[0] + self.assertEqual(event["version"], 1) + self.assertEqual(event["direction"], "request") + self.assertEqual(event["flow_id"], "flow-1") + self.assertEqual(event["provider"], "bedrock") + self.assertEqual(event["operation"], "converse") + self.assertEqual(event["model_id"], "anthropic.claude") + self.assertIn("ok=1", event["path"]) + self.assertNotIn("secret", event["path"]) + self.assertEqual( + [pair for pair in event["headers"] if pair[0].lower() == "authorization"], + [["Authorization", "[redacted]"]], + ) + self.assertEqual(json.loads(event["body_text"])["messages"][0]["role"], "user") + self.assertEqual(flow.metadata["agent_spy"]["operation"], "converse") + + def test_non_bedrock_request_writes_nothing(self): + self.write_config(enabled=True) + agent_spy.capture_request(make_flow(host="api.anthropic.com")) + self.assertEqual(self.read_events(), []) + + def test_response_eventstream_is_spooled_as_b64(self): + self.write_config(enabled=True) + stream = eventstream_message( + {":message-type": "event", ":event-type": "chunk", ":content-type": "application/json"}, + b'{"metadata":{"usage":{"inputTokens":4}}}', + ) + flow = make_flow( + response_headers=[("Content-Type", "application/vnd.amazon.eventstream")], + response_body=stream, + ) + + agent_spy.capture_response(flow) + + event = self.read_events()[0] + self.assertEqual(event["direction"], "response") + self.assertEqual(event["status_code"], 200) + self.assertEqual(event["request_headers"][0][0], "Content-Type") + self.assertEqual(event["body_encoding"], "aws-eventstream") + self.assertEqual(base64.b64decode(event["body_b64"]), stream) + self.assertEqual(event["body_sha256"], agent_spy._sha256_bytes(stream)) + + def test_mitmproxy_error_event_is_provider_gated(self): + self.write_config(enabled=True) + flow = make_flow() + flow.error = types.SimpleNamespace(msg="upstream failed") + + agent_spy.capture_error(flow) + + events = self.read_events() + self.assertEqual(len(events), 1) + self.assertEqual(events[0]["direction"], "error") + self.assertEqual(events[0]["provider"], "bedrock") + self.assertEqual(events[0]["error"], "upstream failed") + + def test_non_bedrock_error_writes_nothing(self): + self.write_config(enabled=True) + flow = make_flow(host="api.anthropic.com") + flow.error = types.SimpleNamespace(msg="not a captured provider") + + agent_spy.capture_error(flow) + + self.assertEqual(self.read_events(), []) + + def test_spool_cap_can_suppress_all_writes(self): + self.write_config(enabled=True, max_bytes=10) + agent_spy.capture_request(make_flow(request_body=b'{"messages":[{"content":[{"text":"large"}]}]}')) + self.assertEqual(self.read_events(), []) + + def test_spool_full_writes_rate_limited_dropped_marker_when_it_fits(self): + self.write_config(enabled=True, max_bytes=1200) + os.makedirs(self.spool_dir, exist_ok=True) + with open(os.path.join(self.spool_dir, "existing.dat"), "wb") as f: + f.write(b"x" * 1000) + + agent_spy.capture_request(make_flow(request_body=b'{"messages":[{"content":[{"text":"' + b"x" * 3000 + b'"}]}]}')) + + events = self.read_events() + self.assertEqual(len(events), 1) + self.assertEqual(events[0]["direction"], "dropped") + self.assertEqual(events[0]["provider"], "bedrock") + self.assertEqual(events[0]["reason"], "spool_full") + self.assertEqual(events[0]["dropped_count"], 1) + + def test_capture_errors_are_swallowed(self): + self.write_config(enabled=True) + + class BadFlow: + id = "bad-flow" + + @property + def request(self): + raise RuntimeError("broken request") + + agent_spy.capture_request(BadFlow()) if __name__ == "__main__": From 9b363333975c5a74b87668e0d0fbe61484df23a5 Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Fri, 22 May 2026 21:20:29 -0400 Subject: [PATCH 04/52] Add Bedrock spy adapter --- PLAN.md | 17 +- src/spy/bedrock.test.ts | 195 +++++++++ src/spy/bedrock.ts | 912 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 1123 insertions(+), 1 deletion(-) create mode 100644 src/spy/bedrock.test.ts create mode 100644 src/spy/bedrock.ts diff --git a/PLAN.md b/PLAN.md index aa67dfe..d88a974 100644 --- a/PLAN.md +++ b/PLAN.md @@ -447,6 +447,21 @@ V1 excludes: - Added Python unit coverage for disabled/default behavior, config parsing, Bedrock detection, redaction, event-stream response encoding, provider-gated errors, spool cap behavior, dropped markers, and failure swallowing. +- Implemented the TypeScript Bedrock adapter on top of the captured fixtures: + - Added `src/spy/bedrock.ts` with `normalizeBedrockCall` and + `normalizeBedrockSpoolEvents` entrypoints. + - Normalizes paired Bedrock request/response spool events into provider + calls, semantic request/response blocks, provider-reported usage records, + decoded stream events, and opt-in raw payload records. + - Supports the captured Bedrock Converse Stream request shape, AWS + event-stream response decoding, response text reconstruction, tool use + reconstruction, usage extraction, stable IDs, stable content hashes, and + conservative Pi/Bedrock provenance classification. + - Added fixture-backed unit coverage for all five real Pi/Bedrock + request/response pairs, request block classification, response tool/text + extraction, usage extraction, stream events, raw payload gating, and hash + stability. + - Verified `bun run typecheck`, `bun run lint`, and `bun run test`. ### V1 @@ -457,7 +472,7 @@ Build the Bedrock/Pi browser spy: adapter work. - [x] Add initial AWS event-stream decoder. - [x] Replace Python spy with minimal provider-gated spool shim. -- [ ] Implement TypeScript Bedrock adapter on top of the captured fixtures. +- [x] Implement TypeScript Bedrock adapter on top of the captured fixtures. - [ ] Implement SQLite persistence, migrations, retention, and clear-data. - [ ] Implement TS web service, API, SSE, and static asset serving. - [ ] Implement React desktop UI with virtualized timeline and call inspector. diff --git a/src/spy/bedrock.test.ts b/src/spy/bedrock.test.ts new file mode 100644 index 0000000..c9e649e --- /dev/null +++ b/src/spy/bedrock.test.ts @@ -0,0 +1,195 @@ +import { readFileSync } from "node:fs"; +import { describe, expect, test } from "vitest"; +import { + normalizeBedrockCall, + normalizeBedrockSpoolEvents, + type NormalizedProviderCall, +} from "./bedrock.ts"; +import { + NormalizedBlockSchema, + ProviderCallSchema, + RawPayloadRecordSchema, + SpoolEventSchema, + StreamEventSchema, + UsageRecordSchema, + type NormalizedBlock, + type SpoolEvent, + type SpoolRequestEvent, + type SpoolResponseEvent, +} from "./schemas.ts"; + +const FIXTURE_PATH = new URL("./fixtures/bedrock-pi-us-sonnet-4-6.ndjson", import.meta.url); + +function fixtureEvents(): SpoolEvent[] { + return readFileSync(FIXTURE_PATH, "utf8") + .trim() + .split("\n") + .filter(Boolean) + .map((line) => SpoolEventSchema.parse(JSON.parse(line) as unknown)); +} + +function fixturePair(flowId: string): readonly [SpoolRequestEvent, SpoolResponseEvent] { + const events = fixtureEvents(); + const request = events.find((event): event is SpoolRequestEvent => ( + event.direction === "request" && event.flow_id === flowId + )); + const response = events.find((event): event is SpoolResponseEvent => ( + event.direction === "response" && event.flow_id === flowId + )); + if (request === undefined || response === undefined) { + throw new Error(`missing fixture pair ${flowId}`); + } + return [request, response]; +} + +function callById(calls: readonly NormalizedProviderCall[], id: string): NormalizedProviderCall { + const call = calls.find((candidate) => candidate.call.id === id); + if (call === undefined) { + throw new Error(`missing normalized call ${id}`); + } + return call; +} + +function blocks(call: NormalizedProviderCall, direction: "request" | "response"): NormalizedBlock[] { + return call.blocks.filter((block) => block.direction === direction); +} + +function firstBlock( + call: NormalizedProviderCall, + direction: "request" | "response", + kind: NormalizedBlock["kind"], +): NormalizedBlock { + const block = call.blocks.find((candidate) => ( + candidate.direction === direction && candidate.kind === kind + )); + if (block === undefined) { + throw new Error(`missing ${direction} ${kind} block in ${call.call.id}`); + } + return block; +} + +describe("Bedrock adapter", () => { + test("normalizes all complete fixture request/response pairs", () => { + const calls = normalizeBedrockSpoolEvents(fixtureEvents()); + expect(calls).toHaveLength(5); + expect(calls.map((call) => call.call.id)).toEqual([ + "call-fixture-flow-simple", + "call-fixture-flow-session-turn-one", + "call-fixture-flow-session-turn-two", + "call-fixture-flow-tool-use", + "call-fixture-flow-tool-result", + ]); + + for (const normalized of calls) { + expect(() => ProviderCallSchema.parse(normalized.call)).not.toThrow(); + for (const block of normalized.blocks) { + expect(() => NormalizedBlockSchema.parse(block)).not.toThrow(); + } + for (const usage of normalized.usage) { + expect(() => UsageRecordSchema.parse(usage)).not.toThrow(); + } + for (const event of normalized.streamEvents) { + expect(() => StreamEventSchema.parse(event)).not.toThrow(); + } + expect(normalized.rawPayloads).toEqual([]); + expect(normalized.call.status).toBe("complete"); + expect(normalized.call.model_id).toBe("us.anthropic.claude-sonnet-4-6"); + expect(normalized.call.operation).toBe("converse-stream"); + expect(normalized.call.request_content_hash).toMatch(/^[a-f0-9]{64}$/); + expect(normalized.call.response_content_hash).toMatch(/^[a-f0-9]{64}$/); + } + + const simple = callById(calls, "call-fixture-flow-simple"); + expect(simple.call).toMatchObject({ + status_code: 200, + request_flow_id: "fixture-flow-simple", + response_flow_id: "fixture-flow-simple", + }); + }); + + test("classifies request system, current input, history, tools, and cache markers", () => { + const calls = normalizeBedrockSpoolEvents(fixtureEvents()); + const simple = callById(calls, "call-fixture-flow-simple"); + const simpleRequest = blocks(simple, "request"); + expect(simpleRequest.filter((block) => block.kind === "current-user-input")).toHaveLength(1); + expect(simpleRequest.filter((block) => block.kind === "harness-system-context")).toHaveLength(1); + expect(simpleRequest.filter((block) => block.kind === "tool-definition")).toHaveLength(4); + expect(simpleRequest.filter((block) => block.kind === "cache-marker")).toHaveLength(2); + expect(firstBlock(simple, "request", "current-user-input").text).toContain("Fixture capture simple prompt"); + + const turnTwo = callById(calls, "call-fixture-flow-session-turn-two"); + const turnTwoRequest = blocks(turnTwo, "request"); + expect(turnTwoRequest.filter((block) => block.kind === "prior-conversation-history")).toHaveLength(2); + expect(firstBlock(turnTwo, "request", "current-user-input").text).toContain("session turn two"); + expect(JSON.stringify(turnTwoRequest.map((block) => block.text))).toContain("turn-one-ok"); + expect(JSON.stringify(turnTwoRequest.map((block) => block.text))).toContain("RCSPY-ALPHA"); + + const toolResult = callById(calls, "call-fixture-flow-tool-result"); + expect(blocks(toolResult, "request").map((block) => block.kind)).toEqual(expect.arrayContaining([ + "tool-call", + "tool-result", + "cache-marker", + ])); + expect(firstBlock(toolResult, "request", "tool-result").text).toContain("success"); + }); + + test("reconstructs response text, tool use, provider usage, and stream events", () => { + const calls = normalizeBedrockSpoolEvents(fixtureEvents()); + const simple = callById(calls, "call-fixture-flow-simple"); + expect(firstBlock(simple, "response", "assistant-output").text).toBe("fixture-simple-ok"); + expect(firstBlock(simple, "response", "provider-envelope").text).toBe("stopReason:end_turn"); + + const simpleUsage = simple.usage[0]; + if (simpleUsage === undefined) { + throw new Error("missing simple usage record"); + } + expect(simpleUsage).toMatchObject({ + source: "provider-reported", + input_tokens: 1936, + output_tokens: 8, + total_tokens: 1944, + }); + expect(simple.streamEvents.map((event) => event.event_type)).toEqual([ + "messageStart", + "contentBlockDelta", + "contentBlockDelta", + "contentBlockStop", + "messageStop", + "metadata", + ]); + + const toolUse = callById(calls, "call-fixture-flow-tool-use"); + const toolCall = firstBlock(toolUse, "response", "tool-call"); + expect(toolCall.text).toContain("bash"); + expect(JSON.stringify(toolCall.json)).toContain("printf"); + expect(JSON.stringify(toolCall.json)).toContain("tool-fixture-ok"); + expect(toolUse.streamEvents.map((event) => event.event_type)).toEqual(expect.arrayContaining([ + "contentBlockStart", + "contentBlockDelta", + "messageStop", + "metadata", + ])); + }); + + test("preserves raw payloads only when requested and keeps hashes stable", () => { + const events = fixtureEvents(); + const firstPass = normalizeBedrockSpoolEvents(events); + const secondPass = normalizeBedrockSpoolEvents(events); + expect(secondPass.map((call) => call.call.request_content_hash)).toEqual( + firstPass.map((call) => call.call.request_content_hash), + ); + expect(secondPass.flatMap((call) => call.blocks.map((block) => block.content_hash))).toEqual( + firstPass.flatMap((call) => call.blocks.map((block) => block.content_hash)), + ); + + const [request, response] = fixturePair("fixture-flow-simple"); + const raw = normalizeBedrockCall(request, response, { storeRaw: true }); + expect(raw.rawPayloads).toHaveLength(2); + for (const payload of raw.rawPayloads) { + expect(() => RawPayloadRecordSchema.parse(payload)).not.toThrow(); + } + expect(raw.rawPayloads.map((payload) => payload.direction)).toEqual(["request", "response"]); + expect(raw.rawPayloads[0]?.body_text).toContain("Fixture capture simple prompt"); + expect(raw.rawPayloads[1]?.body_encoding).toBe("aws-eventstream"); + }); +}); diff --git a/src/spy/bedrock.ts b/src/spy/bedrock.ts new file mode 100644 index 0000000..f1e860f --- /dev/null +++ b/src/spy/bedrock.ts @@ -0,0 +1,912 @@ +import { createHash } from "node:crypto"; +import { decodeAwsEventStreamJson } from "./eventstream.ts"; +import type { + NormalizedBlock, + ProviderCall, + RawPayloadRecord, + SpoolEvent, + SpoolRequestEvent, + SpoolResponseEvent, + StreamEvent, + UsageRecord, +} from "./schemas.ts"; + +export interface BedrockAdapterOptions { + readonly storeRaw?: boolean; +} + +export interface NormalizedProviderCall { + readonly call: ProviderCall; + readonly blocks: readonly NormalizedBlock[]; + readonly usage: readonly UsageRecord[]; + readonly streamEvents: readonly StreamEvent[]; + readonly rawPayloads: readonly RawPayloadRecord[]; +} + +type BlockKind = NormalizedBlock["kind"]; +type Direction = NormalizedBlock["direction"]; + +interface BlockInput { + readonly callId: string; + readonly direction: Direction; + readonly ordinal: number; + readonly kind: BlockKind; + readonly source: string; + readonly role?: string | undefined; + readonly providerPath?: string | undefined; + readonly text?: string | undefined; + readonly json?: unknown; + readonly cacheMarker?: boolean; +} + +interface ResponseBlockBuilder { + readonly index: number; + readonly providerPath: string; + readonly textParts: string[]; + readonly thinkingParts: string[]; + readonly toolInputParts: string[]; + readonly unknownValues: unknown[]; + role?: string | undefined; + toolUseStart?: Record; +} + +interface JsonParseResult { + readonly ok: boolean; + readonly value?: unknown; +} + +export function normalizeBedrockCall( + request: SpoolRequestEvent, + response: SpoolResponseEvent, + options: BedrockAdapterOptions = {}, +): NormalizedProviderCall { + if (request.flow_id !== response.flow_id) { + throw new Error(`cannot normalize Bedrock call with mismatched flow ids: ${request.flow_id} != ${response.flow_id}`); + } + if (request.operation !== response.operation || request.model_id !== response.model_id) { + throw new Error(`cannot normalize Bedrock call with mismatched request/response metadata for flow ${request.flow_id}`); + } + + const callId = stableId("call", request.flow_id); + const requestBody = parseRequestBody(request); + const requestBlocks = normalizeRequestBlocks(callId, requestBody); + const responseNormalization = normalizeResponse(callId, response); + const blocks = [...requestBlocks, ...responseNormalization.blocks]; + const status = response.status_code >= 400 ? "error" : "complete"; + + return { + call: { + id: callId, + provider: "bedrock", + operation: request.operation, + model_id: request.model_id, + status, + started_at: request.ts, + completed_at: response.ts, + status_code: response.status_code, + request_flow_id: request.flow_id, + response_flow_id: response.flow_id, + request_content_hash: hashUnknown(requestBody), + response_content_hash: hashUnknown(responseNormalization.blocks.map((block) => block.content_hash)), + }, + blocks, + usage: responseNormalization.usage, + streamEvents: responseNormalization.streamEvents, + rawPayloads: options.storeRaw === true ? rawPayloads(callId, request, response) : [], + }; +} + +export function normalizeBedrockSpoolEvents( + events: readonly SpoolEvent[], + options: BedrockAdapterOptions = {}, +): NormalizedProviderCall[] { + const requests = new Map(); + const responses = new Map(); + + for (const event of events) { + if (event.direction === "request") { + if (requests.has(event.flow_id)) { + throw new Error(`duplicate Bedrock request event for flow ${event.flow_id}`); + } + requests.set(event.flow_id, event); + } else if (event.direction === "response") { + if (responses.has(event.flow_id)) { + throw new Error(`duplicate Bedrock response event for flow ${event.flow_id}`); + } + responses.set(event.flow_id, event); + } + } + + for (const flowId of responses.keys()) { + if (!requests.has(flowId)) { + throw new Error(`Bedrock response event has no matching request for flow ${flowId}`); + } + } + + return [...requests.values()] + .filter((request) => responses.has(request.flow_id)) + .sort((left, right) => left.ts - right.ts || left.flow_id.localeCompare(right.flow_id)) + .map((request) => { + const response = responses.get(request.flow_id); + if (response === undefined) { + throw new Error(`Bedrock request event has no matching response for flow ${request.flow_id}`); + } + return normalizeBedrockCall(request, response, options); + }); +} + +function normalizeRequestBlocks(callId: string, body: Record): NormalizedBlock[] { + const blocks: NormalizedBlock[] = []; + let ordinal = 0; + const messages = arrayField(body, "messages").filter(isRecord); + const lastUserMessageIndex = findLastUserMessageIndex(messages); + + const addBlock = (input: Omit): void => { + blocks.push(createBlock({ + callId, + direction: "request", + ordinal, + ...input, + })); + ordinal += 1; + }; + + for (const [key, value] of Object.entries(body)) { + if (key === "messages") { + normalizeRequestMessages(value, lastUserMessageIndex, addBlock); + } else if (key === "system") { + normalizeSystemBlocks(value, addBlock); + } else if (key === "toolConfig") { + normalizeToolConfig(value, addBlock); + } else if (key === "inferenceConfig" || key === "additionalModelRequestFields") { + addBlock({ + kind: "provider-envelope", + source: "bedrock-converse-request", + providerPath: `$.${key}`, + text: key, + json: value, + }); + } else { + addBlock({ + kind: "unknown", + source: "bedrock-converse-request", + providerPath: `$.${key}`, + text: key, + json: value, + }); + } + } + + return blocks; +} + +function normalizeSystemBlocks( + value: unknown, + addBlock: (input: Omit) => void, +): void { + if (!Array.isArray(value)) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-system", + providerPath: "$.system", + text: "system", + json: value, + }); + return; + } + + value.forEach((item, index) => { + if (!isRecord(item)) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-system", + providerPath: `$.system[${String(index)}]`, + json: item, + }); + return; + } + + const text = stringField(item, "text"); + if (text !== undefined) { + addBlock({ + kind: "harness-system-context", + source: "pi-bedrock-system", + providerPath: `$.system[${String(index)}].text`, + text, + }); + return; + } + + const cachePoint = item.cachePoint; + if (cachePoint !== undefined) { + addBlock({ + kind: "cache-marker", + source: "bedrock-converse-cache-point", + providerPath: `$.system[${String(index)}].cachePoint`, + text: cachePointText(cachePoint), + json: cachePoint, + cacheMarker: true, + }); + return; + } + + addBlock({ + kind: "unknown", + source: "bedrock-converse-system", + providerPath: `$.system[${String(index)}]`, + json: item, + }); + }); +} + +function normalizeRequestMessages( + value: unknown, + lastUserMessageIndex: number, + addBlock: (input: Omit) => void, +): void { + if (!Array.isArray(value)) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-messages", + providerPath: "$.messages", + text: "messages", + json: value, + }); + return; + } + + value.forEach((message, messageIndex) => { + if (!isRecord(message)) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-message", + providerPath: `$.messages[${String(messageIndex)}]`, + json: message, + }); + return; + } + + const role = stringField(message, "role"); + const content = arrayField(message, "content"); + if (content.length === 0) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-message", + providerPath: `$.messages[${String(messageIndex)}]`, + role, + json: message, + }); + return; + } + + content.forEach((item, contentIndex) => { + const providerPath = `$.messages[${String(messageIndex)}].content[${String(contentIndex)}]`; + normalizeMessageContentBlock(item, { + role, + providerPath, + isCurrentUserMessage: role === "user" && messageIndex === lastUserMessageIndex, + addBlock, + }); + }); + }); +} + +function normalizeMessageContentBlock( + item: unknown, + context: { + readonly role?: string | undefined; + readonly providerPath: string; + readonly isCurrentUserMessage: boolean; + readonly addBlock: (input: Omit) => void; + }, +): void { + if (!isRecord(item)) { + context.addBlock({ + kind: "unknown", + source: "bedrock-converse-message", + providerPath: context.providerPath, + role: context.role, + json: item, + }); + return; + } + + const text = stringField(item, "text"); + if (text !== undefined) { + context.addBlock({ + kind: context.isCurrentUserMessage ? "current-user-input" : "prior-conversation-history", + source: "bedrock-converse-message", + providerPath: `${context.providerPath}.text`, + role: context.role, + text, + }); + return; + } + + const cachePoint = item.cachePoint; + if (cachePoint !== undefined) { + context.addBlock({ + kind: "cache-marker", + source: "bedrock-converse-cache-point", + providerPath: `${context.providerPath}.cachePoint`, + role: context.role, + text: cachePointText(cachePoint), + json: cachePoint, + cacheMarker: true, + }); + return; + } + + const toolUse = item.toolUse; + if (toolUse !== undefined) { + context.addBlock({ + kind: "tool-call", + source: "bedrock-converse-message", + providerPath: `${context.providerPath}.toolUse`, + role: context.role, + text: toolUseText(toolUse), + json: toolUse, + }); + return; + } + + const toolResult = item.toolResult; + if (toolResult !== undefined) { + context.addBlock({ + kind: "tool-result", + source: "bedrock-converse-message", + providerPath: `${context.providerPath}.toolResult`, + role: context.role, + text: toolResultText(toolResult), + json: toolResult, + }); + return; + } + + context.addBlock({ + kind: "unknown", + source: "bedrock-converse-message", + providerPath: context.providerPath, + role: context.role, + json: item, + }); +} + +function normalizeToolConfig( + value: unknown, + addBlock: (input: Omit) => void, +): void { + if (!isRecord(value)) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-tool-config", + providerPath: "$.toolConfig", + text: "toolConfig", + json: value, + }); + return; + } + + const tools = arrayField(value, "tools"); + if (tools.length === 0) { + addBlock({ + kind: "provider-envelope", + source: "bedrock-converse-tool-config", + providerPath: "$.toolConfig", + text: "toolConfig", + json: value, + }); + return; + } + + tools.forEach((tool, index) => { + addBlock({ + kind: "tool-definition", + source: "bedrock-converse-tool-config", + providerPath: `$.toolConfig.tools[${String(index)}]`, + text: toolDefinitionText(tool), + json: tool, + }); + }); +} + +function normalizeResponse( + callId: string, + response: SpoolResponseEvent, +): { + readonly blocks: readonly NormalizedBlock[]; + readonly usage: readonly UsageRecord[]; + readonly streamEvents: readonly StreamEvent[]; +} { + const decoded = decodeAwsEventStreamJson(responseBodyB64(response)); + const blocks: NormalizedBlock[] = []; + const usage: UsageRecord[] = []; + const builders = new Map(); + const finalized = new Set(); + let ordinal = 0; + let responseRole: string | undefined; + + const addBlock = (input: Omit): void => { + blocks.push(createBlock({ + callId, + direction: "response", + ordinal, + ...input, + })); + ordinal += 1; + }; + + const getBuilder = (index: number, providerPath: string): ResponseBlockBuilder => { + const existing = builders.get(index); + if (existing !== undefined) { + return existing; + } + const builder: ResponseBlockBuilder = { + index, + providerPath, + textParts: [], + thinkingParts: [], + toolInputParts: [], + unknownValues: [], + ...(responseRole === undefined ? {} : { role: responseRole }), + }; + builders.set(index, builder); + return builder; + }; + + const finalizeBuilder = (index: number): void => { + if (finalized.has(index)) { + return; + } + const builder = builders.get(index); + if (builder === undefined) { + return; + } + finalized.add(index); + + const role = builder.role ?? responseRole; + if (builder.textParts.length > 0) { + addBlock({ + kind: "assistant-output", + source: "bedrock-converse-response", + providerPath: builder.providerPath, + role, + text: builder.textParts.join(""), + }); + } + + if (builder.thinkingParts.length > 0) { + addBlock({ + kind: "thinking", + source: "bedrock-converse-response", + providerPath: builder.providerPath, + role, + text: builder.thinkingParts.join(""), + }); + } + + if (builder.toolUseStart !== undefined) { + const inputText = builder.toolInputParts.join(""); + const toolUse: Record = { ...builder.toolUseStart }; + if (inputText.length > 0) { + toolUse.inputText = inputText; + const parsed = parseJson(inputText); + toolUse.input = parsed.ok ? parsed.value : inputText; + } + addBlock({ + kind: "tool-call", + source: "bedrock-converse-response", + providerPath: builder.providerPath, + role, + text: toolUseText(toolUse), + json: { toolUse }, + }); + } + + if (builder.unknownValues.length > 0) { + addBlock({ + kind: "unknown", + source: "bedrock-converse-response", + providerPath: builder.providerPath, + role, + json: builder.unknownValues, + }); + } + }; + + decoded.forEach((message, eventIndex) => { + const eventType = stringField(message.headers, ":event-type") ?? "unknown"; + const payload = message.payload; + if (eventType === "messageStart" && isRecord(payload)) { + responseRole = stringField(payload, "role") ?? responseRole; + return; + } + + if (eventType === "contentBlockStart" && isRecord(payload)) { + const blockIndex = numberField(payload, "contentBlockIndex"); + if (blockIndex === undefined) { + return; + } + const builder = getBuilder(blockIndex, `$.eventStream[${String(eventIndex)}].payload`); + if (builder.role === undefined && responseRole !== undefined) { + builder.role = responseRole; + } + const start = recordField(payload, "start"); + const toolUse = recordField(start, "toolUse"); + if (toolUse !== undefined) { + builder.toolUseStart = toolUse; + } else if (start !== undefined) { + builder.unknownValues.push(start); + } + return; + } + + if (eventType === "contentBlockDelta" && isRecord(payload)) { + const blockIndex = numberField(payload, "contentBlockIndex"); + const delta = recordField(payload, "delta"); + if (blockIndex === undefined || delta === undefined) { + return; + } + const builder = getBuilder(blockIndex, `$.eventStream[${String(eventIndex)}].payload`); + if (builder.role === undefined && responseRole !== undefined) { + builder.role = responseRole; + } + const text = stringField(delta, "text"); + if (text !== undefined) { + builder.textParts.push(text); + } + const toolUseInput = stringField(recordField(delta, "toolUse"), "input"); + if (toolUseInput !== undefined) { + builder.toolInputParts.push(toolUseInput); + } + const thinking = thinkingText(delta); + if (thinking !== undefined) { + builder.thinkingParts.push(thinking); + } + if (text === undefined && toolUseInput === undefined && thinking === undefined) { + builder.unknownValues.push(delta); + } + return; + } + + if (eventType === "contentBlockStop" && isRecord(payload)) { + const blockIndex = numberField(payload, "contentBlockIndex"); + if (blockIndex !== undefined) { + finalizeBuilder(blockIndex); + } + return; + } + + if (eventType === "messageStop") { + addBlock({ + kind: "provider-envelope", + source: "bedrock-converse-response", + providerPath: `$.eventStream[${String(eventIndex)}].payload`, + role: responseRole, + text: messageStopText(payload), + json: payload, + }); + return; + } + + if (eventType === "metadata") { + const usageRecord = usageRecordFromMetadata(callId, usage.length, payload); + if (usageRecord !== undefined) { + usage.push(usageRecord); + } + addBlock({ + kind: "provider-envelope", + source: "bedrock-converse-response", + providerPath: `$.eventStream[${String(eventIndex)}].payload`, + role: responseRole, + text: metadataText(payload), + json: payload, + }); + } + }); + + for (const index of builders.keys()) { + finalizeBuilder(index); + } + + return { + blocks, + usage, + streamEvents: decoded.map((message, index) => streamEvent(callId, response.ts, index, message.headers, message.payload)), + }; +} + +function createBlock(input: BlockInput): NormalizedBlock { + const material = input.json === undefined ? input.text ?? "" : canonicalJson(input.json); + return { + id: stableId("block", input.callId, input.direction, String(input.ordinal)), + call_id: input.callId, + direction: input.direction, + ordinal: input.ordinal, + kind: input.kind, + source: input.source, + char_size: material.length, + byte_size: Buffer.byteLength(material, "utf8"), + content_hash: sha256(material), + cache_marker: input.cacheMarker ?? false, + ...(input.role === undefined ? {} : { role: input.role }), + ...(input.providerPath === undefined ? {} : { provider_path: input.providerPath }), + ...(input.text === undefined ? {} : { text: input.text }), + ...(input.json === undefined ? {} : { json: input.json }), + }; +} + +function streamEvent( + callId: string, + observedAt: number, + ordinal: number, + headers: Readonly>, + payload: unknown, +): StreamEvent { + const payloadText = streamPayloadText(payload); + return { + id: stableId("stream", callId, String(ordinal)), + call_id: callId, + ordinal, + event_type: stringField(headers, ":event-type") ?? "unknown", + headers: { ...headers }, + observed_at: observedAt, + ...(payload === undefined ? {} : { payload }), + ...(payloadText === undefined ? {} : { payload_text: payloadText }), + ...(payload === undefined ? {} : { payload_sha256: hashUnknown(payload) }), + }; +} + +function usageRecordFromMetadata(callId: string, index: number, payload: unknown): UsageRecord | undefined { + const usage = recordField(payload, "usage"); + if (usage === undefined) { + return undefined; + } + + return { + id: stableId("usage", callId, String(index)), + call_id: callId, + source: "provider-reported", + raw: usage, + ...optionalIntegerField("input_tokens", firstNumber(usage, ["inputTokens", "input_tokens"])), + ...optionalIntegerField("output_tokens", firstNumber(usage, ["outputTokens", "output_tokens"])), + ...optionalIntegerField("cache_read_tokens", firstNumber(usage, [ + "cacheReadTokens", + "cacheReadInputTokens", + "cache_read_tokens", + "cache_read_input_tokens", + ])), + ...optionalIntegerField("cache_write_tokens", firstNumber(usage, [ + "cacheWriteTokens", + "cacheWriteInputTokens", + "cache_write_tokens", + "cache_write_input_tokens", + ])), + ...optionalIntegerField("total_tokens", firstNumber(usage, ["totalTokens", "total_tokens"])), + }; +} + +function rawPayloads( + callId: string, + request: SpoolRequestEvent, + response: SpoolResponseEvent, +): RawPayloadRecord[] { + return [ + { + id: stableId("raw", callId, "request"), + call_id: callId, + direction: "request", + ...(headerValue(request.headers, "content-type") === undefined ? {} : { content_type: headerValue(request.headers, "content-type") }), + ...bodyFields(request), + }, + { + id: stableId("raw", callId, "response"), + call_id: callId, + direction: "response", + ...(headerValue(response.headers, "content-type") === undefined ? {} : { content_type: headerValue(response.headers, "content-type") }), + ...bodyFields(response), + }, + ]; +} + +function bodyFields(event: { + readonly body_text?: string | undefined; + readonly body_b64?: string | undefined; + readonly body_sha256?: string | undefined; + readonly body_encoding?: "aws-eventstream" | undefined; +}): Pick { + return { + ...(event.body_text === undefined ? {} : { body_text: event.body_text }), + ...(event.body_b64 === undefined ? {} : { body_b64: event.body_b64 }), + ...(event.body_sha256 === undefined ? {} : { body_sha256: event.body_sha256 }), + ...(event.body_encoding === undefined ? {} : { body_encoding: event.body_encoding }), + }; +} + +function parseRequestBody(request: SpoolRequestEvent): Record { + const text = capturedBodyText(request); + const parsed = parseJson(text); + if (!parsed.ok || !isRecord(parsed.value)) { + throw new Error(`Bedrock request body for flow ${request.flow_id} is not a JSON object`); + } + return parsed.value; +} + +function responseBodyB64(response: SpoolResponseEvent): string { + if (response.body_encoding !== "aws-eventstream" || response.body_b64 === undefined) { + throw new Error(`Bedrock response body for flow ${response.flow_id} is not an AWS event stream`); + } + return response.body_b64; +} + +function capturedBodyText(event: { readonly body_text?: string | undefined; readonly body_b64?: string | undefined }): string { + if (event.body_text !== undefined) { + return event.body_text; + } + if (event.body_b64 !== undefined) { + return Buffer.from(event.body_b64, "base64").toString("utf8"); + } + throw new Error("capture event has no body"); +} + +function findLastUserMessageIndex(messages: readonly Record[]): number { + for (let index = messages.length - 1; index >= 0; index -= 1) { + const message = messages[index]; + if (message !== undefined && stringField(message, "role") === "user") { + return index; + } + } + return -1; +} + +function cachePointText(value: unknown): string { + const type = stringField(value, "type"); + return type === undefined ? "cachePoint" : `cachePoint:${type}`; +} + +function toolDefinitionText(value: unknown): string { + const toolSpec = recordField(value, "toolSpec"); + const name = stringField(toolSpec, "name"); + const description = stringField(toolSpec, "description"); + return [name, description].filter((part) => part !== undefined && part.length > 0).join(" "); +} + +function toolUseText(value: unknown): string { + const record = isRecord(value) && isRecord(value.toolUse) ? value.toolUse : value; + const name = stringField(record, "name") ?? "toolUse"; + const toolUseId = stringField(record, "toolUseId"); + const inputText = stringField(record, "inputText"); + return [name, toolUseId, inputText].filter((part) => part !== undefined && part.length > 0).join(" "); +} + +function toolResultText(value: unknown): string { + const toolUseId = stringField(value, "toolUseId") ?? "toolResult"; + const status = stringField(value, "status"); + return status === undefined ? toolUseId : `${toolUseId} ${status}`; +} + +function messageStopText(payload: unknown): string { + const stopReason = stringField(payload, "stopReason"); + return stopReason === undefined ? "messageStop" : `stopReason:${stopReason}`; +} + +function metadataText(payload: unknown): string { + const metrics = recordField(payload, "metrics"); + const latencyMs = numberField(metrics, "latencyMs"); + return latencyMs === undefined ? "metadata" : `latencyMs:${String(latencyMs)}`; +} + +function thinkingText(delta: Record): string | undefined { + const direct = stringField(delta, "thinking") ?? stringField(delta, "reasoning"); + if (direct !== undefined) { + return direct; + } + const reasoningContent = recordField(delta, "reasoningContent"); + return stringField(reasoningContent, "text") ?? stringField(reasoningContent, "reasoningText"); +} + +function streamPayloadText(payload: unknown): string | undefined { + if (!isRecord(payload)) { + return undefined; + } + const delta = recordField(payload, "delta"); + return stringField(delta, "text") + ?? stringField(recordField(delta, "toolUse"), "input") + ?? thinkingText(delta ?? {}); +} + +function optionalIntegerField(name: TName, value: number | undefined): Readonly>> { + if (value === undefined || !Number.isInteger(value) || value < 0) { + return {} as Readonly>>; + } + return { [name]: value } as Readonly>>; +} + +function firstNumber(record: Record, names: readonly string[]): number | undefined { + for (const name of names) { + const value = numberField(record, name); + if (value !== undefined) { + return value; + } + } + return undefined; +} + +function headerValue(headers: readonly (readonly [string, string])[], name: string): string | undefined { + const lowerName = name.toLowerCase(); + return headers.find(([candidate]) => candidate.toLowerCase() === lowerName)?.[1]; +} + +function parseJson(text: string): JsonParseResult { + try { + return { ok: true, value: JSON.parse(text) as unknown }; + } catch { + return { ok: false }; + } +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function recordField(value: unknown, key: string): Record | undefined { + if (!isRecord(value)) { + return undefined; + } + const child = value[key]; + return isRecord(child) ? child : undefined; +} + +function arrayField(value: Record, key: string): unknown[] { + const child = value[key]; + return Array.isArray(child) ? child : []; +} + +function stringField(value: unknown, key: string): string | undefined { + if (!isRecord(value)) { + return undefined; + } + const child = value[key]; + return typeof child === "string" ? child : undefined; +} + +function numberField(value: unknown, key: string): number | undefined { + if (!isRecord(value)) { + return undefined; + } + const child = value[key]; + return typeof child === "number" && Number.isFinite(child) ? child : undefined; +} + +function hashUnknown(value: unknown): string { + return sha256(canonicalJson(value)); +} + +function sha256(value: string): string { + return createHash("sha256").update(value).digest("hex"); +} + +function stableId(prefix: string, ...parts: readonly string[]): string { + return [prefix, ...parts.map(idPart)].join("-"); +} + +function idPart(value: string): string { + const cleaned = value.replaceAll(/[^A-Za-z0-9_.-]+/g, "-").replaceAll(/^-+|-+$/g, ""); + return cleaned.length > 0 ? cleaned : sha256(value).slice(0, 16); +} + +function canonicalJson(value: unknown): string { + if (value === null || typeof value === "string" || typeof value === "boolean") { + return JSON.stringify(value); + } + if (typeof value === "number") { + return JSON.stringify(Number.isFinite(value) ? value : null); + } + if (Array.isArray(value)) { + return `[${value.map((item) => canonicalJson(item)).join(",")}]`; + } + if (isRecord(value)) { + return `{${Object.keys(value).sort().map((key) => `${JSON.stringify(key)}:${canonicalJson(value[key])}`).join(",")}}`; + } + return JSON.stringify(null); +} From 0377b69c197f49fd93cd8fe72d74195c7d7bc6d8 Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Fri, 22 May 2026 21:40:49 -0400 Subject: [PATCH 05/52] Add spy SQLite persistence --- PLAN.md | 22 +- package.json | 6 +- src/spy/bedrock.test.ts | 22 +- src/spy/bedrock.ts | 117 +++++-- src/spy/migrations.ts | 37 ++ src/spy/schemas.test.ts | 17 +- src/spy/schemas.ts | 24 ++ src/spy/store.test.ts | 356 +++++++++++++++++++ src/spy/store.ts | 737 ++++++++++++++++++++++++++++++++++++++++ vitest.config.ts | 5 +- 10 files changed, 1285 insertions(+), 58 deletions(-) create mode 100644 src/spy/store.test.ts create mode 100644 src/spy/store.ts diff --git a/PLAN.md b/PLAN.md index d88a974..e160e1e 100644 --- a/PLAN.md +++ b/PLAN.md @@ -462,6 +462,26 @@ V1 excludes: extraction, usage extraction, stream events, raw payload gating, and hash stability. - Verified `bun run typecheck`, `bun run lint`, and `bun run test`. +- Implemented SQLite persistence, retention, and clear-data for spy capture: + - Added `src/spy/store.ts` with `openSpyStore`, spool batch ingestion, + request persistence, response completion, retention, clear-data, health + snapshots, and close lifecycle. + - Added request-only and response-only Bedrock normalization entrypoints while + preserving the paired fixture normalizer. + - Added typed HTTP event records and a schema v2 migration with + `normalized_block_fts` synchronization triggers. + - Persists pending and completed provider calls, HTTP metadata, normalized + blocks, usage records, stream events, optional raw payloads, dropped/error + counters, and service metadata. + - Defers unmatched response spool files, deletes malformed spool files after + recording counters/metadata, and deletes valid spool files only after + successful commit. + - Converted `src/spy` tests to Bun's native test runner so `bun:sqlite` runs + directly, with the remaining unit tests still running under Vitest. + - Added fixture-backed store coverage for ingestion, pending-to-complete + updates, idempotency, raw payload gating, malformed/drop/error events, + retention with FTS cleanup, and clear-data. + - Verified `bun run typecheck`, `bun run lint`, and `bun run test`. ### V1 @@ -473,7 +493,7 @@ Build the Bedrock/Pi browser spy: - [x] Add initial AWS event-stream decoder. - [x] Replace Python spy with minimal provider-gated spool shim. - [x] Implement TypeScript Bedrock adapter on top of the captured fixtures. -- [ ] Implement SQLite persistence, migrations, retention, and clear-data. +- [x] Implement SQLite persistence, migrations, retention, and clear-data. - [ ] Implement TS web service, API, SSE, and static asset serving. - [ ] Implement React desktop UI with virtualized timeline and call inspector. - [ ] Wire `rootcell provision`, systemd service config, and `rootcell spy` diff --git a/package.json b/package.json index cb11c51..2975370 100644 --- a/package.json +++ b/package.json @@ -5,12 +5,14 @@ "scripts": { "typecheck": "tsc --noEmit", "lint": "eslint \"src/**/*.{ts,js,mjs,cjs}\" eslint.config.ts vitest.config.ts", - "test": "vitest --project unit --run", + "test": "bun test src/spy --timeout 10000 && vitest --project unit --run", + "test:spy": "bun test src/spy --timeout 10000", + "test:unit:vitest": "vitest --project unit --run", "test:integration": "vitest --project integration --run", "test:integration:lima-smoke": "vitest --project integration --run src/rootcell/integration/providers/macos-lima-user-v2/cli-smoke.integration.test.ts", "test:integration:clean": "./test --clean", "test:integration:teardown": "./test --teardown", - "test:all": "vitest --run" + "test:all": "bun run test && bun run test:integration" }, "devDependencies": { "@eslint/js": "10.0.1", diff --git a/src/spy/bedrock.test.ts b/src/spy/bedrock.test.ts index c9e649e..68e7ce9 100644 --- a/src/spy/bedrock.test.ts +++ b/src/spy/bedrock.test.ts @@ -1,5 +1,5 @@ import { readFileSync } from "node:fs"; -import { describe, expect, test } from "vitest"; +import { describe, expect, test } from "bun:test"; import { normalizeBedrockCall, normalizeBedrockSpoolEvents, @@ -125,11 +125,10 @@ describe("Bedrock adapter", () => { expect(JSON.stringify(turnTwoRequest.map((block) => block.text))).toContain("RCSPY-ALPHA"); const toolResult = callById(calls, "call-fixture-flow-tool-result"); - expect(blocks(toolResult, "request").map((block) => block.kind)).toEqual(expect.arrayContaining([ - "tool-call", - "tool-result", - "cache-marker", - ])); + const toolResultKinds = blocks(toolResult, "request").map((block) => block.kind); + expect(toolResultKinds.includes("tool-call")).toBe(true); + expect(toolResultKinds.includes("tool-result")).toBe(true); + expect(toolResultKinds.includes("cache-marker")).toBe(true); expect(firstBlock(toolResult, "request", "tool-result").text).toContain("success"); }); @@ -163,12 +162,11 @@ describe("Bedrock adapter", () => { expect(toolCall.text).toContain("bash"); expect(JSON.stringify(toolCall.json)).toContain("printf"); expect(JSON.stringify(toolCall.json)).toContain("tool-fixture-ok"); - expect(toolUse.streamEvents.map((event) => event.event_type)).toEqual(expect.arrayContaining([ - "contentBlockStart", - "contentBlockDelta", - "messageStop", - "metadata", - ])); + const toolUseEventTypes = toolUse.streamEvents.map((event) => event.event_type); + expect(toolUseEventTypes.includes("contentBlockStart")).toBe(true); + expect(toolUseEventTypes.includes("contentBlockDelta")).toBe(true); + expect(toolUseEventTypes.includes("messageStop")).toBe(true); + expect(toolUseEventTypes.includes("metadata")).toBe(true); }); test("preserves raw payloads only when requested and keeps hashes stable", () => { diff --git a/src/spy/bedrock.ts b/src/spy/bedrock.ts index f1e860f..1fa2237 100644 --- a/src/spy/bedrock.ts +++ b/src/spy/bedrock.ts @@ -23,6 +23,20 @@ export interface NormalizedProviderCall { readonly rawPayloads: readonly RawPayloadRecord[]; } +export interface NormalizedProviderRequest { + readonly call: ProviderCall; + readonly blocks: readonly NormalizedBlock[]; + readonly rawPayloads: readonly RawPayloadRecord[]; +} + +export interface NormalizedProviderResponse { + readonly call: ProviderCall; + readonly blocks: readonly NormalizedBlock[]; + readonly usage: readonly UsageRecord[]; + readonly streamEvents: readonly StreamEvent[]; + readonly rawPayloads: readonly RawPayloadRecord[]; +} + type BlockKind = NormalizedBlock["kind"]; type Direction = NormalizedBlock["direction"]; @@ -67,35 +81,78 @@ export function normalizeBedrockCall( throw new Error(`cannot normalize Bedrock call with mismatched request/response metadata for flow ${request.flow_id}`); } - const callId = stableId("call", request.flow_id); - const requestBody = parseRequestBody(request); - const requestBlocks = normalizeRequestBlocks(callId, requestBody); - const responseNormalization = normalizeResponse(callId, response); - const blocks = [...requestBlocks, ...responseNormalization.blocks]; - const status = response.status_code >= 400 ? "error" : "complete"; + const normalizedRequest = normalizeBedrockRequest(request, options); + const normalizedResponse = normalizeBedrockResponse(response, options); + return { + call: { + ...normalizedRequest.call, + status: normalizedResponse.call.status, + completed_at: normalizedResponse.call.completed_at, + status_code: normalizedResponse.call.status_code, + response_flow_id: normalizedResponse.call.response_flow_id, + response_content_hash: normalizedResponse.call.response_content_hash, + }, + blocks: [...normalizedRequest.blocks, ...normalizedResponse.blocks], + usage: normalizedResponse.usage, + streamEvents: normalizedResponse.streamEvents, + rawPayloads: [...normalizedRequest.rawPayloads, ...normalizedResponse.rawPayloads], + }; +} + +export function normalizeBedrockRequest( + request: SpoolRequestEvent, + options: BedrockAdapterOptions = {}, +): NormalizedProviderRequest { + const callId = bedrockCallIdForFlow(request.flow_id); + const requestBody = parseRequestBody(request); return { call: { id: callId, provider: "bedrock", operation: request.operation, model_id: request.model_id, - status, + status: "pending", started_at: request.ts, + request_flow_id: request.flow_id, + request_content_hash: hashUnknown(requestBody), + }, + blocks: normalizeRequestBlocks(callId, requestBody), + rawPayloads: options.storeRaw === true ? [rawPayload(callId, "request", request)] : [], + }; +} + +export function normalizeBedrockResponse( + response: SpoolResponseEvent, + options: BedrockAdapterOptions = {}, +): NormalizedProviderResponse { + const callId = bedrockCallIdForFlow(response.flow_id); + const responseNormalization = normalizeResponse(callId, response); + return { + call: { + id: callId, + provider: "bedrock", + operation: response.operation, + model_id: response.model_id, + status: response.status_code >= 400 ? "error" : "complete", + started_at: response.ts, completed_at: response.ts, status_code: response.status_code, - request_flow_id: request.flow_id, + request_flow_id: response.flow_id, response_flow_id: response.flow_id, - request_content_hash: hashUnknown(requestBody), response_content_hash: hashUnknown(responseNormalization.blocks.map((block) => block.content_hash)), }, - blocks, + blocks: responseNormalization.blocks, usage: responseNormalization.usage, streamEvents: responseNormalization.streamEvents, - rawPayloads: options.storeRaw === true ? rawPayloads(callId, request, response) : [], + rawPayloads: options.storeRaw === true ? [rawPayload(callId, "response", response)] : [], }; } +export function bedrockCallIdForFlow(flowId: string): string { + return stableId("call", flowId); +} + export function normalizeBedrockSpoolEvents( events: readonly SpoolEvent[], options: BedrockAdapterOptions = {}, @@ -686,27 +743,25 @@ function usageRecordFromMetadata(callId: string, index: number, payload: unknown }; } -function rawPayloads( +function rawPayload( callId: string, - request: SpoolRequestEvent, - response: SpoolResponseEvent, -): RawPayloadRecord[] { - return [ - { - id: stableId("raw", callId, "request"), - call_id: callId, - direction: "request", - ...(headerValue(request.headers, "content-type") === undefined ? {} : { content_type: headerValue(request.headers, "content-type") }), - ...bodyFields(request), - }, - { - id: stableId("raw", callId, "response"), - call_id: callId, - direction: "response", - ...(headerValue(response.headers, "content-type") === undefined ? {} : { content_type: headerValue(response.headers, "content-type") }), - ...bodyFields(response), - }, - ]; + direction: RawPayloadRecord["direction"], + event: { + readonly headers: readonly (readonly [string, string])[]; + readonly body_text?: string | undefined; + readonly body_b64?: string | undefined; + readonly body_sha256?: string | undefined; + readonly body_encoding?: "aws-eventstream" | undefined; + }, +): RawPayloadRecord { + const contentType = headerValue(event.headers, "content-type"); + return { + id: stableId("raw", callId, direction), + call_id: callId, + direction, + ...(contentType === undefined ? {} : { content_type: contentType }), + ...bodyFields(event), + }; } function bodyFields(event: { diff --git a/src/spy/migrations.ts b/src/spy/migrations.ts index a16f1ea..6dcc6b0 100644 --- a/src/spy/migrations.ts +++ b/src/spy/migrations.ts @@ -139,6 +139,43 @@ CREATE TABLE IF NOT EXISTS service_metadata ( value TEXT NOT NULL, updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP ); +`, + }, + { + version: 2, + name: "normalized block fts triggers", + sql: ` +CREATE TRIGGER IF NOT EXISTS normalized_block_fts_insert +AFTER INSERT ON normalized_block +WHEN NEW.text IS NOT NULL +BEGIN + INSERT INTO normalized_block_fts(block_id, text) + VALUES (NEW.id, NEW.text); +END; + +CREATE TRIGGER IF NOT EXISTS normalized_block_fts_update +AFTER UPDATE OF text ON normalized_block +BEGIN + DELETE FROM normalized_block_fts WHERE block_id = OLD.id; + INSERT INTO normalized_block_fts(block_id, text) + SELECT NEW.id, NEW.text WHERE NEW.text IS NOT NULL; +END; + +CREATE TRIGGER IF NOT EXISTS normalized_block_fts_delete +AFTER DELETE ON normalized_block +BEGIN + DELETE FROM normalized_block_fts WHERE block_id = OLD.id; +END; + +INSERT INTO normalized_block_fts(block_id, text) + SELECT id, text + FROM normalized_block + WHERE text IS NOT NULL + AND NOT EXISTS ( + SELECT 1 + FROM normalized_block_fts + WHERE normalized_block_fts.block_id = normalized_block.id + ); `, }, ]; diff --git a/src/spy/schemas.test.ts b/src/spy/schemas.test.ts index f3b5b3f..38641dc 100644 --- a/src/spy/schemas.test.ts +++ b/src/spy/schemas.test.ts @@ -1,5 +1,6 @@ import { readFileSync } from "node:fs"; -import { describe, expect, test } from "vitest"; +import { Database } from "bun:sqlite"; +import { describe, expect, test } from "bun:test"; import { decodeAwsEventStreamJson } from "./eventstream.ts"; import { applySpyMigrations, currentSpySchemaVersion } from "./migrations.ts"; import { @@ -10,7 +11,6 @@ import { } from "./schemas.ts"; const FIXTURE_PATH = new URL("./fixtures/bedrock-pi-us-sonnet-4-6.ndjson", import.meta.url); -const bunSqlite = await import("bun:sqlite").catch(() => null); function fixtureEvents(): SpoolEvent[] { return readFileSync(FIXTURE_PATH, "utf8") @@ -110,17 +110,12 @@ describe("spy fixture capture", () => { }); describe("spy sqlite migrations", () => { - const testWithBunSqlite = bunSqlite === null ? test.skip : test; - - testWithBunSqlite("creates the v1 schema and supports core provider call inserts", () => { - if (bunSqlite === null) { - throw new Error("bun:sqlite unavailable"); - } - const db = new bunSqlite.Database(":memory:"); + test("creates the current schema and supports core provider call inserts", () => { + const db = new Database(":memory:"); try { applySpyMigrations(db); - expect(currentSpySchemaVersion()).toBe(1); - expect(db.query("SELECT version FROM schema_migration").get()).toEqual({ version: 1 }); + expect(currentSpySchemaVersion()).toBe(2); + expect(db.query("SELECT MAX(version) AS version FROM schema_migration").get()).toEqual({ version: 2 }); db.query(` INSERT INTO provider_call ( diff --git a/src/spy/schemas.ts b/src/spy/schemas.ts index 96803b2..f67f6a9 100644 --- a/src/spy/schemas.ts +++ b/src/spy/schemas.ts @@ -78,6 +78,9 @@ export const SpoolEventSchema = z.discriminatedUnion("direction", [ export type SpoolEvent = Readonly>; export type SpoolRequestEvent = Readonly>; export type SpoolResponseEvent = Readonly>; +export type SpoolStreamChunkEvent = Readonly>; +export type SpoolErrorEvent = Readonly>; +export type SpoolDroppedEvent = Readonly>; export const ProviderCallStatusSchema = z.enum([ "pending", @@ -103,6 +106,27 @@ export const ProviderCallSchema = z.object({ export type ProviderCall = Readonly>; +export const HttpEventRecordSchema = z.object({ + id: z.string().min(1), + call_id: z.string().min(1), + direction: z.enum(["request", "response"]), + observed_at: z.number(), + host: z.string().min(1), + method: z.string().min(1), + path: z.string().min(1), + status_code: z.number().int().nonnegative().optional(), + reason: z.string().optional(), + headers: z.array(SpyHeaderPairSchema), + request_headers: z.array(SpyHeaderPairSchema).optional(), + content_type: z.string().optional(), + body_text: z.string().optional(), + body_b64: z.string().optional(), + body_sha256: z.string().optional(), + body_encoding: z.enum(["aws-eventstream"]).optional(), +}).strict(); + +export type HttpEventRecord = Readonly>; + export const NormalizedBlockKindSchema = z.enum([ "provider-envelope", "harness-system-context", diff --git a/src/spy/store.test.ts b/src/spy/store.test.ts new file mode 100644 index 0000000..ec8922e --- /dev/null +++ b/src/spy/store.test.ts @@ -0,0 +1,356 @@ +import { mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { Database } from "bun:sqlite"; +import { afterEach, describe, expect, test } from "bun:test"; +import { bedrockCallIdForFlow } from "./bedrock.ts"; +import { currentSpySchemaVersion } from "./migrations.ts"; +import { + SpoolEventSchema, + SpoolRequestEventSchema, + SpoolResponseEventSchema, + type SpoolEvent, + type SpoolRequestEvent, + type SpoolResponseEvent, +} from "./schemas.ts"; +import { openSpyStore, type SpyStore } from "./store.ts"; + +const FIXTURE_PATH = new URL("./fixtures/bedrock-pi-us-sonnet-4-6.ndjson", import.meta.url); + +interface CountRow { + readonly count: number; +} + +interface StatusRow { + readonly status: string; +} + +interface ValueRow { + readonly value: string; +} + +interface TestStore { + readonly root: string; + readonly dbPath: string; + readonly spoolDir: string; + readonly store: SpyStore; +} + +const tempRoots: string[] = []; + +afterEach(() => { + for (const root of tempRoots.splice(0)) { + rmSync(root, { recursive: true, force: true }); + } +}); + +function fixtureEvents(): SpoolEvent[] { + return readFileSync(FIXTURE_PATH, "utf8") + .trim() + .split("\n") + .filter(Boolean) + .map((line) => SpoolEventSchema.parse(JSON.parse(line) as unknown)); +} + +function fixturePair(flowId = "fixture-flow-simple"): readonly [SpoolRequestEvent, SpoolResponseEvent] { + const events = fixtureEvents(); + const request = events.find((event): event is SpoolRequestEvent => event.direction === "request" && event.flow_id === flowId); + const response = events.find((event): event is SpoolResponseEvent => event.direction === "response" && event.flow_id === flowId); + if (request === undefined || response === undefined) { + throw new Error(`missing fixture pair ${flowId}`); + } + return [request, response]; +} + +function createTestStore(options: { + readonly storeRaw?: boolean | undefined; + readonly retentionDays?: number | undefined; + readonly maxBytes?: number | undefined; + readonly now?: (() => number) | undefined; +} = {}): TestStore { + const root = mkdtempSync(join(tmpdir(), "rootcell-spy-store-")); + tempRoots.push(root); + const dbPath = join(root, "spy.sqlite"); + const spoolDir = join(root, "spool"); + const store = openSpyStore({ dbPath, spoolDir, ...options }); + return { root, dbPath, spoolDir, store }; +} + +function writeSpoolEvents(spoolDir: string, events: readonly SpoolEvent[]): void { + events.forEach((event, index) => { + writeFileSync( + join(spoolDir, `${String(index).padStart(3, "0")}-${event.direction}-${flowIdForName(event)}.json`), + `${JSON.stringify(event)}\n`, + ); + }); +} + +function flowIdForName(event: SpoolEvent): string { + return "flow_id" in event && event.flow_id !== undefined ? event.flow_id : "no-flow"; +} + +function countRows(dbPath: string, table: string, where?: string): number { + const db = new Database(dbPath, { readonly: true }); + try { + const sql = where === undefined ? `SELECT COUNT(*) AS count FROM ${table}` : `SELECT COUNT(*) AS count FROM ${table} WHERE ${where}`; + const row = db.query(sql).get() as CountRow | null; + return row?.count ?? 0; + } finally { + db.close(); + } +} + +function statusForCall(dbPath: string, callId: string): string { + const db = new Database(dbPath, { readonly: true }); + try { + const row = db.query("SELECT status FROM provider_call WHERE id = ?").get(callId) as StatusRow | null; + if (row === null) { + throw new Error(`missing call ${callId}`); + } + return row.status; + } finally { + db.close(); + } +} + +function metadataValue(dbPath: string, key: string): string { + const db = new Database(dbPath, { readonly: true }); + try { + const row = db.query("SELECT value FROM service_metadata WHERE key = ?").get(key) as ValueRow | null; + if (row === null) { + throw new Error(`missing metadata ${key}`); + } + return row.value; + } finally { + db.close(); + } +} + +function retagRequest(event: SpoolRequestEvent, flowId: string, ts: number): SpoolRequestEvent { + return SpoolRequestEventSchema.parse({ ...event, flow_id: flowId, ts }); +} + +function retagResponse(event: SpoolResponseEvent, flowId: string, ts: number): SpoolResponseEvent { + return SpoolResponseEventSchema.parse({ ...event, flow_id: flowId, ts }); +} + +function firstFixtureEvent(): SpoolEvent { + const event = fixtureEvents()[0]; + if (event === undefined) { + throw new Error("missing first fixture event"); + } + return event; +} + +describe("spy SQLite store", () => { + test("ingests Bedrock fixture spool files into normalized SQLite records", () => { + const { dbPath, spoolDir, store } = createTestStore(); + try { + writeSpoolEvents(spoolDir, fixtureEvents()); + + expect(store.ingestSpoolBatch()).toEqual({ + attempted: 10, + ingested: 10, + deleted: 10, + deferred: 0, + malformed: 0, + errors: 0, + }); + + expect(countRows(dbPath, "provider_call")).toBe(5); + expect(countRows(dbPath, "http_event")).toBe(10); + expect(countRows(dbPath, "normalized_block")).toBeGreaterThan(20); + expect(countRows(dbPath, "usage_record")).toBe(5); + expect(countRows(dbPath, "stream_event")).toBeGreaterThan(20); + expect(countRows(dbPath, "raw_payload")).toBe(0); + expect(countRows(dbPath, "http_event", "body_text IS NOT NULL OR body_b64 IS NOT NULL")).toBe(0); + expect(countRows(dbPath, "schema_migration")).toBe(currentSpySchemaVersion()); + expect(readdirSync(spoolDir)).toEqual([]); + + const health = store.getHealthSnapshot(); + expect(health.counters.spool_request_events).toBe(5); + expect(health.counters.spool_response_events).toBe(5); + expect(health.providerCallCount).toBe(5); + expect(health.pendingCallCount).toBe(0); + } finally { + store.close(); + } + }); + + test("persists raw payloads only when raw storage is enabled", () => { + const { dbPath, store } = createTestStore({ storeRaw: true }); + try { + const [request, response] = fixturePair(); + store.persistRequest(request); + expect(store.persistResponse(response)).toBe(true); + + expect(countRows(dbPath, "provider_call")).toBe(1); + expect(countRows(dbPath, "raw_payload")).toBe(2); + expect(countRows(dbPath, "raw_payload", "direction = 'request' AND body_text LIKE '%Fixture capture simple prompt%'")).toBe(1); + expect(countRows(dbPath, "raw_payload", "direction = 'response' AND body_encoding = 'aws-eventstream'")).toBe(1); + } finally { + store.close(); + } + }); + + test("moves calls from pending to complete and remains idempotent", () => { + const { dbPath, store } = createTestStore(); + try { + const [request, response] = fixturePair(); + const callId = bedrockCallIdForFlow(request.flow_id); + + store.persistRequest(request); + expect(statusForCall(dbPath, callId)).toBe("pending"); + expect(countRows(dbPath, "normalized_block", "call_id = 'call-fixture-flow-simple' AND direction = 'request'")).toBeGreaterThan(0); + expect(countRows(dbPath, "normalized_block", "call_id = 'call-fixture-flow-simple' AND direction = 'response'")).toBe(0); + + expect(store.persistResponse(response)).toBe(true); + expect(statusForCall(dbPath, callId)).toBe("complete"); + const blockCount = countRows(dbPath, "normalized_block"); + const streamCount = countRows(dbPath, "stream_event"); + + store.persistRequest(request); + expect(store.persistResponse(response)).toBe(true); + expect(statusForCall(dbPath, callId)).toBe("complete"); + expect(countRows(dbPath, "provider_call")).toBe(1); + expect(countRows(dbPath, "http_event")).toBe(2); + expect(countRows(dbPath, "normalized_block")).toBe(blockCount); + expect(countRows(dbPath, "stream_event")).toBe(streamCount); + } finally { + store.close(); + } + }); + + test("defers unmatched responses and records malformed, dropped, and error events", () => { + const { dbPath, spoolDir, store } = createTestStore(); + try { + const [request, response] = fixturePair(); + writeSpoolEvents(spoolDir, [response]); + + expect(store.ingestSpoolBatch()).toEqual({ + attempted: 1, + ingested: 0, + deleted: 0, + deferred: 1, + malformed: 0, + errors: 0, + }); + expect(readdirSync(spoolDir)).toHaveLength(1); + + store.persistRequest(request); + expect(store.ingestSpoolBatch()).toEqual({ + attempted: 1, + ingested: 1, + deleted: 1, + deferred: 0, + malformed: 0, + errors: 0, + }); + expect(statusForCall(dbPath, bedrockCallIdForFlow(request.flow_id))).toBe("complete"); + + writeFileSync(join(spoolDir, "bad-schema.json"), "{\"direction\":\"request\"}\n"); + writeFileSync(join(spoolDir, "dropped.json"), `${JSON.stringify({ + version: 1, + ts: 2000, + direction: "dropped", + provider: "bedrock", + reason: "spool_full", + dropped_count: 3, + })}\n`); + writeFileSync(join(spoolDir, "error.json"), `${JSON.stringify({ + version: 1, + ts: 2001, + direction: "error", + flow_id: request.flow_id, + provider: "bedrock", + error: "upstream failed", + })}\n`); + + expect(store.ingestSpoolBatch()).toEqual({ + attempted: 3, + ingested: 2, + deleted: 3, + deferred: 0, + malformed: 1, + errors: 0, + }); + const health = store.getHealthSnapshot(); + expect(health.counters.spool_malformed_events).toBe(1); + expect(health.counters.spool_dropped_events).toBe(1); + expect(health.counters.captures_dropped).toBe(3); + expect(health.counters.spool_error_events).toBe(1); + expect(statusForCall(dbPath, bedrockCallIdForFlow(request.flow_id))).toBe("error"); + expect(readdirSync(spoolDir)).toEqual([]); + } finally { + store.close(); + } + }); + + test("retention deletes old and oversized call data with cascaded FTS cleanup", () => { + const [baseRequest, baseResponse] = fixturePair(); + const ageStore = createTestStore({ retentionDays: 1, now: () => 200_000 }); + try { + ageStore.store.persistRequest(retagRequest(baseRequest, "old-flow", 100)); + ageStore.store.persistResponse(retagResponse(baseResponse, "old-flow", 101)); + ageStore.store.persistRequest(retagRequest(baseRequest, "new-flow", 199_990)); + ageStore.store.persistResponse(retagResponse(baseResponse, "new-flow", 199_991)); + + expect(ageStore.store.runRetention()).toEqual({ + deletedByAge: 1, + deletedBySize: 0, + vacuumed: false, + }); + expect(countRows(ageStore.dbPath, "provider_call")).toBe(1); + expect(countRows(ageStore.dbPath, "provider_call", "id = 'call-new-flow'")).toBe(1); + expect(countRows(ageStore.dbPath, "normalized_block", "call_id = 'call-old-flow'")).toBe(0); + expect(countRows(ageStore.dbPath, "normalized_block_fts")).toBe( + countRows(ageStore.dbPath, "normalized_block", "text IS NOT NULL"), + ); + } finally { + ageStore.store.close(); + } + + const sizeStore = createTestStore({ retentionDays: 365, maxBytes: 1, now: () => 200_000 }); + try { + sizeStore.store.persistRequest(retagRequest(baseRequest, "size-one", 10_000)); + sizeStore.store.persistResponse(retagResponse(baseResponse, "size-one", 10_001)); + sizeStore.store.persistRequest(retagRequest(baseRequest, "size-two", 20_000)); + sizeStore.store.persistResponse(retagResponse(baseResponse, "size-two", 20_001)); + + const result = sizeStore.store.runRetention(); + expect(result.deletedByAge).toBe(0); + expect(result.deletedBySize).toBe(2); + expect(countRows(sizeStore.dbPath, "provider_call")).toBe(0); + expect(countRows(sizeStore.dbPath, "normalized_block_fts")).toBe(0); + } finally { + sizeStore.store.close(); + } + }); + + test("clearData removes captured rows and pending spool while preserving migrations", () => { + const { dbPath, spoolDir, store } = createTestStore({ now: () => 1234 }); + try { + writeSpoolEvents(spoolDir, fixtureEvents()); + expect(store.ingestSpoolBatch().ingested).toBe(10); + writeFileSync(join(spoolDir, "pending.json"), `${JSON.stringify(firstFixtureEvent())}\n`); + + expect(store.clearData()).toEqual({ + deletedSpoolFiles: 1, + clearGeneration: 1, + clearBaselineTs: 1234, + }); + + expect(countRows(dbPath, "provider_call")).toBe(0); + expect(countRows(dbPath, "normalized_block")).toBe(0); + expect(countRows(dbPath, "normalized_block_fts")).toBe(0); + expect(countRows(dbPath, "stream_event")).toBe(0); + expect(countRows(dbPath, "health_counter")).toBe(0); + expect(countRows(dbPath, "schema_migration")).toBe(currentSpySchemaVersion()); + expect(metadataValue(dbPath, "clear_generation")).toBe("1"); + expect(metadataValue(dbPath, "clear_baseline_ts")).toBe("1234"); + expect(readdirSync(spoolDir)).toEqual([]); + } finally { + store.close(); + } + }); +}); diff --git a/src/spy/store.ts b/src/spy/store.ts new file mode 100644 index 0000000..4a1679f --- /dev/null +++ b/src/spy/store.ts @@ -0,0 +1,737 @@ +import { existsSync, mkdirSync, readdirSync, readFileSync, rmSync, statSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { Database } from "bun:sqlite"; +import { + bedrockCallIdForFlow, + normalizeBedrockRequest, + normalizeBedrockResponse, +} from "./bedrock.ts"; +import { applySpyMigrations, currentSpySchemaVersion } from "./migrations.ts"; +import { + HttpEventRecordSchema, + SpoolEventSchema, + type HttpEventRecord, + type NormalizedBlock, + type ProviderCall, + type RawPayloadRecord, + type SpoolDroppedEvent, + type SpoolErrorEvent, + type SpoolEvent, + type SpoolRequestEvent, + type SpoolResponseEvent, + type SpoolStreamChunkEvent, + type StreamEvent, + type UsageRecord, +} from "./schemas.ts"; + +const DEFAULT_RETENTION_DAYS = 7; +const DEFAULT_MAX_BYTES = 6 * 1024 * 1024 * 1024; + +export interface SpyStoreOptions { + readonly dbPath: string; + readonly spoolDir: string; + readonly retentionDays?: number | undefined; + readonly maxBytes?: number | undefined; + readonly storeRaw?: boolean | undefined; + readonly now?: (() => number) | undefined; +} + +export interface IngestSpoolBatchOptions { + readonly limit?: number | undefined; +} + +export interface IngestSpoolBatchResult { + readonly attempted: number; + readonly ingested: number; + readonly deleted: number; + readonly deferred: number; + readonly malformed: number; + readonly errors: number; +} + +export interface RetentionResult { + readonly deletedByAge: number; + readonly deletedBySize: number; + readonly vacuumed: boolean; +} + +export interface ClearDataResult { + readonly deletedSpoolFiles: number; + readonly clearGeneration: number; + readonly clearBaselineTs: number; +} + +export interface SpyHealthSnapshot { + readonly schemaVersion: number; + readonly dbSizeBytes: number; + readonly dbUsedBytes: number; + readonly spoolSizeBytes: number; + readonly providerCallCount: number; + readonly pendingCallCount: number; + readonly counters: Readonly>; + readonly metadata: Readonly>; +} + +export interface SpyStore { + ingestSpoolBatch(options?: IngestSpoolBatchOptions): IngestSpoolBatchResult; + persistRequest(event: SpoolRequestEvent): void; + persistResponse(event: SpoolResponseEvent): boolean; + runRetention(): RetentionResult; + clearData(): ClearDataResult; + getHealthSnapshot(): SpyHealthSnapshot; + close(): void; +} + +interface CounterRow { + readonly name: string; + readonly value: number; +} + +interface MetadataRow { + readonly key: string; + readonly value: string; +} + +interface CountRow { + readonly count: number; +} + +interface IdRow { + readonly id: string; +} + +type PragmaRow = Readonly>; + +class BunSqliteSpyStore implements SpyStore { + private readonly db: Database; + private readonly retentionDays: number; + private readonly maxBytes: number; + private readonly storeRaw: boolean; + private readonly now: () => number; + private locked = false; + + constructor(private readonly options: SpyStoreOptions) { + this.retentionDays = positiveNumber(options.retentionDays, DEFAULT_RETENTION_DAYS); + this.maxBytes = positiveNumber(options.maxBytes, DEFAULT_MAX_BYTES); + this.storeRaw = options.storeRaw === true; + this.now = options.now ?? (() => Date.now() / 1000); + + if (options.dbPath !== ":memory:") { + mkdirSync(dirname(options.dbPath), { recursive: true }); + } + mkdirSync(options.spoolDir, { recursive: true }); + + this.db = new Database(options.dbPath, { create: true }); + this.db.run("PRAGMA foreign_keys = ON"); + applySpyMigrations(this.db); + this.setMetadata("schema_version", String(currentSpySchemaVersion())); + } + + ingestSpoolBatch(options: IngestSpoolBatchOptions = {}): IngestSpoolBatchResult { + return this.withWriteLock(() => { + const limit = Math.max(0, Math.trunc(options.limit ?? Number.MAX_SAFE_INTEGER)); + const files = this.spoolFiles().slice(0, limit); + const result = { + attempted: 0, + ingested: 0, + deleted: 0, + deferred: 0, + malformed: 0, + errors: 0, + }; + + for (const path of files) { + result.attempted += 1; + const fileResult = this.ingestSpoolFileUnlocked(path); + if (fileResult === "ingested") { + result.ingested += 1; + result.deleted += 1; + } else if (fileResult === "malformed") { + result.malformed += 1; + result.deleted += 1; + } else if (fileResult === "deferred") { + result.deferred += 1; + } else { + result.errors += 1; + } + } + + if (result.ingested > 0) { + this.setMetadata("last_ingest_at", String(this.now())); + } + return result; + }); + } + + persistRequest(event: SpoolRequestEvent): void { + this.withWriteLock(() => { + this.persistRequestUnlocked(event); + }); + } + + persistResponse(event: SpoolResponseEvent): boolean { + return this.withWriteLock(() => this.persistResponseUnlocked(event)); + } + + runRetention(): RetentionResult { + return this.withWriteLock(() => { + let deletedByAge = 0; + let deletedBySize = 0; + this.db.transaction(() => { + const cutoff = this.now() - this.retentionDays * 24 * 60 * 60; + deletedByAge = this.deleteCallsBefore(cutoff); + + while (this.databaseUsedBytes() > this.maxBytes) { + const deleted = this.deleteOldestCall(); + if (!deleted) { + break; + } + deletedBySize += 1; + } + + if (deletedByAge > 0 || deletedBySize > 0) { + this.setMetadata("last_retention_at", String(this.now())); + this.incrementCounter("retention_deleted_calls", deletedByAge + deletedBySize); + } + })(); + + let vacuumed = false; + if ((deletedByAge > 0 || deletedBySize > 0) && this.databaseSizeBytes() > this.maxBytes) { + this.db.run("VACUUM"); + vacuumed = true; + } + return { deletedByAge, deletedBySize, vacuumed }; + }); + } + + clearData(): ClearDataResult { + return this.withWriteLock(() => { + const clearBaselineTs = this.now(); + const clearGeneration = this.clearGeneration() + 1; + this.db.transaction(() => { + this.db.run("DELETE FROM provider_call"); + this.db.run("DELETE FROM health_counter"); + this.setMetadata("clear_generation", String(clearGeneration)); + this.setMetadata("clear_baseline_ts", String(clearBaselineTs)); + })(); + + const deletedSpoolFiles = this.clearSpoolFiles(); + return { deletedSpoolFiles, clearGeneration, clearBaselineTs }; + }); + } + + getHealthSnapshot(): SpyHealthSnapshot { + return { + schemaVersion: currentSpySchemaVersion(), + dbSizeBytes: this.databaseSizeBytes(), + dbUsedBytes: this.databaseUsedBytes(), + spoolSizeBytes: this.spoolSizeBytes(), + providerCallCount: this.countRows("provider_call"), + pendingCallCount: this.countRows("provider_call", "status = 'pending'"), + counters: this.healthCounters(), + metadata: this.serviceMetadata(), + }; + } + + close(): void { + this.db.close(); + } + + private ingestSpoolFileUnlocked(path: string): "ingested" | "deferred" | "malformed" | "error" { + let event: SpoolEvent; + try { + const parsed: unknown = JSON.parse(readFileSync(path, "utf8")) as unknown; + event = SpoolEventSchema.parse(parsed); + } catch (error) { + this.recordMalformedSpoolFile(path, error); + unlinkIfExists(path); + return "malformed"; + } + + try { + if (event.direction === "request") { + this.persistRequestUnlocked(event); + } else if (event.direction === "response") { + if (!this.persistResponseUnlocked(event)) { + return "deferred"; + } + } else if (event.direction === "dropped") { + this.persistDroppedEventUnlocked(event); + } else if (event.direction === "error") { + this.persistErrorEventUnlocked(event); + } else { + this.persistStreamChunkEventUnlocked(event); + } + } catch (error) { + this.recordIngestError(path, error); + return "error"; + } + + unlinkIfExists(path); + return "ingested"; + } + + private persistRequestUnlocked(event: SpoolRequestEvent): void { + const normalized = normalizeBedrockRequest(event, { storeRaw: this.storeRaw }); + const httpEvent = httpEventFromRequest(event, normalized.call.id); + this.db.transaction(() => { + this.upsertPendingCall(normalized.call); + this.replaceHttpEvent(httpEvent); + this.replaceBlocks(normalized.call.id, "request", normalized.blocks); + this.replaceRawPayloads(normalized.call.id, "request", normalized.rawPayloads); + this.incrementCounter("spool_request_events", 1); + this.setMetadata("last_request_at", String(event.ts)); + })(); + } + + private persistResponseUnlocked(event: SpoolResponseEvent): boolean { + const callId = bedrockCallIdForFlow(event.flow_id); + if (!this.callExists(callId)) { + return false; + } + + const normalized = normalizeBedrockResponse(event, { storeRaw: this.storeRaw }); + const httpEvent = httpEventFromResponse(event, normalized.call.id); + this.db.transaction(() => { + this.updateResponseCall(normalized.call); + this.replaceHttpEvent(httpEvent); + this.replaceBlocks(normalized.call.id, "response", normalized.blocks); + this.replaceUsageRecords(normalized.call.id, normalized.usage); + this.replaceStreamEvents(normalized.call.id, normalized.streamEvents); + this.replaceRawPayloads(normalized.call.id, "response", normalized.rawPayloads); + this.incrementCounter("spool_response_events", 1); + this.setMetadata("last_response_at", String(event.ts)); + })(); + return true; + } + + private persistDroppedEventUnlocked(event: SpoolDroppedEvent): void { + this.db.transaction(() => { + this.incrementCounter("spool_dropped_events", 1); + this.incrementCounter("captures_dropped", event.dropped_count); + this.setMetadata("last_dropped_at", String(event.ts)); + this.setMetadata("last_dropped_event", JSON.stringify(event)); + })(); + } + + private persistErrorEventUnlocked(event: SpoolErrorEvent): void { + this.db.transaction(() => { + this.incrementCounter("spool_error_events", 1); + this.setMetadata("last_spool_error_at", String(event.ts)); + this.setMetadata("last_spool_error", JSON.stringify(event)); + if (event.flow_id !== undefined) { + const callId = bedrockCallIdForFlow(event.flow_id); + if (this.callExists(callId)) { + this.db.query(` +UPDATE provider_call +SET status = 'error', + completed_at = ?, + response_flow_id = ? +WHERE id = ? +`).run(event.ts, event.flow_id, callId); + } + } + })(); + } + + private persistStreamChunkEventUnlocked(event: SpoolStreamChunkEvent): void { + this.db.transaction(() => { + this.incrementCounter("spool_stream_chunk_events", 1); + this.setMetadata("last_stream_chunk_at", String(event.ts)); + this.setMetadata("last_stream_chunk_event", JSON.stringify({ + flow_id: event.flow_id, + chunk_index: event.chunk_index, + body_sha256: event.body_sha256, + })); + })(); + } + + private upsertPendingCall(call: ProviderCall): void { + this.db.query(` +INSERT INTO provider_call ( + id, provider, operation, model_id, status, started_at, + request_flow_id, request_content_hash +) VALUES (?, ?, ?, ?, ?, ?, ?, ?) +ON CONFLICT(id) DO UPDATE SET + provider = excluded.provider, + operation = excluded.operation, + model_id = excluded.model_id, + started_at = excluded.started_at, + request_flow_id = excluded.request_flow_id, + request_content_hash = excluded.request_content_hash, + status = CASE + WHEN provider_call.status = 'pending' THEN excluded.status + ELSE provider_call.status + END +`).run( + call.id, + call.provider, + call.operation, + call.model_id, + call.status, + call.started_at, + call.request_flow_id, + call.request_content_hash ?? null, + ); + } + + private updateResponseCall(call: ProviderCall): void { + this.db.query(` +UPDATE provider_call +SET status = ?, + completed_at = ?, + status_code = ?, + response_flow_id = ?, + response_content_hash = ? +WHERE id = ? +`).run( + call.status, + call.completed_at ?? null, + call.status_code ?? null, + call.response_flow_id ?? null, + call.response_content_hash ?? null, + call.id, + ); + } + + private replaceHttpEvent(event: HttpEventRecord): void { + HttpEventRecordSchema.parse(event); + this.db.query("DELETE FROM http_event WHERE id = ?").run(event.id); + this.db.query(` +INSERT INTO http_event ( + id, call_id, direction, observed_at, host, method, path, status_code, reason, + headers_json, request_headers_json, body_text, body_b64, body_sha256, + body_encoding, content_type +) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) +`).run( + event.id, + event.call_id, + event.direction, + event.observed_at, + event.host, + event.method, + event.path, + event.status_code ?? null, + event.reason ?? null, + JSON.stringify(event.headers), + event.request_headers === undefined ? null : JSON.stringify(event.request_headers), + event.body_text ?? null, + event.body_b64 ?? null, + event.body_sha256 ?? null, + event.body_encoding ?? null, + event.content_type ?? null, + ); + } + + private replaceBlocks( + callId: string, + direction: NormalizedBlock["direction"], + blocks: readonly NormalizedBlock[], + ): void { + this.db.query("DELETE FROM normalized_block WHERE call_id = ? AND direction = ?").run(callId, direction); + const insert = this.db.query(` +INSERT INTO normalized_block ( + id, call_id, direction, ordinal, role, kind, source, provider_path, text, json, + char_size, byte_size, content_hash, cache_marker +) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) +`); + for (const block of blocks) { + insert.run( + block.id, + block.call_id, + block.direction, + block.ordinal, + block.role ?? null, + block.kind, + block.source, + block.provider_path ?? null, + block.text ?? null, + block.json === undefined ? null : JSON.stringify(block.json), + block.char_size, + block.byte_size, + block.content_hash, + block.cache_marker ? 1 : 0, + ); + } + } + + private replaceUsageRecords(callId: string, usage: readonly UsageRecord[]): void { + this.db.query("DELETE FROM usage_record WHERE call_id = ?").run(callId); + const insert = this.db.query(` +INSERT INTO usage_record ( + id, call_id, source, input_tokens, output_tokens, cache_read_tokens, + cache_write_tokens, total_tokens, raw_json +) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) +`); + for (const record of usage) { + insert.run( + record.id, + record.call_id, + record.source, + record.input_tokens ?? null, + record.output_tokens ?? null, + record.cache_read_tokens ?? null, + record.cache_write_tokens ?? null, + record.total_tokens ?? null, + record.raw === undefined ? null : JSON.stringify(record.raw), + ); + } + } + + private replaceStreamEvents(callId: string, events: readonly StreamEvent[]): void { + this.db.query("DELETE FROM stream_event WHERE call_id = ?").run(callId); + const insert = this.db.query(` +INSERT INTO stream_event ( + id, call_id, ordinal, event_type, headers_json, payload_json, + payload_text, payload_sha256, observed_at +) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) +`); + for (const event of events) { + insert.run( + event.id, + event.call_id, + event.ordinal, + event.event_type, + JSON.stringify(event.headers), + event.payload === undefined ? null : JSON.stringify(event.payload), + event.payload_text ?? null, + event.payload_sha256 ?? null, + event.observed_at ?? null, + ); + } + } + + private replaceRawPayloads( + callId: string, + direction: RawPayloadRecord["direction"], + payloads: readonly RawPayloadRecord[], + ): void { + this.db.query("DELETE FROM raw_payload WHERE call_id = ? AND direction = ?").run(callId, direction); + const insert = this.db.query(` +INSERT INTO raw_payload ( + id, call_id, direction, content_type, body_text, body_b64, + body_sha256, body_encoding +) VALUES (?, ?, ?, ?, ?, ?, ?, ?) +`); + for (const payload of payloads) { + insert.run( + payload.id, + payload.call_id, + payload.direction, + payload.content_type ?? null, + payload.body_text ?? null, + payload.body_b64 ?? null, + payload.body_sha256 ?? null, + payload.body_encoding ?? null, + ); + } + } + + private recordMalformedSpoolFile(path: string, error: unknown): void { + this.db.transaction(() => { + this.incrementCounter("spool_malformed_events", 1); + this.setMetadata("last_malformed_spool_file", path); + this.setMetadata("last_malformed_error", errorMessage(error)); + })(); + } + + private recordIngestError(path: string, error: unknown): void { + this.db.transaction(() => { + this.incrementCounter("spool_ingest_errors", 1); + this.setMetadata("last_ingest_error_file", path); + this.setMetadata("last_ingest_error", errorMessage(error)); + })(); + } + + private incrementCounter(name: string, amount: number): void { + this.db.query(` +INSERT INTO health_counter (name, value) +VALUES (?, ?) +ON CONFLICT(name) DO UPDATE SET + value = health_counter.value + excluded.value, + updated_at = CURRENT_TIMESTAMP +`).run(name, amount); + } + + private setMetadata(key: string, value: string): void { + this.db.query(` +INSERT INTO service_metadata (key, value) +VALUES (?, ?) +ON CONFLICT(key) DO UPDATE SET + value = excluded.value, + updated_at = CURRENT_TIMESTAMP +`).run(key, value); + } + + private callExists(callId: string): boolean { + const row = this.db.query("SELECT id FROM provider_call WHERE id = ?").get(callId) as IdRow | null; + return row !== null; + } + + private deleteCallsBefore(cutoff: number): number { + const row = this.db.query("SELECT COUNT(*) AS count FROM provider_call WHERE started_at < ?").get(cutoff) as CountRow | null; + const count = row?.count ?? 0; + this.db.query("DELETE FROM provider_call WHERE started_at < ?").run(cutoff); + return count; + } + + private deleteOldestCall(): boolean { + const row = this.db.query("SELECT id FROM provider_call ORDER BY started_at ASC, id ASC LIMIT 1").get() as IdRow | null; + if (row === null) { + return false; + } + this.db.query("DELETE FROM provider_call WHERE id = ?").run(row.id); + return true; + } + + private countRows(table: string, where?: string): number { + const sql = where === undefined ? `SELECT COUNT(*) AS count FROM ${table}` : `SELECT COUNT(*) AS count FROM ${table} WHERE ${where}`; + const row = this.db.query(sql).get() as CountRow | null; + return row?.count ?? 0; + } + + private clearGeneration(): number { + const row = this.db.query("SELECT value FROM service_metadata WHERE key = 'clear_generation'").get() as { readonly value: string } | null; + const parsed = row === null ? 0 : Number.parseInt(row.value, 10); + return Number.isFinite(parsed) ? parsed : 0; + } + + private databaseSizeBytes(): number { + return this.pragmaNumber("page_count") * this.pragmaNumber("page_size"); + } + + private databaseUsedBytes(): number { + const usedPages = Math.max(0, this.pragmaNumber("page_count") - this.pragmaNumber("freelist_count")); + return usedPages * this.pragmaNumber("page_size"); + } + + private pragmaNumber(name: "page_count" | "page_size" | "freelist_count"): number { + const row = this.db.query(`PRAGMA ${name}`).get() as PragmaRow | null; + return row?.[name] ?? 0; + } + + private healthCounters(): Readonly> { + const rows = this.db.query("SELECT name, value FROM health_counter ORDER BY name").all() as CounterRow[]; + const counters: Record = {}; + for (const row of rows) { + counters[row.name] = row.value; + } + return counters; + } + + private serviceMetadata(): Readonly> { + const rows = this.db.query("SELECT key, value FROM service_metadata ORDER BY key").all() as MetadataRow[]; + const metadata: Record = {}; + for (const row of rows) { + metadata[row.key] = row.value; + } + return metadata; + } + + private spoolFiles(): string[] { + if (!existsSync(this.options.spoolDir)) { + return []; + } + return readdirSync(this.options.spoolDir, { withFileTypes: true }) + .filter((entry) => entry.isFile() && !entry.name.startsWith(".") && entry.name.endsWith(".json")) + .map((entry) => join(this.options.spoolDir, entry.name)) + .sort(); + } + + private clearSpoolFiles(): number { + if (!existsSync(this.options.spoolDir)) { + return 0; + } + let deleted = 0; + for (const entry of readdirSync(this.options.spoolDir, { withFileTypes: true })) { + if (!entry.isFile() || entry.name.startsWith(".")) { + continue; + } + unlinkIfExists(join(this.options.spoolDir, entry.name)); + deleted += 1; + } + return deleted; + } + + private spoolSizeBytes(): number { + if (!existsSync(this.options.spoolDir)) { + return 0; + } + let total = 0; + for (const entry of readdirSync(this.options.spoolDir, { withFileTypes: true })) { + if (!entry.isFile()) { + continue; + } + total += statSync(join(this.options.spoolDir, entry.name)).size; + } + return total; + } + + private withWriteLock(action: () => T): T { + if (this.locked) { + throw new Error("spy store write already in progress"); + } + this.locked = true; + try { + return action(); + } finally { + this.locked = false; + } + } +} + +export function openSpyStore(options: SpyStoreOptions): SpyStore { + return new BunSqliteSpyStore(options); +} + +function httpEventFromRequest(event: SpoolRequestEvent, callId: string): HttpEventRecord { + return { + id: `http-${callId}-request`, + call_id: callId, + direction: "request", + observed_at: event.ts, + host: event.host, + method: event.method, + path: event.path, + headers: event.headers, + ...(contentType(event.headers) === undefined ? {} : { content_type: contentType(event.headers) }), + }; +} + +function httpEventFromResponse(event: SpoolResponseEvent, callId: string): HttpEventRecord { + return { + id: `http-${callId}-response`, + call_id: callId, + direction: "response", + observed_at: event.ts, + host: event.host, + method: event.method, + path: event.path, + status_code: event.status_code, + reason: event.reason, + headers: event.headers, + request_headers: event.request_headers, + ...(contentType(event.headers) === undefined ? {} : { content_type: contentType(event.headers) }), + }; +} + +function contentType(headers: readonly (readonly [string, string])[]): string | undefined { + const pair = headers.find(([name]) => name.toLowerCase() === "content-type"); + return pair?.[1]; +} + +function positiveNumber(value: number | undefined, fallback: number): number { + return value === undefined || !Number.isFinite(value) || value <= 0 ? fallback : value; +} + +function errorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +function unlinkIfExists(path: string): void { + try { + rmSync(path, { force: true }); + } catch { + // Best effort cleanup; ingestion counters preserve the failure context. + } +} diff --git a/vitest.config.ts b/vitest.config.ts index 7e8ca30..43c380f 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -8,7 +8,10 @@ export default defineConfig({ name: "unit", environment: "node", include: ["src/**/*.test.ts"], - exclude: ["src/rootcell/integration/**/*.integration.test.ts"], + exclude: [ + "src/rootcell/integration/**/*.integration.test.ts", + "src/spy/**/*.test.ts", + ], testTimeout: 10_000, hookTimeout: 10_000, }, From df9aa92958e97f36fee9e56801652e61bc825d4d Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Fri, 22 May 2026 21:55:23 -0400 Subject: [PATCH 06/52] Add spy web service API --- PLAN.md | 22 +- src/bin/spy-service.ts | 14 + src/spy/service.test.ts | 276 +++++++++++++++++ src/spy/service.ts | 607 ++++++++++++++++++++++++++++++++++++++ src/spy/store.ts | 636 +++++++++++++++++++++++++++++++++++++++- 5 files changed, 1552 insertions(+), 3 deletions(-) create mode 100644 src/bin/spy-service.ts create mode 100644 src/spy/service.test.ts create mode 100644 src/spy/service.ts diff --git a/PLAN.md b/PLAN.md index e160e1e..72267b6 100644 --- a/PLAN.md +++ b/PLAN.md @@ -482,6 +482,26 @@ V1 excludes: updates, idempotency, raw payload gating, malformed/drop/error events, retention with FTS cleanup, and clear-data. - Verified `bun run typecheck`, `bun run lint`, and `bun run test`. +- Implemented the TypeScript spy web service, API, SSE, and static asset + serving: + - Added `src/spy/service.ts` and `src/bin/spy-service.ts` for the Bun HTTP + service runtime. + - Added environment-backed service config with V1 defaults for bind address, + port, SQLite path, spool path, retention, size caps, raw payload storage, + ingestion cadence, and retention cadence. + - Extended `src/spy/store.ts` with read-side APIs for paginated call + summaries, call details, stream event pages, FTS search, and previous-call + request diffs. + - Implemented same-origin JSON endpoints for health, call list/detail, diff, + stream events, search, and confirmed clear-data. + - Implemented SSE notifications for initial connection, health changes, call + changes, clear-data events, and keepalive comments. + - Added static asset serving with index fallback for browser routes and path + traversal rejection. + - Added fixture-backed Bun coverage for API behavior, pagination, raw payload + gating, clear-data confirmation, SSE updates, static serving, and bad input + handling. + - Verified `bun run typecheck`, `bun run lint`, and `bun run test`. ### V1 @@ -494,7 +514,7 @@ Build the Bedrock/Pi browser spy: - [x] Replace Python spy with minimal provider-gated spool shim. - [x] Implement TypeScript Bedrock adapter on top of the captured fixtures. - [x] Implement SQLite persistence, migrations, retention, and clear-data. -- [ ] Implement TS web service, API, SSE, and static asset serving. +- [x] Implement TS web service, API, SSE, and static asset serving. - [ ] Implement React desktop UI with virtualized timeline and call inspector. - [ ] Wire `rootcell provision`, systemd service config, and `rootcell spy` launcher/tunnel. diff --git a/src/bin/spy-service.ts b/src/bin/spy-service.ts new file mode 100644 index 0000000..ae8aa09 --- /dev/null +++ b/src/bin/spy-service.ts @@ -0,0 +1,14 @@ +#!/usr/bin/env bun +import { spyServiceConfigFromEnv, startSpyService } from "../spy/service.ts"; + +const handle = startSpyService({ config: spyServiceConfigFromEnv() }); +console.log(`rootcell spy service listening on ${handle.url}`); + +const stop = (signal: NodeJS.Signals): void => { + void handle.stop().finally(() => { + process.exit(signal === "SIGINT" ? 130 : 143); + }); +}; + +process.once("SIGINT", stop); +process.once("SIGTERM", stop); diff --git a/src/spy/service.test.ts b/src/spy/service.test.ts new file mode 100644 index 0000000..e660a80 --- /dev/null +++ b/src/spy/service.test.ts @@ -0,0 +1,276 @@ +import { mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, test } from "bun:test"; +import { + type SpyCallDetail, + type SpyCallDiff, + type SpyCallSummary, + type SpyPaginatedResult, +} from "./store.ts"; +import { SpoolEventSchema, type SpoolEvent, type StreamEvent } from "./schemas.ts"; +import { startSpyService, type SpyServiceHandle } from "./service.ts"; + +const FIXTURE_PATH = new URL("./fixtures/bedrock-pi-us-sonnet-4-6.ndjson", import.meta.url); + +interface TestService { + readonly root: string; + readonly dbPath: string; + readonly spoolDir: string; + readonly staticDir: string; + readonly handle: SpyServiceHandle; +} + +interface SseReader { + read(): Promise< + | { readonly done: true; readonly value?: undefined } + | { readonly done: false; readonly value: Uint8Array } + >; +} + +const tempRoots: string[] = []; +const serviceHandles: SpyServiceHandle[] = []; + +afterEach(async () => { + for (const handle of serviceHandles.splice(0)) { + await handle.stop(); + } + for (const root of tempRoots.splice(0)) { + rmSync(root, { recursive: true, force: true }); + } +}); + +function fixtureEvents(): SpoolEvent[] { + return readFileSync(FIXTURE_PATH, "utf8") + .trim() + .split("\n") + .filter(Boolean) + .map((line) => SpoolEventSchema.parse(JSON.parse(line) as unknown)); +} + +function createTestService(options: { + readonly storeRaw?: boolean | undefined; + readonly startIngestion?: boolean | undefined; +} = {}): TestService { + const root = mkdtempSync(join(tmpdir(), "rootcell-spy-service-")); + tempRoots.push(root); + const dbPath = join(root, "spy.sqlite"); + const spoolDir = join(root, "spool"); + const staticDir = join(root, "static"); + mkdirSync(staticDir, { recursive: true }); + writeFileSync(join(staticDir, "index.html"), "Rootcell Spy
"); + mkdirSync(join(staticDir, "assets"), { recursive: true }); + writeFileSync(join(staticDir, "assets", "app.js"), "globalThis.rootcellSpyAsset = true;\n"); + + const handle = startSpyService({ + config: { + bind: "127.0.0.1", + port: 0, + dbPath, + spoolDir, + staticDir, + storeRaw: options.storeRaw === true, + ingestIntervalMs: 60_000, + retentionIntervalMs: 60_000, + }, + startIngestion: options.startIngestion ?? false, + }); + serviceHandles.push(handle); + return { root, dbPath, spoolDir, staticDir, handle }; +} + +function writeSpoolEvents(spoolDir: string, events: readonly SpoolEvent[]): void { + events.forEach((event, index) => { + const flowId = "flow_id" in event && event.flow_id !== undefined ? event.flow_id : "no-flow"; + writeFileSync( + join(spoolDir, `${String(index).padStart(3, "0")}-${event.direction}-${flowId}.json`), + `${JSON.stringify(event)}\n`, + ); + }); +} + +async function jsonAs(response: Response): Promise { + const parsed: unknown = await response.json(); + return parsed as T; +} + +describe("spy web service", () => { + test("serves health, paginated calls, details, diff, stream events, and search", async () => { + const { handle, spoolDir } = createTestService(); + writeSpoolEvents(spoolDir, fixtureEvents()); + + expect(handle.ingestOnce()).toMatchObject({ + attempted: 10, + ingested: 10, + deleted: 10, + deferred: 0, + }); + + const healthResponse = await fetch(`${handle.url}/api/health`); + expect(healthResponse.status).toBe(200); + const health = await jsonAs<{ readonly service: { readonly storeRaw: boolean }; readonly store: { readonly providerCallCount: number } }>(healthResponse); + expect(health.service.storeRaw).toBe(false); + expect(health.store.providerCallCount).toBe(5); + + const firstPageResponse = await fetch(`${handle.url}/api/calls?limit=2`); + const firstPage = await jsonAs>(firstPageResponse); + expect(firstPage.items).toHaveLength(2); + expect(firstPage.nextCursor).toBeDefined(); + + const secondPageResponse = await fetch(`${handle.url}/api/calls?limit=2&cursor=${encodeURIComponent(firstPage.nextCursor ?? "")}`); + const secondPage = await jsonAs>(secondPageResponse); + expect(secondPage.items).toHaveLength(2); + expect(secondPage.items[0]?.call.id).not.toBe(firstPage.items[0]?.call.id); + + const callId = firstPage.items[0]?.call.id; + if (callId === undefined) { + throw new Error("missing call id"); + } + + const detailResponse = await fetch(`${handle.url}/api/calls/${encodeURIComponent(callId)}`); + expect(detailResponse.status).toBe(200); + const detail = await jsonAs(detailResponse); + expect(detail.summary.call.id).toBe(callId); + expect(detail.httpEvents).toHaveLength(2); + expect(detail.blocks.length).toBeGreaterThan(0); + expect(detail.usageRecords.length).toBeGreaterThan(0); + expect(detail.rawPayloads).toHaveLength(0); + + const streamResponse = await fetch(`${handle.url}/api/calls/${encodeURIComponent(callId)}/stream-events?limit=2`); + const streamPage = await jsonAs>(streamResponse); + expect(streamPage.items).toHaveLength(2); + expect(streamPage.nextCursor).toBeDefined(); + + const diffResponse = await fetch(`${handle.url}/api/calls/${encodeURIComponent(callId)}/diff`); + const diff = await jsonAs(diffResponse); + expect(diff.call.call.id).toBe(callId); + expect(diff.blocks.length).toBeGreaterThan(0); + expect(diff.blocks.every((block) => ["new", "repeated", "changed", "unknown"].includes(block.classification))).toBe(true); + + const searchResponse = await fetch(`${handle.url}/api/search?q=${encodeURIComponent("Fixture capture")}`); + const searchPage = await jsonAs>(searchResponse); + expect(searchPage.items.length).toBeGreaterThan(0); + + const invalidCursorResponse = await fetch(`${handle.url}/api/calls?cursor=not-a-cursor`); + expect(invalidCursorResponse.status).toBe(400); + + const missingResponse = await fetch(`${handle.url}/api/calls/missing-call`); + expect(missingResponse.status).toBe(404); + }); + + test("returns raw payloads only when raw storage is enabled", async () => { + const { handle, spoolDir } = createTestService({ storeRaw: true }); + writeSpoolEvents(spoolDir, fixtureEvents()); + expect(handle.ingestOnce().ingested).toBe(10); + + const page = await jsonAs>(await fetch(`${handle.url}/api/calls?limit=1`)); + const callId = page.items[0]?.call.id; + if (callId === undefined) { + throw new Error("missing raw payload call id"); + } + + const detail = await jsonAs(await fetch(`${handle.url}/api/calls/${encodeURIComponent(callId)}`)); + expect(detail.rawPayloads).toHaveLength(2); + expect(detail.rawPayloads.some((payload) => payload.direction === "request" && payload.body_text !== undefined)).toBe(true); + expect(detail.rawPayloads.some((payload) => payload.direction === "response" && payload.body_encoding === "aws-eventstream")).toBe(true); + }); + + test("clears captured rows and pending spool only with confirmation", async () => { + const { handle, spoolDir } = createTestService(); + const events = fixtureEvents(); + writeSpoolEvents(spoolDir, events); + expect(handle.ingestOnce().ingested).toBe(10); + writeSpoolEvents(spoolDir, events.slice(0, 1)); + + const rejected = await fetch(`${handle.url}/api/clear`, { + method: "POST", + body: JSON.stringify({ confirm: false }), + }); + expect(rejected.status).toBe(400); + + const cleared = await fetch(`${handle.url}/api/clear`, { + method: "POST", + body: JSON.stringify({ confirm: true }), + }); + expect(cleared.status).toBe(200); + expect(readdirSync(spoolDir)).toEqual([]); + + const health = await jsonAs<{ readonly store: { readonly providerCallCount: number } }>(await fetch(`${handle.url}/api/health`)); + expect(health.store.providerCallCount).toBe(0); + }); + + test("emits SSE hello, calls-changed, health, and cleared events", async () => { + const { handle, spoolDir } = createTestService(); + const response = await fetch(`${handle.url}/api/events`); + expect(response.status).toBe(200); + if (response.body === null) { + throw new Error("missing SSE body"); + } + const reader = response.body.getReader(); + try { + expect(await readSseUntil(reader, "event: hello")).toContain("event: health"); + + writeSpoolEvents(spoolDir, fixtureEvents().slice(0, 2)); + expect(handle.ingestOnce().ingested).toBe(2); + expect(await readSseUntil(reader, "event: calls-changed")).toContain("event: health"); + + const cleared = await fetch(`${handle.url}/api/clear`, { + method: "POST", + body: JSON.stringify({ confirm: true }), + }); + expect(cleared.status).toBe(200); + expect(await readSseUntil(reader, "event: cleared")).toContain("event: health"); + } finally { + await reader.cancel(); + } + }); + + test("serves static assets, falls back to index, and rejects traversal", async () => { + const { handle } = createTestService(); + + const asset = await fetch(`${handle.url}/assets/app.js`); + expect(asset.status).toBe(200); + expect(asset.headers.get("content-type")).toContain("text/javascript"); + expect(await asset.text()).toContain("rootcellSpyAsset"); + + const route = await fetch(`${handle.url}/calls/call-fixture-flow-simple`, { + headers: { accept: "text/html" }, + }); + expect(route.status).toBe(200); + expect(await route.text()).toContain("Rootcell Spy"); + + const traversal = await fetch(`${handle.url}/..%2fPLAN.md`); + expect(traversal.status).toBe(403); + }); +}); + +async function readSseUntil( + reader: SseReader, + needle: string, +): Promise { + const decoder = new TextDecoder(); + let text = ""; + const deadline = Date.now() + 2_000; + while (!text.includes(needle)) { + const remaining = deadline - Date.now(); + if (remaining <= 0) { + throw new Error(`timed out waiting for ${needle}; saw ${text}`); + } + const result = await Promise.race([ + reader.read(), + sleep(remaining).then(() => "timeout" as const), + ]); + if (result === "timeout") { + throw new Error(`timed out waiting for ${needle}; saw ${text}`); + } + if (result.done) { + throw new Error(`SSE ended before ${needle}; saw ${text}`); + } + text += decoder.decode(result.value, { stream: true }); + } + return text; +} + +async function sleep(ms: number): Promise { + await new Promise((resolve) => setTimeout(resolve, ms)); +} diff --git a/src/spy/service.ts b/src/spy/service.ts new file mode 100644 index 0000000..2df2d13 --- /dev/null +++ b/src/spy/service.ts @@ -0,0 +1,607 @@ +import { existsSync, statSync } from "node:fs"; +import { extname, isAbsolute, join, relative, resolve } from "node:path"; +import { z } from "zod"; +import { + openSpyStore, + type IngestSpoolBatchResult, + type RetentionResult, + type SpyHealthSnapshot, + type SpyListCallsOptions, + type SpySearchCallsOptions, + type SpyStore, + type SpyStoreOptions, + type SpyStreamEventsOptions, +} from "./store.ts"; + +const DEFAULT_BIND = "127.0.0.1"; +const DEFAULT_PORT = 6174; +const DEFAULT_DB_PATH = "/var/lib/rootcell-spy/spy.sqlite"; +const DEFAULT_SPOOL_DIR = "/var/spool/rootcell-spy"; +const DEFAULT_RETENTION_DAYS = 7; +const DEFAULT_MAX_BYTES = 6 * 1024 * 1024 * 1024; +const DEFAULT_SPOOL_MAX_BYTES = 1024 * 1024 * 1024; +const DEFAULT_INGEST_INTERVAL_MS = 500; +const DEFAULT_RETENTION_INTERVAL_MS = 15 * 60 * 1000; +const DEFAULT_INGEST_BATCH_LIMIT = 100; + +const ClearRequestSchema = z.object({ + confirm: z.literal(true), +}).strict(); + +export interface SpyServiceConfig { + readonly bind: string; + readonly port: number; + readonly dbPath: string; + readonly spoolDir: string; + readonly staticDir?: string | undefined; + readonly retentionDays: number; + readonly maxBytes: number; + readonly spoolMaxBytes: number; + readonly storeRaw: boolean; + readonly ingestIntervalMs: number; + readonly retentionIntervalMs: number; + readonly ingestBatchLimit: number; +} + +export interface SpyServiceHealth { + readonly ok: true; + readonly service: { + readonly bind: string; + readonly port: number; + readonly retentionDays: number; + readonly maxBytes: number; + readonly spoolMaxBytes: number; + readonly storeRaw: boolean; + readonly staticAssets: boolean; + }; + readonly store: SpyHealthSnapshot; +} + +export interface StartSpyServiceOptions { + readonly config?: Partial | undefined; + readonly startIngestion?: boolean | undefined; + readonly now?: (() => number) | undefined; +} + +export interface SpyServiceHandle { + readonly url: string; + readonly config: SpyServiceConfig; + readonly store: SpyStore; + ingestOnce(): IngestSpoolBatchResult; + runRetentionOnce(): RetentionResult; + stop(): Promise; +} + +interface SseClient { + readonly id: number; + readonly controller: ReadableStreamDefaultController; + readonly keepalive: ReturnType; +} + +class HttpError extends Error { + constructor( + readonly status: number, + message: string, + ) { + super(message); + } +} + +class SpyHttpService { + private readonly encoder = new TextEncoder(); + private readonly clients = new Map(); + private ingestTimer: ReturnType | undefined; + private retentionTimer: ReturnType | undefined; + private nextClientId = 1; + + constructor( + private readonly config: SpyServiceConfig, + private readonly store: SpyStore, + ) {} + + start(startIngestion: boolean): void { + this.runRetentionOnce(); + if (!startIngestion) { + return; + } + this.ingestTimer = setInterval(() => { + this.ingestOnce(); + }, this.config.ingestIntervalMs); + this.retentionTimer = setInterval(() => { + this.runRetentionOnce(); + }, this.config.retentionIntervalMs); + } + + stop(): void { + if (this.ingestTimer !== undefined) { + clearInterval(this.ingestTimer); + this.ingestTimer = undefined; + } + if (this.retentionTimer !== undefined) { + clearInterval(this.retentionTimer); + this.retentionTimer = undefined; + } + for (const client of this.clients.values()) { + clearInterval(client.keepalive); + try { + client.controller.close(); + } catch { + // The browser may already have disconnected. + } + } + this.clients.clear(); + } + + ingestOnce(): IngestSpoolBatchResult { + const result = this.store.ingestSpoolBatch({ limit: this.config.ingestBatchLimit }); + if (result.ingested > 0 || result.malformed > 0 || result.errors > 0 || result.deleted > 0) { + this.broadcast("calls-changed", { result }); + this.broadcastHealth(); + } + return result; + } + + runRetentionOnce(): RetentionResult { + const result = this.store.runRetention(); + if (result.deletedByAge > 0 || result.deletedBySize > 0) { + this.broadcast("calls-changed", { retention: result }); + this.broadcastHealth(); + } + return result; + } + + async handle(request: Request): Promise { + try { + const url = new URL(request.url); + const path = decodedPathname(url); + if (path.startsWith("/api/")) { + return await this.handleApi(request, url, path); + } + if (path === "/api") { + return jsonError(404, "not found"); + } + return this.serveStatic(path, request.headers); + } catch (error) { + return responseForError(error); + } + } + + private async handleApi(request: Request, url: URL, path: string): Promise { + if (request.method === "GET" && path === "/api/health") { + return jsonResponse(this.health()); + } + if (request.method === "GET" && path === "/api/events") { + return this.sseResponse(); + } + if (request.method === "GET" && path === "/api/calls") { + return jsonResponse(this.store.listCallSummaries(listOptions(url))); + } + if (request.method === "GET" && path === "/api/search") { + return jsonResponse(this.store.searchCallSummaries(searchOptions(url))); + } + if (request.method === "POST" && path === "/api/clear") { + const body = await jsonBody(request); + ClearRequestSchema.parse(body); + const result = this.store.clearData(); + this.broadcast("cleared", result); + this.broadcastHealth(); + return jsonResponse(result); + } + + const streamMatch = /^\/api\/calls\/([^/]+)\/stream-events$/.exec(path); + if (request.method === "GET" && streamMatch !== null) { + const callId = decodeURIComponent(streamMatch[1] ?? ""); + return jsonResponse(this.store.getStreamEvents(callId, streamOptions(url))); + } + + const diffMatch = /^\/api\/calls\/([^/]+)\/diff$/.exec(path); + if (request.method === "GET" && diffMatch !== null) { + const callId = decodeURIComponent(diffMatch[1] ?? ""); + const diff = this.store.getCallDiff(callId); + if (diff === null) { + throw new HttpError(404, "call not found"); + } + return jsonResponse(diff); + } + + const detailMatch = /^\/api\/calls\/([^/]+)$/.exec(path); + if (request.method === "GET" && detailMatch !== null) { + const callId = decodeURIComponent(detailMatch[1] ?? ""); + const detail = this.store.getCallDetail(callId); + if (detail === null) { + throw new HttpError(404, "call not found"); + } + return jsonResponse(detail); + } + + return jsonError(404, "not found"); + } + + private health(): SpyServiceHealth { + return { + ok: true, + service: { + bind: this.config.bind, + port: this.config.port, + retentionDays: this.config.retentionDays, + maxBytes: this.config.maxBytes, + spoolMaxBytes: this.config.spoolMaxBytes, + storeRaw: this.config.storeRaw, + staticAssets: this.config.staticDir !== undefined, + }, + store: this.store.getHealthSnapshot(), + }; + } + + private sseResponse(): Response { + const id = this.nextClientId; + this.nextClientId += 1; + + const stream = new ReadableStream({ + start: (controller) => { + const keepalive = setInterval(() => { + this.sendComment(controller, "keepalive"); + }, 15_000); + const client: SseClient = { id, controller, keepalive }; + this.clients.set(id, client); + this.send(controller, "hello", { id }); + this.send(controller, "health", this.health()); + }, + cancel: () => { + this.removeClient(id); + }, + }); + + return new Response(stream, { + headers: { + "Cache-Control": "no-cache", + "Connection": "keep-alive", + "Content-Type": "text/event-stream; charset=utf-8", + "X-Accel-Buffering": "no", + }, + }); + } + + private serveStatic(path: string, requestHeaders: Headers): Response { + const staticDir = this.config.staticDir; + if (staticDir === undefined) { + return jsonError(404, "not found"); + } + + const root = resolve(staticDir); + let candidate = path === "/" ? join(root, "index.html") : resolve(root, path.slice(1)); + if (!isPathInside(root, candidate)) { + throw new HttpError(403, "forbidden"); + } + + const stat = fileStat(candidate); + if (stat?.isDirectory() === true) { + candidate = join(candidate, "index.html"); + } + + if (fileStat(candidate)?.isFile() !== true) { + if (!shouldFallbackToIndex(path, requestHeaders)) { + return jsonError(404, "not found"); + } + candidate = join(root, "index.html"); + if (!isPathInside(root, candidate) || fileStat(candidate)?.isFile() !== true) { + return jsonError(404, "not found"); + } + } + + return new Response(Bun.file(candidate), { + headers: { + "Cache-Control": "no-cache", + "Content-Type": contentTypeForPath(candidate), + }, + }); + } + + private broadcastHealth(): void { + this.broadcast("health", this.health()); + } + + private broadcast(event: string, data: unknown): void { + for (const client of this.clients.values()) { + this.send(client.controller, event, data); + } + } + + private send(controller: ReadableStreamDefaultController, event: string, data: unknown): void { + try { + controller.enqueue(this.encoder.encode(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`)); + } catch { + // Disconnect cleanup runs through stream cancellation. + } + } + + private sendComment(controller: ReadableStreamDefaultController, comment: string): void { + try { + controller.enqueue(this.encoder.encode(`: ${comment}\n\n`)); + } catch { + // Disconnect cleanup runs through stream cancellation. + } + } + + private removeClient(id: number): void { + const client = this.clients.get(id); + if (client === undefined) { + return; + } + clearInterval(client.keepalive); + this.clients.delete(id); + } +} + +export function spyServiceConfigFromEnv(env: NodeJS.ProcessEnv = process.env): SpyServiceConfig { + const staticDir = nonEmpty(env.ROOTCELL_SPY_STATIC_DIR); + return { + bind: nonEmpty(env.ROOTCELL_SPY_BIND) ?? DEFAULT_BIND, + port: envNumber(env.ROOTCELL_SPY_PORT, DEFAULT_PORT), + dbPath: nonEmpty(env.ROOTCELL_SPY_DB_PATH) ?? DEFAULT_DB_PATH, + spoolDir: nonEmpty(env.ROOTCELL_SPY_SPOOL_DIR) ?? DEFAULT_SPOOL_DIR, + ...(staticDir === undefined ? {} : { staticDir }), + retentionDays: envNumber(env.ROOTCELL_SPY_RETENTION_DAYS, DEFAULT_RETENTION_DAYS), + maxBytes: envNumber(env.ROOTCELL_SPY_MAX_BYTES, DEFAULT_MAX_BYTES), + spoolMaxBytes: envNumber(env.ROOTCELL_SPY_SPOOL_MAX_BYTES, DEFAULT_SPOOL_MAX_BYTES), + storeRaw: envBoolean(env.ROOTCELL_SPY_STORE_RAW, false), + ingestIntervalMs: envNumber(env.ROOTCELL_SPY_INGEST_INTERVAL_MS, DEFAULT_INGEST_INTERVAL_MS), + retentionIntervalMs: envNumber(env.ROOTCELL_SPY_RETENTION_INTERVAL_MS, DEFAULT_RETENTION_INTERVAL_MS), + ingestBatchLimit: envNumber(env.ROOTCELL_SPY_INGEST_BATCH_LIMIT, DEFAULT_INGEST_BATCH_LIMIT), + }; +} + +export function startSpyService(options: StartSpyServiceOptions = {}): SpyServiceHandle { + const config = { ...spyServiceConfigFromEnv({}), ...options.config }; + const storeOptions: SpyStoreOptions = { + dbPath: config.dbPath, + spoolDir: config.spoolDir, + retentionDays: config.retentionDays, + maxBytes: config.maxBytes, + storeRaw: config.storeRaw, + ...(options.now === undefined ? {} : { now: options.now }), + }; + const store = openSpyStore(storeOptions); + let service: SpyHttpService | undefined; + let server: Bun.Server | undefined; + let activeConfig = config; + let lastListenError: unknown; + for (const port of candidatePorts(config.port)) { + activeConfig = { ...config, port }; + service = new SpyHttpService(activeConfig, store); + try { + server = Bun.serve({ + hostname: activeConfig.bind, + port: activeConfig.port, + fetch: (request) => { + if (service === undefined) { + return jsonError(503, "service unavailable"); + } + return service.handle(request); + }, + }); + break; + } catch (error) { + lastListenError = error; + if (config.port !== 0 || !isAddrInUse(error)) { + store.close(); + throw error; + } + } + } + if (server === undefined || service === undefined) { + store.close(); + throw lastListenError instanceof Error ? lastListenError : new Error("failed to start spy service"); + } + try { + service.start(options.startIngestion ?? true); + } catch (error) { + void server.stop(true); + store.close(); + throw error; + } + const actualPort = server.port ?? activeConfig.port; + + return { + url: `http://${activeConfig.bind}:${String(actualPort)}`, + config: { ...activeConfig, port: actualPort }, + store, + ingestOnce: () => service.ingestOnce(), + runRetentionOnce: () => service.runRetentionOnce(), + stop: async () => { + service.stop(); + await server.stop(true); + store.close(); + }, + }; +} + +function listOptions(url: URL): SpyListCallsOptions { + return { + ...(numberParam(url, "since") === undefined ? {} : { since: numberParam(url, "since") }), + ...(stringParam(url, "cursor") === undefined ? {} : { cursor: stringParam(url, "cursor") }), + ...(numberParam(url, "limit") === undefined ? {} : { limit: numberParam(url, "limit") }), + }; +} + +function searchOptions(url: URL): SpySearchCallsOptions { + return { + query: stringParam(url, "q") ?? "", + ...(stringParam(url, "cursor") === undefined ? {} : { cursor: stringParam(url, "cursor") }), + ...(numberParam(url, "limit") === undefined ? {} : { limit: numberParam(url, "limit") }), + }; +} + +function streamOptions(url: URL): SpyStreamEventsOptions { + return { + ...(stringParam(url, "cursor") === undefined ? {} : { cursor: stringParam(url, "cursor") }), + ...(numberParam(url, "limit") === undefined ? {} : { limit: numberParam(url, "limit") }), + }; +} + +async function jsonBody(request: Request): Promise { + try { + return await request.json(); + } catch { + throw new HttpError(400, "invalid JSON body"); + } +} + +function stringParam(url: URL, name: string): string | undefined { + const value = url.searchParams.get(name); + return value === null || value.length === 0 ? undefined : value; +} + +function numberParam(url: URL, name: string): number | undefined { + const value = stringParam(url, name); + if (value === undefined) { + return undefined; + } + const parsed = Number(value); + if (!Number.isFinite(parsed)) { + throw new HttpError(400, `invalid ${name}`); + } + return parsed; +} + +function decodedPathname(url: URL): string { + try { + const decoded = decodeURIComponent(url.pathname); + if (decoded.includes("\0")) { + throw new HttpError(400, "invalid path"); + } + return decoded; + } catch (error) { + if (error instanceof HttpError) { + throw error; + } + throw new HttpError(400, "invalid path"); + } +} + +function jsonResponse(value: unknown, init: ResponseInit = {}): Response { + return new Response(JSON.stringify(value), { + ...(init.status === undefined ? {} : { status: init.status }), + ...(init.statusText === undefined ? {} : { statusText: init.statusText }), + headers: { + "Content-Type": "application/json; charset=utf-8", + }, + }); +} + +function jsonError(status: number, message: string): Response { + return jsonResponse({ error: message }, { status }); +} + +function responseForError(error: unknown): Response { + if (error instanceof HttpError) { + return jsonError(error.status, error.message); + } + if (error instanceof z.ZodError) { + return jsonError(400, error.issues[0]?.message ?? "invalid request"); + } + if (error instanceof Error && error.message.includes("cursor")) { + return jsonError(400, error.message); + } + return jsonError(500, error instanceof Error ? error.message : "internal error"); +} + +function nonEmpty(value: string | undefined): string | undefined { + const trimmed = value?.trim(); + return trimmed === undefined || trimmed.length === 0 ? undefined : trimmed; +} + +function envNumber(value: string | undefined, fallback: number): number { + if (value === undefined || value.trim().length === 0) { + return fallback; + } + const parsed = Number(value); + return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback; +} + +function envBoolean(value: string | undefined, fallback: boolean): boolean { + if (value === undefined || value.trim().length === 0) { + return fallback; + } + return ["1", "true", "yes", "on"].includes(value.trim().toLowerCase()); +} + +function candidatePorts(port: number): number[] { + if (port !== 0) { + return [port]; + } + const ports: number[] = []; + const base = 20_000 + Math.floor(Math.random() * 20_000); + for (let offset = 0; offset < 40; offset += 1) { + ports.push(base + offset); + } + return ports; +} + +function isAddrInUse(error: unknown): boolean { + if (error instanceof Error && error.message.includes("EADDRINUSE")) { + return true; + } + if (error instanceof Error && error.message.includes("in use")) { + return true; + } + if (typeof error !== "object" || error === null) { + return false; + } + if ("code" in error && error.code === "EADDRINUSE") { + return true; + } + return "message" in error + && typeof error.message === "string" + && (error.message.includes("EADDRINUSE") || error.message.includes("in use")); +} + +function isPathInside(root: string, candidate: string): boolean { + const rel = relative(root, candidate); + return rel.length === 0 || (!rel.startsWith("..") && !isAbsolute(rel)); +} + +function fileStat(path: string): ReturnType | null { + if (!existsSync(path)) { + return null; + } + try { + return statSync(path); + } catch { + return null; + } +} + +function shouldFallbackToIndex(path: string, requestHeaders: Headers): boolean { + if (path === "/") { + return true; + } + const accept = requestHeaders.get("accept") ?? ""; + return extname(path).length === 0 || accept.includes("text/html"); +} + +function contentTypeForPath(path: string): string { + const extension = extname(path).toLowerCase(); + if (extension === ".html") { + return "text/html; charset=utf-8"; + } + if (extension === ".js" || extension === ".mjs") { + return "text/javascript; charset=utf-8"; + } + if (extension === ".css") { + return "text/css; charset=utf-8"; + } + if (extension === ".json") { + return "application/json; charset=utf-8"; + } + if (extension === ".svg") { + return "image/svg+xml"; + } + if (extension === ".png") { + return "image/png"; + } + if (extension === ".ico") { + return "image/x-icon"; + } + return "application/octet-stream"; +} diff --git a/src/spy/store.ts b/src/spy/store.ts index 4a1679f..42cf9c8 100644 --- a/src/spy/store.ts +++ b/src/spy/store.ts @@ -9,7 +9,13 @@ import { import { applySpyMigrations, currentSpySchemaVersion } from "./migrations.ts"; import { HttpEventRecordSchema, + NormalizedBlockSchema, + ProviderCallSchema, + RawPayloadRecordSchema, SpoolEventSchema, + StreamEventSchema, + UsageRecordSchema, + type DiffClassification, type HttpEventRecord, type NormalizedBlock, type ProviderCall, @@ -26,6 +32,8 @@ import { const DEFAULT_RETENTION_DAYS = 7; const DEFAULT_MAX_BYTES = 6 * 1024 * 1024 * 1024; +const DEFAULT_QUERY_LIMIT = 100; +const MAX_QUERY_LIMIT = 500; export interface SpyStoreOptions { readonly dbPath: string; @@ -72,10 +80,78 @@ export interface SpyHealthSnapshot { readonly metadata: Readonly>; } +export interface SpyUsageSummary { + readonly inputTokens: number | null; + readonly outputTokens: number | null; + readonly cacheReadTokens: number | null; + readonly cacheWriteTokens: number | null; + readonly totalTokens: number | null; +} + +export interface SpyCallSummary { + readonly call: ProviderCall; + readonly durationMs: number | null; + readonly usage: SpyUsageSummary; + readonly requestBlockCount: number; + readonly responseBlockCount: number; + readonly requestByteSize: number; + readonly responseByteSize: number; + readonly cacheMarkerCount: number; + readonly streamEventCount: number; + readonly rawPayloadCount: number; +} + +export interface SpyPaginatedResult { + readonly items: readonly T[]; + readonly nextCursor?: string | undefined; +} + +export interface SpyListCallsOptions { + readonly since?: number | undefined; + readonly cursor?: string | undefined; + readonly limit?: number | undefined; +} + +export interface SpySearchCallsOptions { + readonly query: string; + readonly cursor?: string | undefined; + readonly limit?: number | undefined; +} + +export interface SpyStreamEventsOptions { + readonly cursor?: string | undefined; + readonly limit?: number | undefined; +} + +export interface SpyCallDetail { + readonly summary: SpyCallSummary; + readonly httpEvents: readonly HttpEventRecord[]; + readonly blocks: readonly NormalizedBlock[]; + readonly usageRecords: readonly UsageRecord[]; + readonly rawPayloads: readonly RawPayloadRecord[]; +} + +export interface SpyBlockDiff { + readonly block: NormalizedBlock; + readonly classification: DiffClassification; + readonly previousBlockId?: string | undefined; +} + +export interface SpyCallDiff { + readonly call: SpyCallSummary; + readonly previousCall: SpyCallSummary | null; + readonly blocks: readonly SpyBlockDiff[]; +} + export interface SpyStore { ingestSpoolBatch(options?: IngestSpoolBatchOptions): IngestSpoolBatchResult; persistRequest(event: SpoolRequestEvent): void; persistResponse(event: SpoolResponseEvent): boolean; + listCallSummaries(options?: SpyListCallsOptions): SpyPaginatedResult; + getCallDetail(callId: string): SpyCallDetail | null; + getCallDiff(callId: string): SpyCallDiff | null; + getStreamEvents(callId: string, options?: SpyStreamEventsOptions): SpyPaginatedResult; + searchCallSummaries(options: SpySearchCallsOptions): SpyPaginatedResult; runRetention(): RetentionResult; clearData(): ClearDataResult; getHealthSnapshot(): SpyHealthSnapshot; @@ -96,12 +172,122 @@ interface CountRow { readonly count: number; } +interface SumRow { + readonly total: number | null; +} + interface IdRow { readonly id: string; } type PragmaRow = Readonly>; +interface ProviderCallRow { + readonly id: string; + readonly provider: "bedrock"; + readonly operation: string; + readonly model_id: string; + readonly status: ProviderCall["status"]; + readonly started_at: number; + readonly completed_at: number | null; + readonly status_code: number | null; + readonly request_flow_id: string; + readonly response_flow_id: string | null; + readonly request_content_hash: string | null; + readonly response_content_hash: string | null; +} + +interface HttpEventRow { + readonly id: string; + readonly call_id: string; + readonly direction: "request" | "response"; + readonly observed_at: number; + readonly host: string; + readonly method: string; + readonly path: string; + readonly status_code: number | null; + readonly reason: string | null; + readonly headers_json: string; + readonly request_headers_json: string | null; + readonly body_text: string | null; + readonly body_b64: string | null; + readonly body_sha256: string | null; + readonly body_encoding: "aws-eventstream" | null; + readonly content_type: string | null; +} + +interface NormalizedBlockRow { + readonly id: string; + readonly call_id: string; + readonly direction: "request" | "response"; + readonly ordinal: number; + readonly role: string | null; + readonly kind: NormalizedBlock["kind"]; + readonly source: string; + readonly provider_path: string | null; + readonly text: string | null; + readonly json: string | null; + readonly char_size: number; + readonly byte_size: number; + readonly content_hash: string; + readonly cache_marker: number; +} + +interface UsageRecordRow { + readonly id: string; + readonly call_id: string; + readonly source: "provider-reported"; + readonly input_tokens: number | null; + readonly output_tokens: number | null; + readonly cache_read_tokens: number | null; + readonly cache_write_tokens: number | null; + readonly total_tokens: number | null; + readonly raw_json: string | null; +} + +interface StreamEventRow { + readonly id: string; + readonly call_id: string; + readonly ordinal: number; + readonly event_type: string; + readonly headers_json: string; + readonly payload_json: string | null; + readonly payload_text: string | null; + readonly payload_sha256: string | null; + readonly observed_at: number | null; +} + +interface RawPayloadRow { + readonly id: string; + readonly call_id: string; + readonly direction: "request" | "response"; + readonly content_type: string | null; + readonly body_text: string | null; + readonly body_b64: string | null; + readonly body_sha256: string | null; + readonly body_encoding: "aws-eventstream" | null; +} + +interface UsageSummaryRow { + readonly input_tokens: number | null; + readonly output_tokens: number | null; + readonly cache_read_tokens: number | null; + readonly cache_write_tokens: number | null; + readonly total_tokens: number | null; +} + +type SqlParam = string | number; + +interface CallCursor { + readonly startedAt: number; + readonly id: string; +} + +interface StreamCursor { + readonly ordinal: number; + readonly id: string; +} + class BunSqliteSpyStore implements SpyStore { private readonly db: Database; private readonly retentionDays: number; @@ -173,6 +359,174 @@ class BunSqliteSpyStore implements SpyStore { return this.withWriteLock(() => this.persistResponseUnlocked(event)); } + listCallSummaries(options: SpyListCallsOptions = {}): SpyPaginatedResult { + const limit = queryLimit(options.limit); + const cursor = options.cursor === undefined ? undefined : decodeCallCursor(options.cursor); + const conditions: string[] = []; + const params: SqlParam[] = []; + + if (options.since !== undefined) { + conditions.push("started_at >= ?"); + params.push(options.since); + } + if (cursor !== undefined) { + conditions.push("(started_at < ? OR (started_at = ? AND id < ?))"); + params.push(cursor.startedAt, cursor.startedAt, cursor.id); + } + + const where = conditions.length === 0 ? "" : `WHERE ${conditions.join(" AND ")}`; + const rows = this.db.query(` +SELECT id, provider, operation, model_id, status, started_at, completed_at, + status_code, request_flow_id, response_flow_id, request_content_hash, + response_content_hash +FROM provider_call +${where} +ORDER BY started_at DESC, id DESC +LIMIT ? +`).all(...params, limit + 1) as ProviderCallRow[]; + + return this.paginatedCallSummaries(rows, limit); + } + + getCallDetail(callId: string): SpyCallDetail | null { + const row = this.callRow(callId); + if (row === null) { + return null; + } + + const summary = this.callSummaryForRow(row); + const httpEvents = this.db.query(` +SELECT id, call_id, direction, observed_at, host, method, path, status_code, + reason, headers_json, request_headers_json, body_text, body_b64, + body_sha256, body_encoding, content_type +FROM http_event +WHERE call_id = ? +ORDER BY observed_at ASC, direction ASC +`).all(callId) as HttpEventRow[]; + + return { + summary, + httpEvents: httpEvents.map(httpEventFromRow), + blocks: this.blocksForCall(callId), + usageRecords: this.usageRecordsForCall(callId), + rawPayloads: this.rawPayloadsForCall(callId), + }; + } + + getCallDiff(callId: string): SpyCallDiff | null { + const row = this.callRow(callId); + if (row === null) { + return null; + } + + const previousRow = this.db.query(` +SELECT id, provider, operation, model_id, status, started_at, completed_at, + status_code, request_flow_id, response_flow_id, request_content_hash, + response_content_hash +FROM provider_call +WHERE provider = ? + AND model_id = ? + AND operation = ? + AND (started_at < ? OR (started_at = ? AND id < ?)) +ORDER BY started_at DESC, id DESC +LIMIT 1 +`).get(row.provider, row.model_id, row.operation, row.started_at, row.started_at, row.id) as ProviderCallRow | null; + + const currentBlocks = this.blocksForCall(callId, "request"); + const previousBlocks = previousRow === null ? [] : this.blocksForCall(previousRow.id, "request"); + const previousByHash = new Map(); + const previousBySignature = new Map(); + for (const block of previousBlocks) { + previousByHash.set(block.content_hash, block); + previousBySignature.set(blockSignature(block), block); + } + + const diffs = currentBlocks.map((block): SpyBlockDiff => { + if (previousRow === null) { + return { block, classification: "unknown" }; + } + const hashMatch = previousByHash.get(block.content_hash); + if (hashMatch !== undefined) { + return { block, classification: "repeated", previousBlockId: hashMatch.id }; + } + const signatureMatch = previousBySignature.get(blockSignature(block)); + if (signatureMatch !== undefined) { + return { block, classification: "changed", previousBlockId: signatureMatch.id }; + } + return { block, classification: "new" }; + }); + + return { + call: this.callSummaryForRow(row), + previousCall: previousRow === null ? null : this.callSummaryForRow(previousRow), + blocks: diffs, + }; + } + + getStreamEvents(callId: string, options: SpyStreamEventsOptions = {}): SpyPaginatedResult { + const limit = queryLimit(options.limit); + const cursor = options.cursor === undefined ? undefined : decodeStreamCursor(options.cursor); + const conditions = ["call_id = ?"]; + const params: SqlParam[] = [callId]; + if (cursor !== undefined) { + conditions.push("(ordinal > ? OR (ordinal = ? AND id > ?))"); + params.push(cursor.ordinal, cursor.ordinal, cursor.id); + } + + const rows = this.db.query(` +SELECT id, call_id, ordinal, event_type, headers_json, payload_json, + payload_text, payload_sha256, observed_at +FROM stream_event +WHERE ${conditions.join(" AND ")} +ORDER BY ordinal ASC, id ASC +LIMIT ? +`).all(...params, limit + 1) as StreamEventRow[]; + + const pageRows = rows.slice(0, limit); + const items = pageRows.map(streamEventFromRow); + if (rows.length <= limit) { + return { items }; + } + const last = pageRows.at(-1); + return last === undefined ? { items } : { items, nextCursor: encodeStreamCursor({ ordinal: last.ordinal, id: last.id }) }; + } + + searchCallSummaries(options: SpySearchCallsOptions): SpyPaginatedResult { + const ftsQuery = ftsQueryForSearch(options.query); + if (ftsQuery === null) { + return { items: [] }; + } + + const limit = queryLimit(options.limit); + const cursor = options.cursor === undefined ? undefined : decodeCallCursor(options.cursor); + const callConditions: string[] = []; + const params: SqlParam[] = [ftsQuery]; + if (cursor !== undefined) { + callConditions.push("(pc.started_at < ? OR (pc.started_at = ? AND pc.id < ?))"); + params.push(cursor.startedAt, cursor.startedAt, cursor.id); + } + const callWhere = callConditions.length === 0 ? "" : `WHERE ${callConditions.join(" AND ")}`; + + const rows = this.db.query(` +WITH matched_call AS ( + SELECT DISTINCT nb.call_id + FROM normalized_block_fts + JOIN normalized_block nb ON nb.id = normalized_block_fts.block_id + WHERE normalized_block_fts MATCH ? +) +SELECT pc.id, pc.provider, pc.operation, pc.model_id, pc.status, pc.started_at, + pc.completed_at, pc.status_code, pc.request_flow_id, pc.response_flow_id, + pc.request_content_hash, pc.response_content_hash +FROM provider_call pc +JOIN matched_call matched ON matched.call_id = pc.id +${callWhere} +ORDER BY pc.started_at DESC, pc.id DESC +LIMIT ? +`).all(...params, limit + 1) as ProviderCallRow[]; + + return this.paginatedCallSummaries(rows, limit); + } + runRetention(): RetentionResult { return this.withWriteLock(() => { let deletedByAge = 0; @@ -527,6 +881,101 @@ INSERT INTO raw_payload ( } } + private paginatedCallSummaries(rows: readonly ProviderCallRow[], limit: number): SpyPaginatedResult { + const pageRows = rows.slice(0, limit); + const items = pageRows.map((row) => this.callSummaryForRow(row)); + if (rows.length <= limit) { + return { items }; + } + const last = pageRows.at(-1); + return last === undefined ? { items } : { + items, + nextCursor: encodeCallCursor({ startedAt: last.started_at, id: last.id }), + }; + } + + private callRow(callId: string): ProviderCallRow | null { + return this.db.query(` +SELECT id, provider, operation, model_id, status, started_at, completed_at, + status_code, request_flow_id, response_flow_id, request_content_hash, + response_content_hash +FROM provider_call +WHERE id = ? +`).get(callId) as ProviderCallRow | null; + } + + private callSummaryForRow(row: ProviderCallRow): SpyCallSummary { + const call = providerCallFromRow(row); + const completedAt = call.completed_at ?? null; + return { + call, + durationMs: completedAt === null ? null : Math.max(0, Math.round((completedAt - call.started_at) * 1000)), + usage: this.usageSummaryForCall(call.id), + requestBlockCount: this.countRows("normalized_block", "call_id = ? AND direction = 'request'", [call.id]), + responseBlockCount: this.countRows("normalized_block", "call_id = ? AND direction = 'response'", [call.id]), + requestByteSize: this.sumRows("normalized_block", "byte_size", "call_id = ? AND direction = 'request'", [call.id]), + responseByteSize: this.sumRows("normalized_block", "byte_size", "call_id = ? AND direction = 'response'", [call.id]), + cacheMarkerCount: this.countRows("normalized_block", "call_id = ? AND cache_marker = 1", [call.id]), + streamEventCount: this.countRows("stream_event", "call_id = ?", [call.id]), + rawPayloadCount: this.countRows("raw_payload", "call_id = ?", [call.id]), + }; + } + + private usageSummaryForCall(callId: string): SpyUsageSummary { + const row = this.db.query(` +SELECT SUM(input_tokens) AS input_tokens, + SUM(output_tokens) AS output_tokens, + SUM(cache_read_tokens) AS cache_read_tokens, + SUM(cache_write_tokens) AS cache_write_tokens, + SUM(total_tokens) AS total_tokens +FROM usage_record +WHERE call_id = ? +`).get(callId) as UsageSummaryRow | null; + + return { + inputTokens: row?.input_tokens ?? null, + outputTokens: row?.output_tokens ?? null, + cacheReadTokens: row?.cache_read_tokens ?? null, + cacheWriteTokens: row?.cache_write_tokens ?? null, + totalTokens: row?.total_tokens ?? null, + }; + } + + private blocksForCall(callId: string, direction?: NormalizedBlock["direction"]): NormalizedBlock[] { + const where = direction === undefined ? "call_id = ?" : "call_id = ? AND direction = ?"; + const params = direction === undefined ? [callId] : [callId, direction]; + const rows = this.db.query(` +SELECT id, call_id, direction, ordinal, role, kind, source, provider_path, + text, json, char_size, byte_size, content_hash, cache_marker +FROM normalized_block +WHERE ${where} +ORDER BY direction ASC, ordinal ASC, id ASC +`).all(...params) as NormalizedBlockRow[]; + return rows.map(normalizedBlockFromRow); + } + + private usageRecordsForCall(callId: string): UsageRecord[] { + const rows = this.db.query(` +SELECT id, call_id, source, input_tokens, output_tokens, cache_read_tokens, + cache_write_tokens, total_tokens, raw_json +FROM usage_record +WHERE call_id = ? +ORDER BY id ASC +`).all(callId) as UsageRecordRow[]; + return rows.map(usageRecordFromRow); + } + + private rawPayloadsForCall(callId: string): RawPayloadRecord[] { + const rows = this.db.query(` +SELECT id, call_id, direction, content_type, body_text, body_b64, body_sha256, + body_encoding +FROM raw_payload +WHERE call_id = ? +ORDER BY direction ASC, id ASC +`).all(callId) as RawPayloadRow[]; + return rows.map(rawPayloadFromRow); + } + private recordMalformedSpoolFile(path: string, error: unknown): void { this.db.transaction(() => { this.incrementCounter("spool_malformed_events", 1); @@ -584,12 +1033,17 @@ ON CONFLICT(key) DO UPDATE SET return true; } - private countRows(table: string, where?: string): number { + private countRows(table: string, where?: string, params: readonly SqlParam[] = []): number { const sql = where === undefined ? `SELECT COUNT(*) AS count FROM ${table}` : `SELECT COUNT(*) AS count FROM ${table} WHERE ${where}`; - const row = this.db.query(sql).get() as CountRow | null; + const row = this.db.query(sql).get(...params) as CountRow | null; return row?.count ?? 0; } + private sumRows(table: string, column: string, where: string, params: readonly SqlParam[]): number { + const row = this.db.query(`SELECT SUM(${column}) AS total FROM ${table} WHERE ${where}`).get(...params) as SumRow | null; + return row?.total ?? 0; + } + private clearGeneration(): number { const row = this.db.query("SELECT value FROM service_metadata WHERE key = 'clear_generation'").get() as { readonly value: string } | null; const parsed = row === null ? 0 : Number.parseInt(row.value, 10); @@ -684,6 +1138,184 @@ export function openSpyStore(options: SpyStoreOptions): SpyStore { return new BunSqliteSpyStore(options); } +function providerCallFromRow(row: ProviderCallRow): ProviderCall { + return ProviderCallSchema.parse({ + id: row.id, + provider: row.provider, + operation: row.operation, + model_id: row.model_id, + status: row.status, + started_at: row.started_at, + ...(row.completed_at === null ? {} : { completed_at: row.completed_at }), + ...(row.status_code === null ? {} : { status_code: row.status_code }), + request_flow_id: row.request_flow_id, + ...(row.response_flow_id === null ? {} : { response_flow_id: row.response_flow_id }), + ...(row.request_content_hash === null ? {} : { request_content_hash: row.request_content_hash }), + ...(row.response_content_hash === null ? {} : { response_content_hash: row.response_content_hash }), + }); +} + +function httpEventFromRow(row: HttpEventRow): HttpEventRecord { + return HttpEventRecordSchema.parse({ + id: row.id, + call_id: row.call_id, + direction: row.direction, + observed_at: row.observed_at, + host: row.host, + method: row.method, + path: row.path, + ...(row.status_code === null ? {} : { status_code: row.status_code }), + ...(row.reason === null ? {} : { reason: row.reason }), + headers: parseJson(row.headers_json), + ...(row.request_headers_json === null ? {} : { request_headers: parseJson(row.request_headers_json) }), + ...(row.body_text === null ? {} : { body_text: row.body_text }), + ...(row.body_b64 === null ? {} : { body_b64: row.body_b64 }), + ...(row.body_sha256 === null ? {} : { body_sha256: row.body_sha256 }), + ...(row.body_encoding === null ? {} : { body_encoding: row.body_encoding }), + ...(row.content_type === null ? {} : { content_type: row.content_type }), + }); +} + +function normalizedBlockFromRow(row: NormalizedBlockRow): NormalizedBlock { + return NormalizedBlockSchema.parse({ + id: row.id, + call_id: row.call_id, + direction: row.direction, + ordinal: row.ordinal, + ...(row.role === null ? {} : { role: row.role }), + kind: row.kind, + source: row.source, + ...(row.provider_path === null ? {} : { provider_path: row.provider_path }), + ...(row.text === null ? {} : { text: row.text }), + ...(row.json === null ? {} : { json: parseJson(row.json) }), + char_size: row.char_size, + byte_size: row.byte_size, + content_hash: row.content_hash, + cache_marker: row.cache_marker === 1, + }); +} + +function usageRecordFromRow(row: UsageRecordRow): UsageRecord { + return UsageRecordSchema.parse({ + id: row.id, + call_id: row.call_id, + source: row.source, + ...(row.input_tokens === null ? {} : { input_tokens: row.input_tokens }), + ...(row.output_tokens === null ? {} : { output_tokens: row.output_tokens }), + ...(row.cache_read_tokens === null ? {} : { cache_read_tokens: row.cache_read_tokens }), + ...(row.cache_write_tokens === null ? {} : { cache_write_tokens: row.cache_write_tokens }), + ...(row.total_tokens === null ? {} : { total_tokens: row.total_tokens }), + ...(row.raw_json === null ? {} : { raw: parseJson(row.raw_json) }), + }); +} + +function streamEventFromRow(row: StreamEventRow): StreamEvent { + return StreamEventSchema.parse({ + id: row.id, + call_id: row.call_id, + ordinal: row.ordinal, + event_type: row.event_type, + headers: parseJson(row.headers_json), + ...(row.payload_json === null ? {} : { payload: parseJson(row.payload_json) }), + ...(row.payload_text === null ? {} : { payload_text: row.payload_text }), + ...(row.payload_sha256 === null ? {} : { payload_sha256: row.payload_sha256 }), + ...(row.observed_at === null ? {} : { observed_at: row.observed_at }), + }); +} + +function rawPayloadFromRow(row: RawPayloadRow): RawPayloadRecord { + return RawPayloadRecordSchema.parse({ + id: row.id, + call_id: row.call_id, + direction: row.direction, + ...(row.content_type === null ? {} : { content_type: row.content_type }), + ...(row.body_text === null ? {} : { body_text: row.body_text }), + ...(row.body_b64 === null ? {} : { body_b64: row.body_b64 }), + ...(row.body_sha256 === null ? {} : { body_sha256: row.body_sha256 }), + ...(row.body_encoding === null ? {} : { body_encoding: row.body_encoding }), + }); +} + +function parseJson(value: string): unknown { + return JSON.parse(value) as unknown; +} + +function queryLimit(value: number | undefined): number { + if (value === undefined || !Number.isFinite(value)) { + return DEFAULT_QUERY_LIMIT; + } + return Math.min(MAX_QUERY_LIMIT, Math.max(1, Math.trunc(value))); +} + +function encodeCallCursor(cursor: CallCursor): string { + return Buffer.from(JSON.stringify(cursor)).toString("base64url"); +} + +function decodeCallCursor(cursor: string): CallCursor { + const value = parseCursor(cursor); + if ( + typeof value === "object" + && value !== null + && "startedAt" in value + && "id" in value + && typeof value.startedAt === "number" + && typeof value.id === "string" + ) { + return { startedAt: value.startedAt, id: value.id }; + } + throw new Error("invalid call cursor"); +} + +function encodeStreamCursor(cursor: StreamCursor): string { + return Buffer.from(JSON.stringify(cursor)).toString("base64url"); +} + +function decodeStreamCursor(cursor: string): StreamCursor { + const value = parseCursor(cursor); + if ( + typeof value === "object" + && value !== null + && "ordinal" in value + && "id" in value + && typeof value.ordinal === "number" + && typeof value.id === "string" + ) { + return { ordinal: value.ordinal, id: value.id }; + } + throw new Error("invalid stream cursor"); +} + +function parseCursor(cursor: string): unknown { + try { + return JSON.parse(Buffer.from(cursor, "base64url").toString("utf8")) as unknown; + } catch { + throw new Error("invalid cursor"); + } +} + +function ftsQueryForSearch(query: string): string | null { + const tokens = query + .trim() + .split(/[^A-Za-z0-9_]+/) + .filter((token) => token.length > 0) + .slice(0, 16); + if (tokens.length === 0) { + return null; + } + return tokens.map((token) => `"${token.replaceAll("\"", "\"\"")}"`).join(" AND "); +} + +function blockSignature(block: NormalizedBlock): string { + return [ + block.direction, + block.ordinal, + block.kind, + block.role ?? "", + block.source, + block.provider_path ?? "", + ].join("\u001f"); +} + function httpEventFromRequest(event: SpoolRequestEvent, callId: string): HttpEventRecord { return { id: `http-${callId}-request`, From 89b0c3605e31682b6addd157cc08595a91065ef3 Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Sat, 23 May 2026 06:06:36 -0400 Subject: [PATCH 07/52] Add browser spy UI --- PLAN.md | 25 +- bun.lock | 103 +- eslint.config.ts | 2 +- package.json | 20 +- src/spy/service.ts | 2 +- src/spy/ui/e2e/spy-ui.playwright.ts | 45 + src/spy/ui/index.html | 12 + src/spy/ui/playwright.config.ts | 31 + src/spy/ui/src/App.tsx | 1168 +++++++++++++++++++++++ src/spy/ui/src/api.test.ts | 16 + src/spy/ui/src/api.ts | 101 ++ src/spy/ui/src/components/ui/badge.tsx | 26 + src/spy/ui/src/components/ui/button.tsx | 43 + src/spy/ui/src/components/ui/input.tsx | 19 + src/spy/ui/src/components/ui/select.tsx | 19 + src/spy/ui/src/format.test.ts | 27 + src/spy/ui/src/format.ts | 144 +++ src/spy/ui/src/lib/utils.ts | 6 + src/spy/ui/src/main.tsx | 15 + src/spy/ui/src/styles.css | 60 ++ src/spy/ui/src/types.ts | 52 + src/spy/ui/test-server.ts | 96 ++ src/spy/ui/tsconfig.json | 21 + src/spy/ui/vite.config.ts | 20 + tsconfig.json | 3 +- 25 files changed, 2069 insertions(+), 7 deletions(-) create mode 100644 src/spy/ui/e2e/spy-ui.playwright.ts create mode 100644 src/spy/ui/index.html create mode 100644 src/spy/ui/playwright.config.ts create mode 100644 src/spy/ui/src/App.tsx create mode 100644 src/spy/ui/src/api.test.ts create mode 100644 src/spy/ui/src/api.ts create mode 100644 src/spy/ui/src/components/ui/badge.tsx create mode 100644 src/spy/ui/src/components/ui/button.tsx create mode 100644 src/spy/ui/src/components/ui/input.tsx create mode 100644 src/spy/ui/src/components/ui/select.tsx create mode 100644 src/spy/ui/src/format.test.ts create mode 100644 src/spy/ui/src/format.ts create mode 100644 src/spy/ui/src/lib/utils.ts create mode 100644 src/spy/ui/src/main.tsx create mode 100644 src/spy/ui/src/styles.css create mode 100644 src/spy/ui/src/types.ts create mode 100644 src/spy/ui/test-server.ts create mode 100644 src/spy/ui/tsconfig.json create mode 100644 src/spy/ui/vite.config.ts diff --git a/PLAN.md b/PLAN.md index 72267b6..0e1a773 100644 --- a/PLAN.md +++ b/PLAN.md @@ -502,6 +502,28 @@ V1 excludes: gating, clear-data confirmation, SSE updates, static serving, and bad input handling. - Verified `bun run typecheck`, `bun run lint`, and `bun run test`. +- Implemented the React desktop spy UI with virtualized timeline and call + inspector: + - Added a Vite + React + TypeScript app under `src/spy/ui` with Tailwind, + local shadcn-style primitives, lucide icons, and TanStack virtualization. + - Added UI package scripts for dev, build, unit tests, and Playwright e2e + tests, plus TSX-aware lint/typecheck wiring and locked frontend + dependencies. + - Built the live-from-now conversation-analysis screen with explicit + historical range controls, search, status/model/block-kind filters, + virtualized provider-call timeline rows, SSE refresh, and call selection. + - Built the call-native inspector with request/response block rendering, + semantic highlighting, composition summaries, provider usage, request diff, + network metadata and headers, on-demand stream event loading, raw payload + availability, health/settings data, and confirmed clear-data. + - Added a fixture-backed UI test server and Playwright coverage for app load, + SSE live updates, call selection, inspector sections, historical loading, + search, stream events on demand, and clear-data confirmation. + - Reduced the service SSE keepalive interval so long-lived browser event + streams stay open under Bun's default idle timeout. + - Verified `bun run typecheck`, `bun run lint`, `bun run test`, + `bun run test:spy-ui:unit`, `bun run build:spy-ui`, and + `bun run test:spy-ui:e2e`. ### V1 @@ -515,7 +537,7 @@ Build the Bedrock/Pi browser spy: - [x] Implement TypeScript Bedrock adapter on top of the captured fixtures. - [x] Implement SQLite persistence, migrations, retention, and clear-data. - [x] Implement TS web service, API, SSE, and static asset serving. -- [ ] Implement React desktop UI with virtualized timeline and call inspector. +- [x] Implement React desktop UI with virtualized timeline and call inspector. - [ ] Wire `rootcell provision`, systemd service config, and `rootcell spy` launcher/tunnel. - [ ] Raise firewall disk/root volume defaults to 64 GiB. @@ -646,6 +668,7 @@ V1 tests: - opens inspector sections - loads historical range - searches + - loads stream events on demand - clears data Fixture strategy: diff --git a/bun.lock b/bun.lock index 39df676..a71fbbd 100644 --- a/bun.lock +++ b/bun.lock @@ -10,14 +10,25 @@ "@aws-sdk/client-secrets-manager": "^3.1050.0", "@aws-sdk/client-sts": "^3.1050.0", "@aws-sdk/credential-providers": "^3.1050.0", + "@tanstack/react-virtual": "^3.13.25", + "clsx": "^2.1.1", + "lucide-react": "^1.16.0", + "react": "^19.2.6", + "react-dom": "^19.2.6", + "tailwind-merge": "^3.6.0", "yargs": "18.0.0", "zod": "^4.4.3", }, "devDependencies": { "@eslint/js": "10.0.1", + "@playwright/test": "^1.60.0", + "@tailwindcss/vite": "^4.3.0", "@types/bun": "1.3.14", "@types/node": "25.7.0", + "@types/react": "^19.2.15", + "@types/react-dom": "^19.2.3", "@types/yargs": "17.0.35", + "@vitejs/plugin-react": "^6.0.2", "eslint": "10.3.0", "jiti": "2.7.0", "typescript": "5.9.3", @@ -135,14 +146,24 @@ "@humanwhocodes/retry": ["@humanwhocodes/retry@0.4.3", "", {}, "sha512-bV0Tgo9K4hfPCek+aMAn81RppFKv2ySDQeMoSZuvTASywNTnVJCArCZE2FWqpvIatKu7VMRLWlR1EazvVhDyhQ=="], + "@jridgewell/gen-mapping": ["@jridgewell/gen-mapping@0.3.13", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.0", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA=="], + + "@jridgewell/remapping": ["@jridgewell/remapping@2.3.5", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ=="], + + "@jridgewell/resolve-uri": ["@jridgewell/resolve-uri@3.1.2", "", {}, "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw=="], + "@jridgewell/sourcemap-codec": ["@jridgewell/sourcemap-codec@1.5.5", "", {}, "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og=="], + "@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + "@napi-rs/wasm-runtime": ["@napi-rs/wasm-runtime@1.1.4", "", { "dependencies": { "@tybys/wasm-util": "^0.10.1" }, "peerDependencies": { "@emnapi/core": "^1.7.1", "@emnapi/runtime": "^1.7.1" } }, "sha512-3NQNNgA1YSlJb/kMH1ildASP9HW7/7kYnRI2szWJaofaS1hWmbGI4H+d3+22aGzXXN9IJ+n+GiFVcGipJP18ow=="], "@nodable/entities": ["@nodable/entities@2.1.0", "", {}, "sha512-nyT7T3nbMyBI/lvr6L5TyWbFJAI9FTgVRakNoBqCD+PmID8DzFrrNdLLtHMwMszOtqZa8PAOV24ZqDnQrhQINA=="], "@oxc-project/types": ["@oxc-project/types@0.130.0", "", {}, "sha512-ibD2usx9JRu7f5pu2tMKMI4cpA4NgXJQoYRP4pQ7Pxmn1l6k/53qWtQWZayhYy3X4QZkt90Ot+mJEaeXouio6Q=="], + "@playwright/test": ["@playwright/test@1.60.0", "", { "dependencies": { "playwright": "1.60.0" }, "bin": { "playwright": "cli.js" } }, "sha512-O71yZIbAh/PxDMNGns37GHBIfrVkEVyn+AXyIa5dOTfb4/xNvRWV+Vv/NMbNCtODB/pO7vLlF2OTmMVLhmr7Ag=="], + "@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.0.1", "", { "os": "android", "cpu": "arm64" }, "sha512-fJI3I0r3C3Oj/zdBCpaCmBRZYf07xpaq4yCfDDoSFm+beWNzbIl26puW8RraUdugoJw/95zerNOn6jasAhzSmg=="], "@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.0.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-cKnAhWEsV7TPcA/5EAteDp6KcJZBQ2G+BqE7zayMMi7kMvwRsbv7WT9aOnn0WNl4SKEIf43vjS31iUPu80nzXg=="], @@ -195,6 +216,40 @@ "@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], + "@tailwindcss/node": ["@tailwindcss/node@4.3.0", "", { "dependencies": { "@jridgewell/remapping": "^2.3.5", "enhanced-resolve": "^5.21.0", "jiti": "^2.6.1", "lightningcss": "1.32.0", "magic-string": "^0.30.21", "source-map-js": "^1.2.1", "tailwindcss": "4.3.0" } }, "sha512-aFb4gUhFOgdh9AXo4IzBEOzBkkAxm9VigwDJnMIYv3lcfXCJVesNfbEaBl4BNgVRyid92AmdviqwBUBRKSeY3g=="], + + "@tailwindcss/oxide": ["@tailwindcss/oxide@4.3.0", "", { "optionalDependencies": { "@tailwindcss/oxide-android-arm64": "4.3.0", "@tailwindcss/oxide-darwin-arm64": "4.3.0", "@tailwindcss/oxide-darwin-x64": "4.3.0", "@tailwindcss/oxide-freebsd-x64": "4.3.0", "@tailwindcss/oxide-linux-arm-gnueabihf": "4.3.0", "@tailwindcss/oxide-linux-arm64-gnu": "4.3.0", "@tailwindcss/oxide-linux-arm64-musl": "4.3.0", "@tailwindcss/oxide-linux-x64-gnu": "4.3.0", "@tailwindcss/oxide-linux-x64-musl": "4.3.0", "@tailwindcss/oxide-wasm32-wasi": "4.3.0", "@tailwindcss/oxide-win32-arm64-msvc": "4.3.0", "@tailwindcss/oxide-win32-x64-msvc": "4.3.0" } }, "sha512-F7HZGBeN9I0/AuuJS5PwcD8xayx5ri5GhjYUDBEVYUkexyA/giwbDNjRVrxSezE3T250OU2K/wp/ltWx3UOefg=="], + + "@tailwindcss/oxide-android-arm64": ["@tailwindcss/oxide-android-arm64@4.3.0", "", { "os": "android", "cpu": "arm64" }, "sha512-TJPiq67tKlLuObP6RkwvVGDoxCMBVtDgKkLfa/uyj7/FyxvQwHS+UOnVrXXgbEsfUaMgiVvC4KbJnRr26ho4Ng=="], + + "@tailwindcss/oxide-darwin-arm64": ["@tailwindcss/oxide-darwin-arm64@4.3.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-oMN/WZRb+SO37BmUElEgeEWuU8E/HXRkiODxJxLe1UTHVXLrdVSgfaJV7pSlhRGMSOiXLuxTIjfsF3wYvz8cgQ=="], + + "@tailwindcss/oxide-darwin-x64": ["@tailwindcss/oxide-darwin-x64@4.3.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-N6CUmu4a6bKVADfw77p+iw6Yd9Q3OBhe0veaDX+QazfuVYlQsHfDgxBrsjQ/IW+zywL8mTrNd0SdJT/zgtvMdA=="], + + "@tailwindcss/oxide-freebsd-x64": ["@tailwindcss/oxide-freebsd-x64@4.3.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-zDL5hBkQdH5C6MpqbK3gQAgP80tsMwSI26vjOzjJtNCMUo0lFgOItzHKBIupOZNQxt3ouPH7RPhvNhiTfCe5CQ=="], + + "@tailwindcss/oxide-linux-arm-gnueabihf": ["@tailwindcss/oxide-linux-arm-gnueabihf@4.3.0", "", { "os": "linux", "cpu": "arm" }, "sha512-R06HdNi7A7OEoMsf6d4tjZ71RCWnZQPHj2mnotSFURjNLdBC+cIgXQ7l81CqeoiQftjf6OOblxXMInMgN2VzMA=="], + + "@tailwindcss/oxide-linux-arm64-gnu": ["@tailwindcss/oxide-linux-arm64-gnu@4.3.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-qTJHELX8jetjhRQHCLilkVLmybpzNQAtaI/gaoVoidn/ufbNDbAo8KlK2J+yPoc8wQxvDxCmh/5lr8nC1+lTbg=="], + + "@tailwindcss/oxide-linux-arm64-musl": ["@tailwindcss/oxide-linux-arm64-musl@4.3.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-Z6sukiQsngnWO+l39X4pPbiWT81IC+PLKF+PHxIlyZbGNb9MODfYlXEVlFvej5BOZInWX01kVyzeLvHsXhfczQ=="], + + "@tailwindcss/oxide-linux-x64-gnu": ["@tailwindcss/oxide-linux-x64-gnu@4.3.0", "", { "os": "linux", "cpu": "x64" }, "sha512-DRNdQRpSGzRGfARVuVkxvM8Q12nh19l4BF/G7zGA1oe+9wcC6saFBHTISrpIcKzhiXtSrlSrluCfvMuledoCTQ=="], + + "@tailwindcss/oxide-linux-x64-musl": ["@tailwindcss/oxide-linux-x64-musl@4.3.0", "", { "os": "linux", "cpu": "x64" }, "sha512-Z0IADbDo8bh6I7h2IQMx601AdXBLfFpEdUotft86evd/8ZPflZe9COPO8Q1vw+pfLWIUo9zN/JGZvwuAJqduqg=="], + + "@tailwindcss/oxide-wasm32-wasi": ["@tailwindcss/oxide-wasm32-wasi@4.3.0", "", { "dependencies": { "@emnapi/core": "^1.10.0", "@emnapi/runtime": "^1.10.0", "@emnapi/wasi-threads": "^1.2.1", "@napi-rs/wasm-runtime": "^1.1.4", "@tybys/wasm-util": "^0.10.1", "tslib": "^2.8.1" }, "cpu": "none" }, "sha512-HNZGOUxEmElksYR7S6sC5jTeNGpobAsy9u7Gu0AskJ8/20FR9GqebUyB+HBcU/ax6BHuiuJi+Oda4B+YX6H1yA=="], + + "@tailwindcss/oxide-win32-arm64-msvc": ["@tailwindcss/oxide-win32-arm64-msvc@4.3.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-Pe+RPVTi1T+qymuuRpcdvwSVZjnll/f7n8gBxMMh3xLTctMDKqpdfGimbMyioqtLhUYZxdJ9wGNhV7MKHvgZsQ=="], + + "@tailwindcss/oxide-win32-x64-msvc": ["@tailwindcss/oxide-win32-x64-msvc@4.3.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Mvrf2kXW/yeW/OTezZlCGOirXRcUuLIBx/5Y12BaPM7wJoryG6dfS/NJL8aBPqtTEx/Vm4T4vKzFUcKDT+TKUA=="], + + "@tailwindcss/vite": ["@tailwindcss/vite@4.3.0", "", { "dependencies": { "@tailwindcss/node": "4.3.0", "@tailwindcss/oxide": "4.3.0", "tailwindcss": "4.3.0" }, "peerDependencies": { "vite": "^5.2.0 || ^6 || ^7 || ^8" } }, "sha512-t6J3OrB5Fc0ExuhohouH0fWUGMYL6PTLhW+E7zIk/pdbnJARZDCwjBznFnkh5ynRnIRSI4YjtTH0t6USjJISrw=="], + + "@tanstack/react-virtual": ["@tanstack/react-virtual@3.13.25", "", { "dependencies": { "@tanstack/virtual-core": "3.15.0" }, "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", "react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-bmNoqMu6gcAW9JGrKVB0Q1tN1i5RONZF8r1fW0bbE4Oyf3DwEGnzzQJ2OW+Ozg1P4s8PyugkHg2ULZoFQN+cqw=="], + + "@tanstack/virtual-core": ["@tanstack/virtual-core@3.15.0", "", {}, "sha512-0AwPGx0I8QxPYjAxShT/+z+ZOe9u8mW5rsXvivCTjRfRmz9a43+3mRyi4wwlyoUqOC56q/jatKa0Bh9M99BEHQ=="], + "@tybys/wasm-util": ["@tybys/wasm-util@0.10.2", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-RoBvJ2X0wuKlWFIjrwffGw1IqZHKQqzIchKaadZZfnNpsAYp2mM0h36JtPCjNDAHGgYez/15uMBpfGwchhiMgg=="], "@types/bun": ["@types/bun@1.3.14", "", { "dependencies": { "bun-types": "1.3.14" } }, "sha512-h1hFqFVcvAvD9j9K7ZW7vd82aSA+rTdznZa+5bwvCwqSB1jmmfLcbIWhOLx1/+boy/xmjgCs/OMUL8hRJSmnPw=="], @@ -211,6 +266,10 @@ "@types/node": ["@types/node@25.7.0", "", { "dependencies": { "undici-types": "~7.21.0" } }, "sha512-z+pdZyxE+RTQE9AcboAZCb4otwcrvgHD+GlBpPgn0emDVt0ohrTMhAwlr2Wd9nZ+nihhYFxO2pThz3C5qSu2Eg=="], + "@types/react": ["@types/react@19.2.15", "", { "dependencies": { "csstype": "^3.2.2" } }, "sha512-eRwcGNHve+E8qtEQSSRl6urh+rFop4v8gm6O8rGv25CodbvFdLjA1vVQ1KkiFE0w0UPOnb8tDiFKL5lp0rtY5Q=="], + + "@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="], + "@types/yargs": ["@types/yargs@17.0.35", "", { "dependencies": { "@types/yargs-parser": "*" } }, "sha512-qUHkeCyQFxMXg79wQfTtfndEC+N9ZZg76HJftDJp+qH2tV7Gj4OJi7l+PiWwJ+pWtW8GwSmqsDj/oymhrTWXjg=="], "@types/yargs-parser": ["@types/yargs-parser@21.0.3", "", {}, "sha512-I4q9QU9MQv4oEOz4tAHJtNz1cwuLxn2F3xcc2iV5WdqLPpUnj30aUuxt1mAxYTG+oe8CZMV/+6rU4S4gRDzqtQ=="], @@ -235,6 +294,8 @@ "@typescript-eslint/visitor-keys": ["@typescript-eslint/visitor-keys@8.59.3", "", { "dependencies": { "@typescript-eslint/types": "8.59.3", "eslint-visitor-keys": "^5.0.0" } }, "sha512-f1UQF7ggd42YiwI5wGrRaPsa+P0CINBlrkLPmGfpq/u/I/oVtecoEIfFR9ag/oa1sLOsRNZ6xehf6qMZhQGBDg=="], + "@vitejs/plugin-react": ["@vitejs/plugin-react@6.0.2", "", { "dependencies": { "@rolldown/pluginutils": "^1.0.0" }, "peerDependencies": { "@rolldown/plugin-babel": "^0.1.7 || ^0.2.0", "babel-plugin-react-compiler": "^1.0.0", "vite": "^8.0.0" }, "optionalPeers": ["@rolldown/plugin-babel", "babel-plugin-react-compiler"] }, "sha512-DlSMqo4WhThw4vB8Mpn0Woe9J+Jfq1geJ61AKW0QEgLzGMNwtIMdxbDUzLxcun8W7NbJO0e2Jg/Nxm3cCSVzzg=="], + "@vitest/expect": ["@vitest/expect@4.1.6", "", { "dependencies": { "@standard-schema/spec": "^1.1.0", "@types/chai": "^5.2.2", "@vitest/spy": "4.1.6", "@vitest/utils": "4.1.6", "chai": "^6.2.2", "tinyrainbow": "^3.1.0" } }, "sha512-7EHDquPthALSV0jhhjgEW8FXaviMx7rSqu8W6oqCoAuOhKov814P99QDV1pxMA3QPv21YudvJngIhjrNI4opLg=="], "@vitest/mocker": ["@vitest/mocker@4.1.6", "", { "dependencies": { "@vitest/spy": "4.1.6", "estree-walker": "^3.0.3", "magic-string": "^0.30.21" }, "peerDependencies": { "msw": "^2.4.9", "vite": "^6.0.0 || ^7.0.0 || ^8.0.0" }, "optionalPeers": ["msw", "vite"] }, "sha512-MCFc63czMjEInOlcY2cpQCvCN+KgbAn+60xu9cMgP4sKaLC5JNAKw7JH8QdAnoAC88hW1IiSNZ+GgVXlN1UcMQ=="], @@ -273,10 +334,14 @@ "cliui": ["cliui@9.0.1", "", { "dependencies": { "string-width": "^7.2.0", "strip-ansi": "^7.1.0", "wrap-ansi": "^9.0.0" } }, "sha512-k7ndgKhwoQveBL+/1tqGJYNz097I7WOvwbmmU2AR5+magtbjPWQTS1C5vzGkBC8Ym8UWRzfKUzUUqFLypY4Q+w=="], + "clsx": ["clsx@2.1.1", "", {}, "sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA=="], + "convert-source-map": ["convert-source-map@2.0.0", "", {}, "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg=="], "cross-spawn": ["cross-spawn@7.0.6", "", { "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0", "which": "^2.0.1" } }, "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA=="], + "csstype": ["csstype@3.2.3", "", {}, "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ=="], + "debug": ["debug@4.4.3", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA=="], "deep-is": ["deep-is@0.1.4", "", {}, "sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ=="], @@ -285,6 +350,8 @@ "emoji-regex": ["emoji-regex@10.6.0", "", {}, "sha512-toUI84YS5YmxW219erniWD0CIVOo46xGKColeNQRgOzDorgBi1v4D71/OFzgD9GO2UGKIv1C3Sp8DAn0+j5w7A=="], + "enhanced-resolve": ["enhanced-resolve@5.22.0", "", { "dependencies": { "graceful-fs": "^4.2.4", "tapable": "^2.3.3" } }, "sha512-xYcDWrpELkFzz9SpZ3PlI6Eu6eD93Yf0WLDRxikGhWJ3MAir2SNZTIVCVZqZ/NUyx8AdMc2gT9C0gPiw18kG+A=="], + "es-module-lexer": ["es-module-lexer@2.1.0", "", {}, "sha512-n27zTYMjYu1aj4MjCWzSP7G9r75utsaoc8m61weK+W8JMBGGQybd43GstCXZ3WNmSFtGT9wi59qQTW6mhTR5LQ=="], "escalade": ["escalade@3.2.0", "", {}, "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA=="], @@ -331,7 +398,7 @@ "flatted": ["flatted@3.4.2", "", {}, "sha512-PjDse7RzhcPkIJwy5t7KPWQSZ9cAbzQXcafsetQoD7sOJRQlGikNbx7yZp2OotDnJyrDcbyRq3Ttb18iYOqkxA=="], - "fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="], + "fsevents": ["fsevents@2.3.2", "", { "os": "darwin" }, "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA=="], "get-caller-file": ["get-caller-file@2.0.5", "", {}, "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg=="], @@ -339,6 +406,8 @@ "glob-parent": ["glob-parent@6.0.2", "", { "dependencies": { "is-glob": "^4.0.3" } }, "sha512-XxwI8EOhVQgWp6iDL+3b0r86f4d6AX6zSU55HfB4ydCEuXLXc5FcYeOu+nnGftS4TEju/11rt4KJPTMgbfmv4A=="], + "graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="], + "ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="], "imurmurhash": ["imurmurhash@0.1.4", "", {}, "sha512-JmXMZ6wuvDmLiHEml9ykzqO6lwFbof0GG4IkcGaENdCRDDmMVnny7s5HsIgHCbaq0w2MyPhDqkhTUgS2LU2PHA=="], @@ -387,6 +456,8 @@ "locate-path": ["locate-path@6.0.0", "", { "dependencies": { "p-locate": "^5.0.0" } }, "sha512-iPZK6eYjbxRu3uB4/WZ3EsEIMJFMqAoopl3R+zuq0UjcAm/MO6KCweDgPfP3elTztoKP3KtnVHxTn2NHBSDVUw=="], + "lucide-react": ["lucide-react@1.16.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-dYwyPzb4MEKpGUmNYk3WKWPnMrHs3FKM+q94kAnJrcDIqqn1hq2xY8scaS2ovsOCM5D51ey2gaRG3PBb1vgoYQ=="], + "magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="], "minimatch": ["minimatch@10.2.5", "", { "dependencies": { "brace-expansion": "^5.0.5" } }, "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg=="], @@ -417,14 +488,24 @@ "picomatch": ["picomatch@4.0.4", "", {}, "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A=="], + "playwright": ["playwright@1.60.0", "", { "dependencies": { "playwright-core": "1.60.0" }, "optionalDependencies": { "fsevents": "2.3.2" }, "bin": { "playwright": "cli.js" } }, "sha512-hheHdokM8cdqCb0lcE3s+zT4t4W+vvjpGxsZlDnikarzx8tSzMebh3UiFtgqwFwnTnjYQcsyMF8ei2mCO/tpeA=="], + + "playwright-core": ["playwright-core@1.60.0", "", { "bin": { "playwright-core": "cli.js" } }, "sha512-9bW6zvX/m0lEbgTKJ6YppOKx8H3VOPBMOCFh2irXFOT4BbHgrx5hPjwJYLT40Lu+4qtD36qKc/Hn56StUW57IA=="], + "postcss": ["postcss@8.5.14", "", { "dependencies": { "nanoid": "^3.3.11", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" } }, "sha512-SoSL4+OSEtR99LHFZQiJLkT59C5B1amGO1NzTwj7TT1qCUgUO6hxOvzkOYxD+vMrXBM3XJIKzokoERdqQq/Zmg=="], "prelude-ls": ["prelude-ls@1.2.1", "", {}, "sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g=="], "punycode": ["punycode@2.3.1", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="], + "react": ["react@19.2.6", "", {}, "sha512-sfWGGfavi0xr8Pg0sVsyHMAOziVYKgPLNrS7ig+ivMNb3wbCBw3KxtflsGBAwD3gYQlE/AEZsTLgToRrSCjb0Q=="], + + "react-dom": ["react-dom@19.2.6", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.6" } }, "sha512-0prMI+hvBbPjsWnxDLxlCGyM8PN6UuWjEUCYmZhO67xIV9Xasa/r/vDnq+Xyq4Lo27g8QSbO5YzARu0D1Sps3g=="], + "rolldown": ["rolldown@1.0.1", "", { "dependencies": { "@oxc-project/types": "=0.130.0", "@rolldown/pluginutils": "^1.0.0" }, "optionalDependencies": { "@rolldown/binding-android-arm64": "1.0.1", "@rolldown/binding-darwin-arm64": "1.0.1", "@rolldown/binding-darwin-x64": "1.0.1", "@rolldown/binding-freebsd-x64": "1.0.1", "@rolldown/binding-linux-arm-gnueabihf": "1.0.1", "@rolldown/binding-linux-arm64-gnu": "1.0.1", "@rolldown/binding-linux-arm64-musl": "1.0.1", "@rolldown/binding-linux-ppc64-gnu": "1.0.1", "@rolldown/binding-linux-s390x-gnu": "1.0.1", "@rolldown/binding-linux-x64-gnu": "1.0.1", "@rolldown/binding-linux-x64-musl": "1.0.1", "@rolldown/binding-openharmony-arm64": "1.0.1", "@rolldown/binding-wasm32-wasi": "1.0.1", "@rolldown/binding-win32-arm64-msvc": "1.0.1", "@rolldown/binding-win32-x64-msvc": "1.0.1" }, "bin": { "rolldown": "bin/cli.mjs" } }, "sha512-X0KQHljNnEkWNqqiz9zJrGunh1B0HgOxLXvnFpCOcadzcy5qohZ3tqMEUg00vncoRovXuK3ZqCT9KnnKzoInFQ=="], + "scheduler": ["scheduler@0.27.0", "", {}, "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q=="], + "semver": ["semver@7.8.0", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-AcM7dV/5ul4EekoQ29Agm5vri8JNqRyj39o0qpX6vDF2GZrtutZl5RwgD1XnZjiTAfncsJhMI48QQH3sN87YNA=="], "shebang-command": ["shebang-command@2.0.0", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="], @@ -445,6 +526,12 @@ "strnum": ["strnum@2.3.0", "", {}, "sha512-ums3KNd42PGyx5xaoVTO1mjU1bH3NpY4vsrVlnv9PNGqQj8wd7rJ6nEypLrJ7z5vxK5RP0yMLo6J/Gsm62DI5Q=="], + "tailwind-merge": ["tailwind-merge@3.6.0", "", {}, "sha512-uxL7qAVQriqRQPAyK3pj66VqskWqoZ37PW94jwOTwNfq/z9oyu1V+eqrZqtR2+fCiXdYOZe/Modt8GtvqNzu+w=="], + + "tailwindcss": ["tailwindcss@4.3.0", "", {}, "sha512-y6nxMGB1nMW9R6k96e5gdIFzcfL/gTJRNaqGes1YvkLnPVXzWgbqFF2yLC0T8G774n24cx3Pe8XrKoniCOAH+Q=="], + + "tapable": ["tapable@2.3.3", "", {}, "sha512-uxc/zpqFg6x7C8vOE7lh6Lbda8eEL9zmVm/PLeTPBRhh1xCgdWaQ+J1CUieGpIfm2HdtsUpRv+HshiasBMcc6A=="], + "tinybench": ["tinybench@2.9.0", "", {}, "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg=="], "tinyexec": ["tinyexec@1.1.2", "", {}, "sha512-dAqSqE/RabpBKI8+h26GfLq6Vb3JVXs30XYQjdMjaj/c2tS8IYYMbIzP599KtRj7c57/wYApb3QjgRgXmrCukA=="], @@ -493,6 +580,20 @@ "@eslint-community/eslint-utils/eslint-visitor-keys": ["eslint-visitor-keys@3.4.3", "", {}, "sha512-wpc+LXeiyiisxPlEkUzU6svyS1frIO3Mgxj1fdy7Pm8Ygzguax2N3Fa/D/ag1WqbOprdI+uY6wMUl8/a2G+iag=="], + "@tailwindcss/oxide-wasm32-wasi/@emnapi/core": ["@emnapi/core@1.10.0", "", { "dependencies": { "@emnapi/wasi-threads": "1.2.1", "tslib": "^2.4.0" }, "bundled": true }, "sha512-yq6OkJ4p82CAfPl0u9mQebQHKPJkY7WrIuk205cTYnYe+k2Z8YBh11FrbRG/H6ihirqcacOgl2BIO8oyMQLeXw=="], + + "@tailwindcss/oxide-wasm32-wasi/@emnapi/runtime": ["@emnapi/runtime@1.10.0", "", { "dependencies": { "tslib": "^2.4.0" }, "bundled": true }, "sha512-ewvYlk86xUoGI0zQRNq/mC+16R1QeDlKQy21Ki3oSYXNgLb45GV1P6A0M+/s6nyCuNDqe5VpaY84BzXGwVbwFA=="], + + "@tailwindcss/oxide-wasm32-wasi/@emnapi/wasi-threads": ["@emnapi/wasi-threads@1.2.1", "", { "dependencies": { "tslib": "^2.4.0" }, "bundled": true }, "sha512-uTII7OYF+/Mes/MrcIOYp5yOtSMLBWSIoLPpcgwipoiKbli6k322tcoFsxoIIxPDqW01SQGAgko4EzZi2BNv2w=="], + + "@tailwindcss/oxide-wasm32-wasi/@napi-rs/wasm-runtime": ["@napi-rs/wasm-runtime@1.1.4", "", { "dependencies": { "@tybys/wasm-util": "^0.10.1" }, "peerDependencies": { "@emnapi/core": "^1.7.1", "@emnapi/runtime": "^1.7.1" }, "bundled": true }, "sha512-3NQNNgA1YSlJb/kMH1ildASP9HW7/7kYnRI2szWJaofaS1hWmbGI4H+d3+22aGzXXN9IJ+n+GiFVcGipJP18ow=="], + + "@tailwindcss/oxide-wasm32-wasi/@tybys/wasm-util": ["@tybys/wasm-util@0.10.2", "", { "dependencies": { "tslib": "^2.4.0" }, "bundled": true }, "sha512-RoBvJ2X0wuKlWFIjrwffGw1IqZHKQqzIchKaadZZfnNpsAYp2mM0h36JtPCjNDAHGgYez/15uMBpfGwchhiMgg=="], + + "@tailwindcss/oxide-wasm32-wasi/tslib": ["tslib@2.8.1", "", { "bundled": true }, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + "@typescript-eslint/eslint-plugin/ignore": ["ignore@7.0.5", "", {}, "sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg=="], + + "vite/fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="], } } diff --git a/eslint.config.ts b/eslint.config.ts index 3b2de9c..7643dfa 100644 --- a/eslint.config.ts +++ b/eslint.config.ts @@ -10,7 +10,7 @@ export default defineConfig( ...tseslint.configs.strictTypeChecked, ...tseslint.configs.stylisticTypeChecked, { - files: ["**/*.ts"], + files: ["**/*.{ts,tsx}"], languageOptions: { parserOptions: { projectService: true, diff --git a/package.json b/package.json index 2975370..0cd3d4e 100644 --- a/package.json +++ b/package.json @@ -3,10 +3,15 @@ "private": true, "type": "module", "scripts": { - "typecheck": "tsc --noEmit", - "lint": "eslint \"src/**/*.{ts,js,mjs,cjs}\" eslint.config.ts vitest.config.ts", + "typecheck": "tsc --noEmit && tsc --noEmit -p src/spy/ui/tsconfig.json", + "lint": "eslint \"src/**/*.{ts,tsx,js,mjs,cjs}\" eslint.config.ts vitest.config.ts", "test": "bun test src/spy --timeout 10000 && vitest --project unit --run", "test:spy": "bun test src/spy --timeout 10000", + "dev:spy-ui": "vite --config src/spy/ui/vite.config.ts --host 127.0.0.1", + "build:spy-ui": "vite build --config src/spy/ui/vite.config.ts", + "test:spy-ui:unit": "bun test src/spy/ui/src --timeout 10000", + "test:spy-ui:e2e": "bun run build:spy-ui && playwright test -c src/spy/ui/playwright.config.ts", + "test:spy-ui": "bun run test:spy-ui:unit && bun run test:spy-ui:e2e", "test:unit:vitest": "vitest --project unit --run", "test:integration": "vitest --project integration --run", "test:integration:lima-smoke": "vitest --project integration --run src/rootcell/integration/providers/macos-lima-user-v2/cli-smoke.integration.test.ts", @@ -16,9 +21,14 @@ }, "devDependencies": { "@eslint/js": "10.0.1", + "@playwright/test": "^1.60.0", + "@tailwindcss/vite": "^4.3.0", "@types/bun": "1.3.14", "@types/node": "25.7.0", + "@types/react": "^19.2.15", + "@types/react-dom": "^19.2.3", "@types/yargs": "17.0.35", + "@vitejs/plugin-react": "^6.0.2", "eslint": "10.3.0", "jiti": "2.7.0", "typescript": "5.9.3", @@ -31,6 +41,12 @@ "@aws-sdk/client-secrets-manager": "^3.1050.0", "@aws-sdk/client-sts": "^3.1050.0", "@aws-sdk/credential-providers": "^3.1050.0", + "@tanstack/react-virtual": "^3.13.25", + "clsx": "^2.1.1", + "lucide-react": "^1.16.0", + "react": "^19.2.6", + "react-dom": "^19.2.6", + "tailwind-merge": "^3.6.0", "yargs": "18.0.0", "zod": "^4.4.3" } diff --git a/src/spy/service.ts b/src/spy/service.ts index 2df2d13..aa42e84 100644 --- a/src/spy/service.ts +++ b/src/spy/service.ts @@ -241,7 +241,7 @@ class SpyHttpService { start: (controller) => { const keepalive = setInterval(() => { this.sendComment(controller, "keepalive"); - }, 15_000); + }, 5_000); const client: SseClient = { id, controller, keepalive }; this.clients.set(id, client); this.send(controller, "hello", { id }); diff --git a/src/spy/ui/e2e/spy-ui.playwright.ts b/src/spy/ui/e2e/spy-ui.playwright.ts new file mode 100644 index 0000000..5d0fdad --- /dev/null +++ b/src/spy/ui/e2e/spy-ui.playwright.ts @@ -0,0 +1,45 @@ +import { expect, test } from "@playwright/test"; + +test.describe.configure({ mode: "serial" }); + +test("loads fixture calls and receives live updates", async ({ page }) => { + await page.goto("/?since=0"); + await expect(page.getByRole("heading", { name: "Rootcell Spy" })).toBeVisible(); + await expect(page.getByTestId("timeline-row")).toHaveCount(4); + await expect(page.getByTestId("timeline-row")).toHaveCount(5); +}); + +test("selects a call and opens inspector sections", async ({ page }) => { + await page.goto("/?since=0"); + await expect(page.getByTestId("timeline-row")).toHaveCount(5); + await page.getByTestId("timeline-row").first().click(); + await expect(page.getByText("Request Blocks", { exact: true })).toBeVisible(); + await expect(page.getByText("Network Metadata", { exact: true })).toBeVisible(); +}); + +test("loads historical ranges and searches normalized text", async ({ page }) => { + await page.goto("/?since=0"); + await page.getByRole("button", { name: "10 min" }).click(); + await expect(page.getByTestId("timeline-row").first()).toBeVisible(); + await page.getByLabel("Search normalized text").fill("Fixture capture"); + await page.getByRole("button", { name: "Search" }).click(); + await expect(page.getByTestId("timeline-row").first()).toBeVisible(); +}); + +test("loads stream events on demand", async ({ page }) => { + await page.goto("/?since=0"); + await expect(page.getByTestId("timeline-row")).toHaveCount(5); + await page.getByTestId("timeline-row").first().click(); + await page.getByText("Stream Events", { exact: true }).click(); + await page.getByRole("button", { name: "Load Stream Events" }).click(); + await expect(page.getByText("messageStart").first()).toBeVisible(); +}); + +test("clears data with confirmation", async ({ page }) => { + await page.goto("/?since=0"); + await expect(page.getByTestId("timeline-row")).toHaveCount(5); + await page.getByLabel("Clear spy data").click(); + await expect(page.getByRole("dialog", { name: "Clear Spy Data" })).toBeVisible(); + await page.getByRole("button", { name: "Clear", exact: true }).click(); + await expect(page.getByText("No provider calls in this range.")).toBeVisible(); +}); diff --git a/src/spy/ui/index.html b/src/spy/ui/index.html new file mode 100644 index 0000000..8535aab --- /dev/null +++ b/src/spy/ui/index.html @@ -0,0 +1,12 @@ + + + + + + Rootcell Spy + + +
+ + + diff --git a/src/spy/ui/playwright.config.ts b/src/spy/ui/playwright.config.ts new file mode 100644 index 0000000..3fd33c2 --- /dev/null +++ b/src/spy/ui/playwright.config.ts @@ -0,0 +1,31 @@ +import { defineConfig, devices } from "@playwright/test"; +import { resolve } from "node:path"; + +const port = 4674; +const uiRoot = import.meta.dirname; +const staticDir = resolve(uiRoot, "../../../dist/spy-ui"); +const testServer = resolve(uiRoot, "test-server.ts"); + +export default defineConfig({ + testDir: "./e2e", + testMatch: ["*.playwright.ts"], + timeout: 30_000, + expect: { + timeout: 10_000, + }, + use: { + ...devices["Desktop Chrome"], + baseURL: `http://127.0.0.1:${String(port)}`, + trace: "retain-on-failure", + }, + webServer: { + command: `bun run ${shellQuote(testServer)} --port ${String(port)} --static ${shellQuote(staticDir)}`, + url: `http://127.0.0.1:${String(port)}/api/health`, + reuseExistingServer: false, + timeout: 15_000, + }, +}); + +function shellQuote(value: string): string { + return `'${value.replaceAll("'", "'\\''")}'`; +} diff --git a/src/spy/ui/src/App.tsx b/src/spy/ui/src/App.tsx new file mode 100644 index 0000000..71483dc --- /dev/null +++ b/src/spy/ui/src/App.tsx @@ -0,0 +1,1168 @@ +import { useVirtualizer } from "@tanstack/react-virtual"; +import { + Activity, + AlertTriangle, + BadgeInfo, + Clock, + Database, + Filter, + Loader2, + RefreshCcw, + Search, + Server, + Trash2, + Wifi, + WifiOff, +} from "lucide-react"; +import * as React from "react"; +import { SpyApiClient, initialSinceFromLocation } from "./api.ts"; +import { Badge } from "./components/ui/badge.tsx"; +import { Button } from "./components/ui/button.tsx"; +import { Input } from "./components/ui/input.tsx"; +import { Select } from "./components/ui/select.tsx"; +import { + blockKindLabel, + blockText, + clipped, + currentSeconds, + formatBytes, + formatDateTime, + formatDuration, + formatNumber, + formatTime, + formatUsageTotal, + secondsForPreset, + shortModelId, + statusTone, + summarizeBlocks, +} from "./format.ts"; +import { cn } from "./lib/utils.ts"; +import type { + DiffClassification, + HttpEventRecord, + NormalizedBlock, + RawPayloadRecord, + SpyCallDetail, + SpyCallDiff, + SpyCallSummary, + SpyServiceHealth, + StreamEvent, + TimePreset, + UiFilters, + UsageRecord, +} from "./types.ts"; + +const api = new SpyApiClient(); +const CALL_LIMIT = 100; +const ALL_FILTER = "all"; + +type LoadState = "idle" | "loading" | "error"; + +interface DetailState { + readonly callId: string; + readonly detail: SpyCallDetail | null; + readonly diff: SpyCallDiff | null; + readonly state: LoadState; + readonly error?: string | undefined; +} + +type LoadedDetailState = DetailState & { + readonly detail: SpyCallDetail; + readonly diff: SpyCallDiff; +}; + +interface StreamState { + readonly callId: string; + readonly items: readonly StreamEvent[]; + readonly nextCursor?: string | undefined; + readonly state: LoadState; + readonly error?: string | undefined; +} + +export function App(): React.ReactElement { + const [preset, setPreset] = React.useState("live"); + const [since, setSince] = React.useState(() => initialSinceFromLocation(window.location)); + const [customStart, setCustomStart] = React.useState(() => datetimeLocalValue(initialSinceFromLocation(window.location))); + const [searchDraft, setSearchDraft] = React.useState(""); + const [search, setSearch] = React.useState(""); + const [filters, setFilters] = React.useState({ model: ALL_FILTER, status: ALL_FILTER, blockKind: ALL_FILTER }); + const [calls, setCalls] = React.useState([]); + const [nextCursor, setNextCursor] = React.useState(); + const [callState, setCallState] = React.useState("idle"); + const [callError, setCallError] = React.useState(); + const [selectedCallId, setSelectedCallId] = React.useState(); + const [detailState, setDetailState] = React.useState(null); + const [streamState, setStreamState] = React.useState(null); + const [health, setHealth] = React.useState(null); + const [sseConnected, setSseConnected] = React.useState(false); + const [clearOpen, setClearOpen] = React.useState(false); + const [clearing, setClearing] = React.useState(false); + + const loadCalls = React.useCallback(async (options: { readonly cursor?: string | undefined; readonly append?: boolean | undefined } = {}) => { + setCallState("loading"); + setCallError(undefined); + try { + const page = await api.calls({ + since, + search, + limit: CALL_LIMIT, + ...(options.cursor === undefined ? {} : { cursor: options.cursor }), + }); + setCalls((current) => options.append === true ? [...current, ...page.items] : page.items); + setNextCursor(page.nextCursor); + setCallState("idle"); + setSelectedCallId((current) => current ?? page.items[0]?.call.id); + } catch (error) { + setCallState("error"); + setCallError(error instanceof Error ? error.message : "failed to load calls"); + } + }, [search, since]); + + React.useEffect(() => { + void loadCalls(); + }, [loadCalls]); + + React.useEffect(() => { + let cancelled = false; + void api.health().then((snapshot) => { + if (!cancelled) { + setHealth(snapshot); + } + }).catch(() => { + if (!cancelled) { + setHealth(null); + } + }); + return () => { + cancelled = true; + }; + }, []); + + React.useEffect(() => { + const source = new EventSource("/api/events"); + const onOpen = (): void => { + setSseConnected(true); + }; + const onError = (): void => { + setSseConnected(false); + }; + const onHealth = (event: MessageEvent): void => { + setHealth(JSON.parse(event.data) as SpyServiceHealth); + setSseConnected(true); + }; + const onCallsChanged = (): void => { + void loadCalls(); + }; + const onCleared = (): void => { + setCalls([]); + setNextCursor(undefined); + setSelectedCallId(undefined); + setDetailState(null); + setStreamState(null); + void api.health().then(setHealth); + }; + + source.addEventListener("open", onOpen); + source.addEventListener("error", onError); + source.addEventListener("health", onHealth as EventListener); + source.addEventListener("calls-changed", onCallsChanged); + source.addEventListener("cleared", onCleared); + + return () => { + source.removeEventListener("open", onOpen); + source.removeEventListener("error", onError); + source.removeEventListener("health", onHealth as EventListener); + source.removeEventListener("calls-changed", onCallsChanged); + source.removeEventListener("cleared", onCleared); + source.close(); + }; + }, [loadCalls]); + + React.useEffect(() => { + if (selectedCallId === undefined) { + return; + } + let cancelled = false; + setDetailState({ callId: selectedCallId, detail: null, diff: null, state: "loading" }); + setStreamState(null); + void Promise.all([ + api.callDetail(selectedCallId), + api.callDiff(selectedCallId), + ]).then(([detail, diff]) => { + if (!cancelled) { + setDetailState({ callId: selectedCallId, detail, diff, state: "idle" }); + } + }).catch((error: unknown) => { + if (!cancelled) { + setDetailState({ + callId: selectedCallId, + detail: null, + diff: null, + state: "error", + error: error instanceof Error ? error.message : "failed to load call detail", + }); + } + }); + return () => { + cancelled = true; + }; + }, [selectedCallId]); + + const filteredCalls = React.useMemo(() => { + return calls.filter((summary) => { + if (filters.status !== ALL_FILTER && summary.call.status !== filters.status) { + return false; + } + if (filters.model !== ALL_FILTER && summary.call.model_id !== filters.model) { + return false; + } + return true; + }); + }, [calls, filters.model, filters.status]); + + const modelOptions = React.useMemo(() => { + return [...new Set(calls.map((summary) => summary.call.model_id))].sort(); + }, [calls]); + + const selectedSummary = React.useMemo(() => { + return calls.find((summary) => summary.call.id === selectedCallId) ?? null; + }, [calls, selectedCallId]); + + function setPresetSince(nextPreset: TimePreset): void { + setPreset(nextPreset); + if (nextPreset === "live") { + const next = currentSeconds(); + setSince(next); + setCustomStart(datetimeLocalValue(next)); + } else if (nextPreset === "10m" || nextPreset === "1h" || nextPreset === "today") { + const next = secondsForPreset(nextPreset); + setSince(next); + setCustomStart(datetimeLocalValue(next)); + } + } + + function applyCustomStart(): void { + const next = secondsFromDatetimeLocal(customStart); + if (next !== null) { + setPreset("custom"); + setSince(next); + } + } + + function submitSearch(event: React.SyntheticEvent): void { + event.preventDefault(); + setSearch(searchDraft); + } + + async function loadMore(): Promise { + if (nextCursor !== undefined) { + await loadCalls({ cursor: nextCursor, append: true }); + } + } + + async function loadStreamEvents(more = false): Promise { + if (selectedCallId === undefined) { + return; + } + const cursor = more ? streamState?.nextCursor : undefined; + setStreamState((current) => ({ + callId: selectedCallId, + items: more ? current?.items ?? [] : [], + state: "loading", + ...(cursor === undefined ? {} : { nextCursor: cursor }), + })); + try { + const page = await api.streamEvents(selectedCallId, cursor); + setStreamState((current) => ({ + callId: selectedCallId, + items: more ? [...(current?.items ?? []), ...page.items] : page.items, + nextCursor: page.nextCursor, + state: "idle", + })); + } catch (error) { + setStreamState({ + callId: selectedCallId, + items: more ? streamState?.items ?? [] : [], + state: "error", + error: error instanceof Error ? error.message : "failed to load stream events", + }); + } + } + + async function clearData(): Promise { + setClearing(true); + try { + await api.clearData(); + setClearOpen(false); + } finally { + setClearing(false); + } + } + + return ( +
+
+
+
+
+
+

Rootcell Spy

+

+ {preset === "live" ? "Live from now" : `Since ${formatDateTime(since)}`} +

+
+
+ +
+ + {sseConnected ? + raw {health?.service.storeRaw === true ? "on" : "off"} + + +
+
+ +
+
+ + {callError === undefined ? null : ( +
+
+ )} + { + void loadMore(); + }} + /> +
+ + { + void loadStreamEvents(false); + }} + onLoadMoreStream={() => { + void loadStreamEvents(true); + }} + /> +
+ + {clearOpen ? ( + { + setClearOpen(false); + }} + onConfirm={() => { + void clearData(); + }} + /> + ) : null} +
+ ); +} + +function TimelineControls(props: { + readonly preset: TimePreset; + readonly customStart: string; + readonly searchDraft: string; + readonly filters: UiFilters; + readonly modelOptions: readonly string[]; + readonly callState: LoadState; + readonly onPreset: (preset: TimePreset) => void; + readonly onCustomStart: (value: string) => void; + readonly onApplyCustomStart: () => void; + readonly onSearchDraft: (value: string) => void; + readonly onSubmitSearch: (event: React.SyntheticEvent) => void; + readonly onFilters: (filters: UiFilters) => void; +}): React.ReactElement { + const { filters } = props; + return ( +
+
+ { + props.onPreset("live"); + }}>Live + { + props.onPreset("10m"); + }}>10 min + { + props.onPreset("1h"); + }}>1 hour + { + props.onPreset("today"); + }}>Today +
+
+
+ +
+
+
+
+ ); +} + +function SegmentButton(props: { + readonly active: boolean; + readonly children: React.ReactNode; + readonly onClick: () => void; +}): React.ReactElement { + return ( + + ); +} + +function Timeline(props: { + readonly calls: readonly SpyCallSummary[]; + readonly selectedCallId: string | undefined; + readonly loading: boolean; + readonly hasMore: boolean; + readonly onSelect: (callId: string) => void; + readonly onLoadMore: () => void; +}): React.ReactElement { + const parentRef = React.useRef(null); + const virtualizer = useVirtualizer({ + count: props.calls.length, + getScrollElement: () => parentRef.current, + estimateSize: () => 118, + overscan: 8, + }); + const virtualItems = virtualizer.getVirtualItems(); + + if (props.calls.length === 0 && !props.loading) { + return ( +
+ No provider calls in this range. +
+ ); + } + + return ( +
+
+ {virtualItems.map((virtualRow) => { + const summary = props.calls[virtualRow.index]; + if (summary === undefined) { + return null; + } + return ( +
+ { + props.onSelect(summary.call.id); + }} + /> +
+ ); + })} +
+
+
+ {formatNumber(props.calls.length)} calls + +
+
+
+ ); +} + +function TimelineRow(props: { + readonly summary: SpyCallSummary; + readonly selected: boolean; + readonly onSelect: () => void; +}): React.ReactElement { + const { summary } = props; + return ( + + ); +} + +function Metric(props: { readonly label: string; readonly value: string }): React.ReactElement { + return ( + + {props.label}{" "} + {props.value} + + ); +} + +function CallInspector(props: { + readonly summary: SpyCallSummary | null; + readonly detailState: DetailState | null; + readonly streamState: StreamState | null; + readonly filters: UiFilters; + readonly health: SpyServiceHealth | null; + readonly onFilters: (filters: UiFilters) => void; + readonly onLoadStream: () => void; + readonly onLoadMoreStream: () => void; +}): React.ReactElement { + const detailState = props.detailState; + let content: React.ReactNode; + if (props.summary === null) { + content = ; + } else if (detailState?.state === "loading") { + content = ; + } else if (detailState?.state === "error") { + content = ; + } else if (!isLoadedDetailState(detailState)) { + content = ; + } else { + content = ( + + ); + } + + return ( + + ); +} + +function isLoadedDetailState(state: DetailState | null): state is LoadedDetailState { + return state?.detail !== undefined && state.detail !== null && state.diff !== null; +} + +function InspectorContent(props: { + readonly detail: SpyCallDetail; + readonly diff: SpyCallDiff; + readonly streamState: StreamState | null; + readonly filters: UiFilters; + readonly health: SpyServiceHealth | null; + readonly onFilters: (filters: UiFilters) => void; + readonly onLoadStream: () => void; + readonly onLoadMoreStream: () => void; +}): React.ReactElement { + const requestBlocks = props.detail.blocks.filter((block) => block.direction === "request"); + const responseBlocks = props.detail.blocks.filter((block) => block.direction === "response"); + const diffByBlockId = React.useMemo(() => { + return new Map(props.diff.blocks.map((entry) => [entry.block.id, entry.classification])); + }, [props.diff.blocks]); + + return ( + <> + + +
+ + +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+ + ); +} + +function SummaryPanel(props: { readonly detail: SpyCallDetail }): React.ReactElement { + const { summary } = props.detail; + return ( +
+
+
+
+ ); +} + +function PanelMetric(props: { + readonly icon: React.ReactNode; + readonly label: string; + readonly value: string; +}): React.ReactElement { + return ( +
+
+ {props.icon} + {props.label} +
+
{props.value}
+
+ ); +} + +function BlockSummaryPanel(props: { + readonly requestBlocks: readonly NormalizedBlock[]; + readonly responseBlocks: readonly NormalizedBlock[]; +}): React.ReactElement { + const summaries = [...summarizeBlocks(props.requestBlocks), ...summarizeBlocks(props.responseBlocks)]; + return ( +
+

Composition

+
+ {summaries.slice(0, 10).map((summary, index) => ( +
+
{blockKindLabel(summary.kind)}
+
{summary.count} blocks · {formatBytes(summary.bytes)}
+
+ ))} +
+
+ ); +} + +function BlockToolbar(props: { + readonly filters: UiFilters; + readonly onFilters: (filters: UiFilters) => void; +}): React.ReactElement { + const kinds: readonly NormalizedBlock["kind"][] = [ + "provider-envelope", + "harness-system-context", + "user-visible-message", + "prior-conversation-history", + "current-user-input", + "assistant-output", + "thinking", + "tool-definition", + "tool-call", + "tool-result", + "cache-marker", + "media-summary", + "unknown", + ]; + return ( +
+ +
+ ); +} + +function BlockList(props: { + readonly blocks: readonly NormalizedBlock[]; + readonly filterKind: string; + readonly diffByBlockId: ReadonlyMap; +}): React.ReactElement { + const blocks = props.filterKind === ALL_FILTER + ? props.blocks + : props.blocks.filter((block) => block.kind === props.filterKind); + if (blocks.length === 0) { + return
No blocks.
; + } + return ( +
+ {blocks.map((block) => ( + + ))} +
+ ); +} + +function BlockRow(props: { + readonly block: NormalizedBlock; + readonly diff: DiffClassification | undefined; +}): React.ReactElement { + const text = blockText(props.block); + return ( +
+
+ {blockKindLabel(props.block.kind)} + {props.block.role === undefined ? null : {props.block.role}} + {props.block.cache_marker ? cache marker : null} + {props.diff === undefined ? null : {props.diff}} + {formatBytes(props.block.byte_size)} +
+ {text.length === 0 ? null : ( + 
+          {clipped(text, 6_000)}
+
+ )} +
{props.block.provider_path ?? props.block.source}
+
+ ); +} + +function DiffPanel(props: { readonly diff: SpyCallDiff }): React.ReactElement { + const previous = props.diff.previousCall; + const counts = props.diff.blocks.reduce>((current, entry) => { + current[entry.classification] += 1; + return current; + }, { new: 0, repeated: 0, changed: 0, unknown: 0 }); + return ( +
+
+ Previous comparable request: {previous === null ? "none" : `${formatDateTime(previous.call.started_at)} · ${previous.call.id}`} +
+
+ {(["new", "changed", "repeated", "unknown"] as const).map((classification) => ( +
+
{classification}
+
{formatNumber(counts[classification])}
+
+ ))} +
+
+ ); +} + +function UsagePanel(props: { readonly records: readonly UsageRecord[] }): React.ReactElement { + if (props.records.length === 0) { + return
No provider usage record.
; + } + return ( +
+ {props.records.map((record) => ( +
+ + + + + +
+ ))} +
+ ); +} + +function UsageCell(props: { readonly label: string; readonly value: number | undefined }): React.ReactElement { + return ( +
+
{props.label}
+
{formatNumber(props.value)}
+
+ ); +} + +function NetworkPanel(props: { readonly events: readonly HttpEventRecord[] }): React.ReactElement { + return ( +
+ {props.events.map((event) => ( +
+
+ {event.direction} + {event.method} {event.path} + {event.status_code ?? ""} {event.reason ?? ""} +
+
{event.host} · {formatDateTime(event.observed_at)}
+ +
+ ))} +
+ ); +} + +function HeaderList(props: { readonly headers: readonly (readonly [string, string])[] }): React.ReactElement { + return ( +
+ {props.headers.map(([name, value]) => ( + + {name} + {value} + + ))} +
+ ); +} + +function StreamPanel(props: { + readonly streamState: StreamState | null; + readonly count: number; + readonly onLoad: () => void; + readonly onLoadMore: () => void; +}): React.ReactElement { + if (props.streamState === null) { + return ( +
+ {formatNumber(props.count)} stream events + +
+ ); + } + if (props.streamState.state === "error") { + return ; + } + return ( +
+ {props.streamState.state === "loading" && props.streamState.items.length === 0 ? : null} + {props.streamState.items.map((event) => ( +
+
+ {event.event_type} + #{formatNumber(event.ordinal)} +
+ 
+            {clipped(JSON.stringify(event.payload ?? event.payload_text ?? event.headers, null, 2), 4_000)}
+
+
+ ))} + +
+ ); +} + +function RawPayloadPanel(props: { + readonly rawPayloads: readonly RawPayloadRecord[]; + readonly rawPayloadCount: number; +}): React.ReactElement { + if (props.rawPayloads.length === 0) { + return
Raw storage disabled or no raw payloads stored for this call.
; + } + return ( +
+ {props.rawPayloads.map((payload) => ( +
+
+ {payload.direction} + {payload.content_type ?? payload.body_encoding ?? "payload"} · {payload.body_sha256 ?? "no hash"} +
+ {payload.body_text === undefined ? ( +
base64 payload · {payload.body_b64 === undefined ? "not available" : `${formatNumber(payload.body_b64.length)} encoded chars`}
+ ) : ( + 
+              {clipped(payload.body_text, 4_000)}
+
+ )} +
+ ))} +
Stored payload records: {formatNumber(props.rawPayloadCount)}
+
+ ); +} + +function HealthPanel(props: { readonly health: SpyServiceHealth | null }): React.ReactElement { + if (props.health === null) { + return
Health unavailable.
; + } + const { service, store } = props.health; + return ( +
+ + + + + + + + + +
+ ); +} + +function HealthCell(props: { readonly label: string; readonly value: string }): React.ReactElement { + return ( +
+
{props.label}
+
{props.value}
+
+ ); +} + +function Section(props: { + readonly title: string; + readonly defaultOpen?: boolean | undefined; + readonly children: React.ReactNode; +}): React.ReactElement { + return ( +
+ {props.title} +
{props.children}
+
+ ); +} + +function EmptyInspector(): React.ReactElement { + return ( +
+ Select a timeline row to inspect the provider call. +
+ ); +} + +function LoadingPanel(props: { readonly label: string }): React.ReactElement { + return ( +
+
+ ); +} + +function ErrorPanel(props: { readonly message: string }): React.ReactElement { + return ( +
+
+ ); +} + +function ClearDialog(props: { + readonly clearing: boolean; + readonly onCancel: () => void; + readonly onConfirm: () => void; +}): React.ReactElement { + return ( +
+
+

Clear Spy Data

+

+ Captured calls and pending spool files will be deleted. Schema metadata is kept. +

+
+ + +
+
+
+ ); +} + +function blockTone(kind: NormalizedBlock["kind"]): "neutral" | "green" | "amber" | "red" | "blue" | "teal" { + if (kind === "current-user-input" || kind === "user-visible-message") { + return "teal"; + } + if (kind === "assistant-output") { + return "green"; + } + if (kind === "thinking" || kind === "cache-marker") { + return "blue"; + } + if (kind === "tool-call" || kind === "tool-result" || kind === "tool-definition") { + return "amber"; + } + return "neutral"; +} + +function diffTone(diff: DiffClassification): "neutral" | "green" | "amber" | "blue" { + if (diff === "new") { + return "green"; + } + if (diff === "changed") { + return "amber"; + } + if (diff === "repeated") { + return "blue"; + } + return "neutral"; +} + +function blockBorderClass(kind: NormalizedBlock["kind"]): string { + if (kind === "current-user-input" || kind === "user-visible-message") { + return "border-l-4 border-l-teal-600"; + } + if (kind === "assistant-output") { + return "border-l-4 border-l-emerald-600"; + } + if (kind === "thinking" || kind === "cache-marker") { + return "border-l-4 border-l-sky-600"; + } + if (kind === "tool-call" || kind === "tool-result" || kind === "tool-definition") { + return "border-l-4 border-l-amber-500"; + } + return "border-stone-200"; +} + +function datetimeLocalValue(seconds: number): string { + const date = new Date(seconds * 1000); + const offsetMs = date.getTimezoneOffset() * 60 * 1000; + return new Date(date.getTime() - offsetMs).toISOString().slice(0, 16); +} + +function secondsFromDatetimeLocal(value: string): number | null { + const parsed = new Date(value); + const ms = parsed.getTime(); + return Number.isFinite(ms) ? Math.floor(ms / 1000) : null; +} diff --git a/src/spy/ui/src/api.test.ts b/src/spy/ui/src/api.test.ts new file mode 100644 index 0000000..528a29e --- /dev/null +++ b/src/spy/ui/src/api.test.ts @@ -0,0 +1,16 @@ +import { describe, expect, test } from "bun:test"; +import { callsUrl, streamEventsUrl } from "./api.ts"; + +describe("spy UI API helpers", () => { + test("builds call list URLs with since and cursors", () => { + expect(callsUrl({ since: 123, cursor: "next", limit: 25 })).toBe("/api/calls?limit=25&cursor=next&since=123"); + }); + + test("uses search endpoint when query text is present", () => { + expect(callsUrl({ since: 123, search: "fixture capture" })).toBe("/api/search?limit=100&q=fixture+capture"); + }); + + test("encodes stream event call ids", () => { + expect(streamEventsUrl("call/one", "cursor:1")).toBe("/api/calls/call%2Fone/stream-events?limit=100&cursor=cursor%3A1"); + }); +}); diff --git a/src/spy/ui/src/api.ts b/src/spy/ui/src/api.ts new file mode 100644 index 0000000..927aedd --- /dev/null +++ b/src/spy/ui/src/api.ts @@ -0,0 +1,101 @@ +import type { + CallQuery, + ClearDataResult, + SpyCallDetail, + SpyCallDiff, + SpyCallSummary, + SpyPaginatedResult, + SpyServiceHealth, + StreamEvent, +} from "./types.ts"; + +const DEFAULT_CALL_LIMIT = 100; +const DEFAULT_STREAM_LIMIT = 100; + +export function initialSinceFromLocation(location: Location, nowSeconds: () => number = currentSeconds): number { + const value = new URLSearchParams(location.search).get("since"); + if (value === null || value.trim().length === 0) { + return nowSeconds(); + } + const parsed = Number(value); + return Number.isFinite(parsed) && parsed >= 0 ? parsed : nowSeconds(); +} + +export function callsUrl(query: CallQuery): string { + const params = new URLSearchParams(); + const search = query.search?.trim(); + if (query.limit !== undefined) { + params.set("limit", String(query.limit)); + } else { + params.set("limit", String(DEFAULT_CALL_LIMIT)); + } + if (query.cursor !== undefined) { + params.set("cursor", query.cursor); + } + if (search !== undefined && search.length > 0) { + params.set("q", search); + return `/api/search?${params.toString()}`; + } + if (query.since !== undefined) { + params.set("since", String(query.since)); + } + return `/api/calls?${params.toString()}`; +} + +export function streamEventsUrl(callId: string, cursor?: string): string { + const params = new URLSearchParams({ limit: String(DEFAULT_STREAM_LIMIT) }); + if (cursor !== undefined) { + params.set("cursor", cursor); + } + return `/api/calls/${encodeURIComponent(callId)}/stream-events?${params.toString()}`; +} + +export async function fetchJson(url: string, init?: RequestInit): Promise { + const headers = new Headers(init?.headers); + headers.set("Accept", "application/json"); + const response = await fetch(url, { + ...init, + headers, + }); + if (!response.ok) { + const detail = await response.text(); + throw new Error(`${String(response.status)} ${response.statusText}${detail.length > 0 ? `: ${detail}` : ""}`); + } + return await response.json() as T; +} + +export class SpyApiClient { + health(): Promise { + return fetchJson("/api/health"); + } + + calls(query: CallQuery): Promise> { + return fetchJson>(callsUrl(query)); + } + + callDetail(callId: string): Promise { + return fetchJson(`/api/calls/${encodeURIComponent(callId)}`); + } + + callDiff(callId: string): Promise { + return fetchJson(`/api/calls/${encodeURIComponent(callId)}/diff`); + } + + streamEvents(callId: string, cursor?: string): Promise> { + return fetchJson>(streamEventsUrl(callId, cursor)); + } + + clearData(): Promise { + return fetchJson("/api/clear", { + method: "POST", + body: JSON.stringify({ confirm: true }), + headers: { + "Content-Type": "application/json", + }, + }); + } +} + +function currentSeconds(): number { + return Math.floor(Date.now() / 1000); +} diff --git a/src/spy/ui/src/components/ui/badge.tsx b/src/spy/ui/src/components/ui/badge.tsx new file mode 100644 index 0000000..8440263 --- /dev/null +++ b/src/spy/ui/src/components/ui/badge.tsx @@ -0,0 +1,26 @@ +import { cn } from "../../lib/utils.ts"; +import type { ReactElement, ReactNode } from "react"; + +type BadgeTone = "neutral" | "green" | "amber" | "red" | "blue" | "teal"; + +const toneClass: Record = { + neutral: "border-stone-300 bg-stone-50 text-stone-700", + green: "border-emerald-200 bg-emerald-50 text-emerald-800", + amber: "border-amber-200 bg-amber-50 text-amber-800", + red: "border-red-200 bg-red-50 text-red-800", + blue: "border-sky-200 bg-sky-50 text-sky-800", + teal: "border-teal-200 bg-teal-50 text-teal-800", +}; + +export function Badge(props: { + readonly children: ReactNode; + readonly tone?: BadgeTone | undefined; + readonly className?: string | undefined; +}): ReactElement { + const { children, tone = "neutral", className } = props; + return ( + + {children} + + ); +} diff --git a/src/spy/ui/src/components/ui/button.tsx b/src/spy/ui/src/components/ui/button.tsx new file mode 100644 index 0000000..ed6a1b6 --- /dev/null +++ b/src/spy/ui/src/components/ui/button.tsx @@ -0,0 +1,43 @@ +import * as React from "react"; +import { cn } from "../../lib/utils.ts"; + +type ButtonVariant = "primary" | "secondary" | "ghost" | "danger"; +type ButtonSize = "sm" | "md" | "icon"; + +const variantClass: Record = { + primary: "border-emerald-700 bg-emerald-700 text-white hover:bg-emerald-800", + secondary: "border-stone-300 bg-white text-stone-900 hover:bg-stone-100", + ghost: "border-transparent bg-transparent text-stone-700 hover:bg-stone-100", + danger: "border-red-700 bg-red-700 text-white hover:bg-red-800", +}; + +const sizeClass: Record = { + sm: "h-8 gap-1.5 px-2.5 text-xs", + md: "h-9 gap-2 px-3 text-sm", + icon: "h-8 w-8 p-0", +}; + +export interface ButtonProps extends React.ButtonHTMLAttributes { + readonly variant?: ButtonVariant | undefined; + readonly size?: ButtonSize | undefined; +} + +export const Button = React.forwardRef(function Button( + { className, variant = "secondary", size = "md", type = "button", ...props }, + ref, +) { + return ( + @@ -857,9 +856,9 @@ function TimelineRow(props: { function Metric(props: { readonly label: string; readonly value: string }): React.ReactElement { return ( - - {props.label}{" "} - {props.value} + + {props.label} + {props.value} ); } From 49341bc657d356e42ef26611e90b30d7bba93291 Mon Sep 17 00:00:00 2001 From: Jim Pudar Date: Sat, 23 May 2026 21:28:53 -0400 Subject: [PATCH 34/52] Fix Bedrock reasoning classification --- PLAN.md | 10 +- docs/bugfix/SPY-QA-19-RCA.md | 240 +++++++++++++++++++++++++++++++++++ src/spy/bedrock.test.ts | 228 +++++++++++++++++++++++++++++++++ src/spy/bedrock.ts | 61 ++++++++- 4 files changed, 528 insertions(+), 11 deletions(-) create mode 100644 docs/bugfix/SPY-QA-19-RCA.md diff --git a/PLAN.md b/PLAN.md index bc31a74..5346d4b 100644 --- a/PLAN.md +++ b/PLAN.md @@ -952,9 +952,14 @@ P3 is polish, minor copy, or secondary accessibility. ambiguous request cache-marker count badge, and move byte sizes/duration into the row metadata line. Added Playwright coverage for a cache-heavy call proving the row no longer shows total `tok` usage or `cache 2`. -- [ ] [P1] SPY-QA-19: Fix Bedrock reasoning classification. Prior-history +- [x] [P1] SPY-QA-19: Fix Bedrock reasoning classification. Prior-history `reasoningContent` and signature-only reasoning chunks show as `Unknown` instead of thinking/reasoning metadata. + - Fixed on 2026-05-24: the Bedrock adapter now classifies request + `reasoningContent` and response reasoning deltas as `thinking`, including + nested `reasoningText.text` and signature-only reasoning metadata. Added + adapter coverage for prior-history reasoning, nested response reasoning + text, and signature-only response reasoning chunks. - [ ] [P1] SPY-QA-20: Fix pending-row formatting. Pending rows can render `usage usage n/a`. - [ ] [P1] SPY-QA-21: Fix modal focus management for Clear spy data. Focus stays @@ -1008,9 +1013,6 @@ ID-keyed evidence notes: These notes are retained only where they clarify an open bug ID. They are not separate tasks. -- SPY-QA-19: Prior request history containing Bedrock `reasoningContent` is - classified as `unknown` in request blocks. Response signature-only reasoning - chunks also surface as `Unknown` when they contain metadata but no text. - SPY-QA-20: Pending timeline rows can render `usage usage n/a`. - SPY-QA-21: The clear-data confirmation dialog does not take keyboard focus when it opens, background focus is not effectively inert, and Escape did not diff --git a/docs/bugfix/SPY-QA-19-RCA.md b/docs/bugfix/SPY-QA-19-RCA.md new file mode 100644 index 0000000..953f4c2 --- /dev/null +++ b/docs/bugfix/SPY-QA-19-RCA.md @@ -0,0 +1,240 @@ +# SPY-QA-19 RCA: Bedrock Reasoning Content Is Classified As Unknown + +## Scope + +This RCA covers the highest-priority open spy bug in `PLAN.md`: `SPY-QA-19`. + +Triage notes: + +- `PLAN.md` marks all P0 spy bugs closed. +- `PLAN.md` marks `SPY-QA-01` through `SPY-QA-18` complete or closed. +- `SPY-QA-19` is the first unchecked item in the prioritized handoff, and it is + a P1 issue. +- This document was written before any product-code fix for `SPY-QA-19`. + +## Bug Definition + +`PLAN.md:955-957` defines the current bug: + +```text +[P1] SPY-QA-19: Fix Bedrock reasoning classification. Prior-history +`reasoningContent` and signature-only reasoning chunks show as `Unknown` +instead of thinking/reasoning metadata. +``` + +## Reproduction Used + +I used the current Bedrock normalizer directly with two synthetic but schema-shaped +Bedrock Converse captures: + +- A request whose prior assistant history contains + `content[].reasoningContent.reasoningText.text` plus a signature. +- A response AWS event-stream whose `contentBlockDelta.delta.reasoningContent` + contains only `reasoningText.signature`. + +This isolates the adapter behavior without relying on browser rendering. + +Command used: + +```sh +bun --eval '