feat: consume SSE event stream — frontend implementation

[vredchenko]

Summary

Implement frontend infrastructure to consume the backend SSE event stream (GET /frontend/events/stream) and display live updates in the SmartEM dashboard.

Depends on: DiamondLightSource/smartem-decisions#247 (backend PR, closes #246)
Related: #65 (parent issue — scope and filtering rationale)

Why the SSE endpoint needs a hand-written hook

The API client is auto-generated from the backend OpenAPI spec via Orval (React Query + Axios). Axios is request-response only — it doesn't support the EventSource streaming protocol. The generated hook for the SSE endpoint would make a regular GET that either hangs or returns garbage. A custom useEventStream hook using the browser's native EventSource API is required.

Implementation steps

Step 1: Regenerate API client

npm run api:fetch:local   # Fetch OpenAPI spec from local backend (must be running with new endpoints)
npm run api:generate      # Regenerate hooks, types, MSW mocks

Picks up: AgentLog model, AgentLogBatchRequest/AgentLogBatchResponse types, updated Micrograph with updated_at.

Step 2: Create event type definitions

Create packages/api/src/events.ts — TypeScript types mirroring the backend FrontendEventType enum and event data payloads. Hand-written (not generated) because SSE event payloads aren't modelled as response schemas in OpenAPI.

Step 3: Create useEventStream hook

Create packages/api/src/useEventStream.ts:

function useEventStream(options: {
  acquisitionUuid?: string
  agentId?: string
  eventTypes?: FrontendEventType[]
  enabled?: boolean
  onAgentStatus?: (data: AgentStatusEvent) => void
  onAcquisitionProgress?: (data: AcquisitionProgressEvent) => void
  onInstructionLifecycle?: (data: InstructionLifecycleEvent) => void
  onProcessingMetric?: (data: ProcessingMetricEvent) => void
  onAgentLog?: (data: AgentLogEvent) => void
}): {
  connectionState: 'connecting' | 'open' | 'closed'
  lastEventId: string | null
}

• Browser EventSource API with query param filtering
• Automatic reconnection via Last-Event-ID
• Connection state tracking
• Cleanup on unmount

Step 4: React Query cache integration

Wrapper hook useSmartEMEventStream combining useEventStream with useQueryClient:

• onAcquisitionProgress → invalidate acquisition detail queries
• onProcessingMetric → invalidate micrograph queries
• Or: directly update cache via queryClient.setQueryData for lower latency

Step 5: UI components

• Connection indicator — SSE connection state badge in header
• Agent status badges — online/offline/stale per agent
• Live acquisition counters — grid/gridsquare/foilhole/micrograph counts
• Log viewer panel — scrolling log view with level/agent filtering
• Processing metrics — motion correction / CTF stats updating in real time

Step 6: Export from packages/api

Export useEventStream, event types, and useSmartEMEventStream from packages/api/src/index.ts.

Key files

File	Action
packages/api/src/openapi.json	Regenerate
packages/api/src/generated/	Regenerate
packages/api/src/events.ts	Create
packages/api/src/useEventStream.ts	Create
packages/api/src/useSmartEMEventStream.ts	Create
packages/api/src/index.ts	Modify
apps/smartem/src/components/...	Create — UI per event type

Documentation updates needed

The following docs in DiamondLightSource/smartem-devtools (see PR #160) should be updated when this work is implemented:

• docs/decision-records/smartem-frontend-design.md — add section on real-time data architecture (SSE transport, EventSource hook, cache invalidation strategy)
• docs/decision-records/smartem-frontend-requirements.md — add live dashboard requirements (agent status, acquisition progress, log viewer, processing metrics)
• docs/architecture/index.md — update data flow to show SSE path from backend to frontend
• Consider an ADR for the SSE transport choice (why SSE over WebSocket, why hand-written hook over generated client)

Future design considerations

These are out of scope for this issue but should be kept in mind and refined:

• API surface segmentation — not all consumers need all of the API. Consider whether the backend API should be split into separate parts (e.g. agent API vs frontend API vs admin API). The SSE endpoint is frontend-only; log ingestion is agent-only.
• Proxy to pato backend — some API surface may need to proxy to the pato backend rather than serving directly. This affects how the frontend API client is structured.
• Depositions API separation — if ARIA deposition functionality gets API endpoints, these may need a separate API surface or at minimum a distinct OpenAPI tag/router to keep concerns clean.
• User types and permissions — different user roles (operators, PIs, admins) will need different access levels. The SSE stream and its filters should respect permissions (e.g. an operator sees only their microscope's events). Auth is placeholder-only currently but this needs to be designed before production.

Verification

1. Regenerate client, verify new types in generated/models/
2. Connect to local backend with SSE: browser console shows events arriving
3. UI components display live data when agent + EPUPlayer are running
4. Connection indicator reflects actual SSE state
5. Log viewer shows agent logs in real time with level filtering

Related issues and PRs

• Parent issue: feat: consume SSE event stream for live dashboard updates #65
• Backend issue: feat: SSE event stream for frontend + agent log shipping smartem-decisions#246
• Backend PR: feat: SSE event stream for frontend + agent log shipping smartem-decisions#247
• Frontend design docs: Frontend redesign documentation smartem-devtools#160