Skip to content

Live-render Slack-triggered turns on an open session tab#239

Open
ishaan-berri wants to merge 2 commits into
mainfrom
litellm_slack-ui-live-stream
Open

Live-render Slack-triggered turns on an open session tab#239
ishaan-berri wants to merge 2 commits into
mainfrom
litellm_slack-ui-live-stream

Conversation

@ishaan-berri
Copy link
Copy Markdown
Contributor

What

When you're sitting on a session page and a turn fires from Slack (or Linear), the assistant reply now streams in live instead of only showing up after you navigate away and back.

Why it was broken

Two data paths feed the session view:

  • durable thread (GET /messages) — pulled once on mount and after a local send
  • live SDK streamclaude_sdk_message frames over SSE, rendered via liveTurns

The SDK stream carries the assistant reply but not the prompt that started the turn — a Slack prompt only lands in the durable thread. The liveTurns render gate requires the last durable row to be a user message, so when the previous turn ended in an assistant message the gate stayed shut and nothing painted. Navigating re-ran refreshThread, which is why it "only rendered when you went to it."

Fix

Bracket externally-triggered turns at their boundaries:

  • turn start (first live frame) → pull the prompt into the durable thread once, so the gate opens and the assistant streams under it. Skipped while a local send is in flight (that path owns its own refresh).
  • turn end (session.idle) → finalize by pulling the committed assistant turn so it persists across navigation.

The in-flight assistant still renders from the live stream — we never re-pull the session to render streaming tokens.

useSdkMessageStream now exposes a live-only liveFrameCount and re-enters streaming on each new turn so the view can detect a fresh turn per-turn.

Test

  • Open a ready session, @mention the agent in Slack, watch the reply stream into the open tab without navigating
  • Send a normal message in the UI — unchanged
  • Navigate away and back mid-turn — thread restores, live stream resumes

…turns

The SDK stream hook only flipped status to streaming once at connect and
never reported per-turn frame activity. Add a live-only frame counter and
re-enter streaming whenever a new claude_sdk_message arrives after a prior
session.idle, so the session view can detect a fresh turn (e.g. a Slack
@mention) that streams in while the tab is open.
A Slack/Linear turn streams its assistant reply over the SDK bus, but not
the prompt that started it — that only lives in the durable harness thread.
The liveTurns render gate requires the last durable row to be a user
message, so when the prior turn ended in an assistant message the gate
stayed shut and nothing painted until the user navigated (re-running
refreshThread).

Bracket the turn at its boundaries: on the first live frame pull the prompt
into the durable thread (skipped while a local send is in flight, which owns
its own refresh), and on session.idle finalize by pulling the committed
assistant turn. The in-flight assistant still renders from the live stream;
we never re-pull the session to render streaming tokens.
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 20, 2026

Greptile Summary

This PR fixes live rendering of externally-triggered (Slack/Linear) assistant turns by pulling the missing prompt into the durable thread at turn boundaries and re-entering streaming status per turn. useSdkMessageStream gains a cumulative liveFrameCount and status re-entry; two new useEffect hooks in the session view bracket each turn's start and end.

  • sdk-stream.tsx: adds liveFrameCount (live-only, cumulative) and setStatus(\"streaming\") on every claude_sdk_message frame so the view sees a fresh streaming state per turn.
  • view.tsx: a turn-start effect pulls the prompt once on the first live frame (skipped during local sends); a turn-end effect finalises the committed assistant turn on session.idle and re-arms the prompt-sync ref for the next turn.

Confidence Score: 3/5

The core streaming fix works for the pure-Slack scenario, but a timing race in the two new effects can silently disable live rendering for any Slack turn that follows a local UI send.

The turn-start effect uses liveFrameCount, a cumulative counter that accumulates frames from all turns — local sends included. After a local send, session.idle arrives before listSessionMessages resolves (the common case), causing the turn-end effect to reset the boolean ref first; then, when hasInProgress drops to false, the turn-start effect fires again, sets promptSyncedRef.current = true, and leaves the ref armed. The next Slack or Linear turn then skips its prompt sync, the liveTurns render gate stays closed, and the streaming reply is invisible until the turn ends — exactly the regression this PR was written to prevent.

src/app/sessions/[sid]/view.tsx — the two new useEffect hooks interact in a way that breaks the intended behavior after any local send.

Important Files Changed

Filename Overview
src/app/sessions/[sid]/sdk-stream.tsx Adds liveFrameCount (cumulative live-frame counter) and re-enters streaming status on each new frame so the view can detect per-turn starts. Logic is straightforward; the exposed counter is cumulative by design and works correctly in isolation.
src/app/sessions/[sid]/view.tsx Adds two useEffect hooks to bracket externally-triggered turns. A race between the SSE session.idle delivery and the queue drain's refreshThread() resolution can leave promptSyncedRef.current set to true after every local send, causing the next Slack/Linear turn to skip its prompt sync and miss live rendering — the original bug this PR was fixing.

Reviews (1): Last reviewed commit: "fix(sessions): paint externally-triggere..." | Re-trigger Greptile

Comment on lines +522 to +537
useEffect(() => {
if (liveFrameCount === 0) return;
if (promptSyncedRef.current) return;
if (hasInProgress) return;
promptSyncedRef.current = true;
void refreshThread();
}, [liveFrameCount, hasInProgress, refreshThread]);

// Turn end: when the agent loop settles, pull the now-committed assistant
// turn so it persists in the durable thread (and survives navigation), then
// re-arm prompt-sync for the next turn.
useEffect(() => {
if (sdkStatus !== "completed") return;
promptSyncedRef.current = false;
void refreshThread();
}, [sdkStatus, refreshThread]);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 promptSyncedRef left armed after a local UI send, blocking the next Slack turn

After a local send, the queue drain calls refreshThread(). Because session.idle arrives on the SSE bus before that HTTP fetch resolves, the turn-end effect fires first and resets promptSyncedRef.current = false. When refreshThread() eventually resolves, setMessages runs, hasInProgress drops to false, and the turn-start effect re-fires with liveFrameCount > 0 (cumulative, includes frames from the local send) and promptSyncedRef.current = false — so it sets promptSyncedRef.current = true and calls an extra refreshThread(). That leaves the ref armed (true). The next externally-triggered turn then arrives, liveFrameCount increments, the turn-start effect fires, hits promptSyncedRef.current === true, and returns early — the Slack prompt is never pulled into the durable thread, the liveTurns render gate stays closed, and the streaming reply doesn't paint. The exact regression this PR set out to fix reappears after any local UI send.

The underlying issue is that liveFrameCount is cumulative across all turns (local and external), so the > 0 check is true whenever any previous turn produced frames, not only when a new external turn just started. Replacing the boolean with a watermark (promptSyncedAtRef = useRef(0)) and checking liveFrameCount > promptSyncedAtRef.current would make the guard per-increment rather than per-boolean-flip.

Comment on lines +533 to +537
useEffect(() => {
if (sdkStatus !== "completed") return;
promptSyncedRef.current = false;
void refreshThread();
}, [sdkStatus, refreshThread]);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Duplicate refreshThread() after every local UI send

The turn-end effect fires on every sdkStatus === "completed" transition, regardless of whether the turn was triggered from Slack or from the local composer. For local sends the queue drain already calls refreshThread() after sendMessageStream resolves, so this adds a second concurrent HTTP call to listSessionMessages on every UI-initiated turn. The calls are effectively idempotent, but whichever resolves last wins the setMessages race and could transiently overwrite the latency-stamped assistant row with one that has no latency_ms field.

Consider guarding with !hasInProgress (analogous to the turn-start guard) to limit the turn-end refresh to turns that were not locally originated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant