feat(react-dogfood): VisionAgents integration#2222
Merged
Conversation
…eader Adds a "Vision Agent" pill in the active call header that POSTs to api.demo.visionagents.ai to invite a vision agent into the current call. Disabled while in flight; logs failures to the console.
|
Contributor
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds Vision Agent UI to the React dogfood app: a call-aware AskAIAgentButton that invites/removes vision agents via an external API, a dismissible AIAgentStatusPanel derived from call participants, wiring into ActiveCall, new styles, and translations. ChangesAI Vision Agent Integration
Sequence DiagramsequenceDiagram
participant User
participant AskAIAgentButton
participant CallState as Call State
participant VisionAPI as Vision Agents API
participant ActiveCall
participant AIAgentStatusPanel
participant UI
User->>AskAIAgentButton: Click
AskAIAgentButton->>CallState: Read current call & participants
alt Agent not present
AskAIAgentButton->>VisionAPI: POST /calls/{id}/sessions
VisionAPI-->>AskAIAgentButton: { session_id }
AskAIAgentButton->>ActiveCall: onSessionCreated(session_id)
else Agent present
AskAIAgentButton->>VisionAPI: DELETE /sessions/{sessionId}
Note right of AskAIAgentButton: on failure, call.kickUser fallback
AskAIAgentButton->>ActiveCall: onSessionCleared()
end
ActiveCall->>AIAgentStatusPanel: update (agent presence)
AIAgentStatusPanel->>UI: render status or null
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
sample-apps/react/react-dogfood/components/ActiveCallHeader.tsx (1)
114-145:⚠️ Potential issue | 🟠 MajorGate the Vision Agent invite behind the demo flag.
The header already knows whether it is in the demo environment, but this control renders unconditionally. Since it posts to the demo Vision Agents service, please hide it when
isDemois false so non-demo calls do not expose a third-party invite flow or leak call metadata.Suggested change
- <InviteVisionAgentButton /> + {isDemo && <InviteVisionAgentButton />}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sample-apps/react/react-dogfood/components/ActiveCallHeader.tsx` around lines 114 - 145, The InviteVisionAgentButton is rendered unconditionally but should only appear in demo environments; update the ActiveCallHeader render so InviteVisionAgentButton is only rendered when isDemo is true (e.g., wrap the InviteVisionAgentButton render with an isDemo check), keeping the rest of the controls (RecordingIndicator, ParticipantCountIndicator, Elapsed, LatencyIndicator) unchanged and preserving the existing controls group structure.
🧹 Nitpick comments (1)
sample-apps/react/react-dogfood/components/InviteVisionAgentButton.tsx (1)
1-44: Abort the invite request on unmount.This POST can outlive the component, which leaves the request running and can still hit
setIsInviting(false)after unmount. Please wire the fetch through anAbortControllerand abort it in cleanup.As per coding guidelines, Use AbortController to cancel network requests and media operations on component unmount.Suggested change
-import { useState } from 'react'; +import { useEffect, useRef, useState } from 'react'; import { Icon, useCall, useI18n } from '@stream-io/video-react-sdk'; export const InviteVisionAgentButton = () => { const call = useCall(); const { t } = useI18n(); const [isInviting, setIsInviting] = useState(false); + const abortControllerRef = useRef<AbortController | null>(null); + + useEffect(() => () => abortControllerRef.current?.abort(), []); const onClick = async () => { if (!call?.id || isInviting) return; setIsInviting(true); + const abortController = new AbortController(); + abortControllerRef.current = abortController; try { const response = await fetch( `https://api.demo.visionagents.ai/calls/${call.id}/sessions`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, + signal: abortController.signal, body: JSON.stringify({ call_type: call.type ?? 'default' }), }, ); @@ } catch (err) { + if ((err as DOMException).name === 'AbortError') return; console.error('Failed to invite vision agent', err); } finally { + if (abortControllerRef.current === abortController) { + abortControllerRef.current = null; + } setIsInviting(false); } };🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sample-apps/react/react-dogfood/components/InviteVisionAgentButton.tsx` around lines 1 - 44, The POST in InviteVisionAgentButton's onClick can continue after unmount and still call setIsInviting; fix this by wiring the fetch through an AbortController: create an AbortController for the request (store it in a ref or closure), pass controller.signal into fetch inside onClick, and add a useEffect cleanup that calls controller.abort() on unmount (or abort any stored controller refs). In the try/catch/finally around fetch, check for abort (e.g., err.name === 'AbortError') and avoid logging it as an error, and ensure you only call setIsInviting(false) when the request was not aborted (or the component is still mounted) so setIsInviting is not invoked after unmount.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@sample-apps/react/react-dogfood/components/ActiveCallHeader.tsx`:
- Around line 114-145: The InviteVisionAgentButton is rendered unconditionally
but should only appear in demo environments; update the ActiveCallHeader render
so InviteVisionAgentButton is only rendered when isDemo is true (e.g., wrap the
InviteVisionAgentButton render with an isDemo check), keeping the rest of the
controls (RecordingIndicator, ParticipantCountIndicator, Elapsed,
LatencyIndicator) unchanged and preserving the existing controls group
structure.
---
Nitpick comments:
In `@sample-apps/react/react-dogfood/components/InviteVisionAgentButton.tsx`:
- Around line 1-44: The POST in InviteVisionAgentButton's onClick can continue
after unmount and still call setIsInviting; fix this by wiring the fetch through
an AbortController: create an AbortController for the request (store it in a ref
or closure), pass controller.signal into fetch inside onClick, and add a
useEffect cleanup that calls controller.abort() on unmount (or abort any stored
controller refs). In the try/catch/finally around fetch, check for abort (e.g.,
err.name === 'AbortError') and avoid logging it as an error, and ensure you only
call setIsInviting(false) when the request was not aborted (or the component is
still mounted) so setIsInviting is not invoked after unmount.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fe6182e4-8df5-4408-b27d-24fb28cbdea8
📒 Files selected for processing (4)
sample-apps/react/react-dogfood/components/ActiveCallHeader.tsxsample-apps/react/react-dogfood/components/InviteVisionAgentButton.tsxsample-apps/react/react-dogfood/style/CallHeader/CallHeader-layout.scsssample-apps/react/react-dogfood/translations/en.json
…anel) Replaces the v1 header pill with a fuller Vision Agents UI: - "Ask AI Agent" gradient button in the bottom call controls (between the record and leave buttons). - "Ask an AI agent" modal with two informational cards (Talk to it / Knows Stream), an Invite agent CTA that POSTs to api.demo.visionagents.ai, and a Build your own AI Agent link (https://visionagents.ai/, new tab). - Floating "AI agent in call" status panel in the bottom-right of the stage; appears whenever a participant with a userId starting with "agent-" is detected. Remove first DELETEs the session via the Vision Agents API and falls back to call.kickUser if the session id is unknown or the DELETE fails.
oliverlaz
changed the title
- Detect the agent participant via userId.startsWith("agent") so calls
where the agent joins as user "agent" (no dash) light up the floating
status panel.
- Give the modal "Invite agent" CTA the same blue→purple gradient as
"Build your own AI Agent" so the primary action of the popup reads
as prominent.
flex-shrink: 0 and white-space: nowrap so the button stops collapsing into two lines when the call controls bar is crowded.
Contributor
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
Clicking "Ask AI Agent" in the call controls now POSTs to api.demo.visionagents.ai immediately instead of opening an interim modal. The button shows "Inviting…" while the request is in flight and the floating status panel still appears once the agent joins. Also removes the now-unused modal component, modal SCSS, and the translation keys that were only used by the modal copy.
…ent present - Drop the translateY hover effect so the pill no longer shifts on pointer enter/leave; only the gradient glow remains. - Disable the button while an agent participant (userId starting with "agent") is already in the call so the user cannot trigger duplicate invites.
The call-controls button now flips to "Remove Agent" once a participant with a userId starting with "agent" is in the call. Clicking it DELETEs the Vision Agents session (falling back to call.kickUser if the session id is unknown). The floating status panel keeps its own Remove button as a redundant entry point. Also points the "Build your own AI Agent" link in the floating panel at https://getstream.io/vision-agents/.
oliverlaz
changed the title
The call-controls button is now the single Remove entry point. The floating panel becomes a pure status indicator and adds a small close (X) so it can be dismissed; it re-appears when a new agent session arrives.
- Switch the call-controls pill to a red->purple gradient (mirroring the invite gradient's blue->purple shift) when an agent is in the call, with a resting shadow and a punchier hover glow. - min-width and justify-content keep the pill the same size as the label flips between Ask / Inviting / Remove / Removing.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@sample-apps/react/react-dogfood/components/VisionAgent/AIAgentStatusPanel.tsx`:
- Around line 34-38: The IconButton rendered in AIAgentStatusPanel is icon-only
(icon="close") and lacks an accessible name; update the IconButton props to
include an explicit accessible label (e.g., title or aria-label) using the
translation function (for example title={t('Dismiss')} or
aria-label={t('Dismiss')}) where the component is created (the IconButton with
onClick={() => setDismissed(true)}), so screen readers get a clear label for the
dismiss action.
In `@sample-apps/react/react-dogfood/components/VisionAgent/AskAIAgentButton.tsx`:
- Around line 54-60: The URL path interpolations use unescaped dynamic segments
(call.id and sessionId) which can break routing; update the fetch calls that
build `${VISION_AGENTS_API_BASE}/calls/${call.id}/sessions` and any other places
using `${sessionId}` (e.g., the invite/remove calls around lines 83–85) to wrap
dynamic path segments with encodeURIComponent (encodeURIComponent(call.id) and
encodeURIComponent(sessionId)) before interpolating so reserved characters
cannot alter the endpoint.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a2eb9ff3-e278-4252-a31f-48721f857fe0
📒 Files selected for processing (7)
sample-apps/react/react-dogfood/components/ActiveCall.tsxsample-apps/react/react-dogfood/components/VisionAgent/AIAgentStatusPanel.tsxsample-apps/react/react-dogfood/components/VisionAgent/AskAIAgentButton.tsxsample-apps/react/react-dogfood/components/VisionAgent/index.tssample-apps/react/react-dogfood/style/VisionAgent/index.scsssample-apps/react/react-dogfood/style/index.scsssample-apps/react/react-dogfood/translations/en.json
✅ Files skipped from review due to trivial changes (3)
- sample-apps/react/react-dogfood/style/index.scss
- sample-apps/react/react-dogfood/translations/en.json
- sample-apps/react/react-dogfood/style/VisionAgent/index.scss
🚧 Files skipped from review as they are similar to previous changes (1)
- sample-apps/react/react-dogfood/components/ActiveCall.tsx
Comment on lines
+34
to
+38
| <IconButton | ||
| className="rd__ai-agent-status__close" | ||
| icon="close" | ||
| onClick={() => setDismissed(true)} | ||
| /> |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n sample-apps/react/react-dogfood/components/VisionAgent/AIAgentStatusPanel.tsxRepository: GetStream/stream-video-js
Length of output: 1905
🏁 Script executed:
# Search for IconButton usage patterns in the codebase to understand how it handles accessible names
rg "IconButton" sample-apps/react/react-dogfood/components/ -A 3 -B 1 | head -50Repository: GetStream/stream-video-js
Length of output: 4219
🏁 Script executed:
# Check the imports to understand where IconButton comes from
rg "import.*IconButton" sample-apps/react/react-dogfood/Repository: GetStream/stream-video-js
Length of output: 450
🏁 Script executed:
# Search for IconButton type definitions or implementation in the SDK
find sample-apps -name "*.d.ts" -o -name "*.ts" | xargs grep -l "IconButton" | head -5Repository: GetStream/stream-video-js
Length of output: 586
🏁 Script executed:
# Check if there are any IconButton implementations or types exported from the SDK package
rg "export.*IconButton|interface IconButton|type IconButton" --type ts --type tsx | head -20Repository: GetStream/stream-video-js
Length of output: 97
🏁 Script executed:
# Look for node_modules or SDK package to check IconButton props
fd -t d "@stream-io" sample-apps/react/react-dogfood/node_modules 2>/dev/null | head -3Repository: GetStream/stream-video-js
Length of output: 51
🏁 Script executed:
# Search for aria-label or title patterns used with IconButton in the codebase
rg "IconButton.*aria-label|IconButton.*title|title.*IconButton|aria-label.*IconButton" sample-apps/react/react-dogfood/components/ -A 3 -B 1Repository: GetStream/stream-video-js
Length of output: 51
🏁 Script executed:
# Search for any IconButton usage that includes title, aria-label, or aria-description
rg "IconButton" sample-apps/react/react-dogfood/ -A 5 | rg -B 5 "title|aria-label|aria-description"Repository: GetStream/stream-video-js
Length of output: 51
🏁 Script executed:
# Check the package.json to understand the SDK version being used
cat sample-apps/react/react-dogfood/package.json | grep -A 2 -B 2 "video-react-sdk"Repository: GetStream/stream-video-js
Length of output: 278
🏁 Script executed:
# Look for any other icon-only buttons with accessible names for comparison
rg "icon=" sample-apps/react/react-dogfood/components/ -B 2 -A 2 | head -40Repository: GetStream/stream-video-js
Length of output: 3839
🏁 Script executed:
# Search for IconButton implementation in the SDK source code
find . -path "./packages/*/src" -name "*.tsx" -o -name "*.ts" | xargs rg "IconButton" -l 2>/dev/null | grep -v node_modulesRepository: GetStream/stream-video-js
Length of output: 121
🏁 Script executed:
# Look specifically in the video-react-sdk package
find packages -name "*IconButton*" -o -path "*/components/*" -name "*.tsx" | head -10Repository: GetStream/stream-video-js
Length of output: 817
🏁 Script executed:
# Search for IconButton export and definition in the SDK
rg "export.*IconButton|const IconButton" packages/ --type ts --type tsx -B 2 -A 10Repository: GetStream/stream-video-js
Length of output: 97
🏁 Script executed:
# Search for IconButton in the react-sdk Button directory
cat packages/react-sdk/src/components/Button/index.tsRepository: GetStream/stream-video-js
Length of output: 164
🏁 Script executed:
# List files in the Button directory
ls -la packages/react-sdk/src/components/Button/Repository: GetStream/stream-video-js
Length of output: 442
🏁 Script executed:
cat packages/react-sdk/src/components/Button/IconButton.tsxRepository: GetStream/stream-video-js
Length of output: 944
🏁 Script executed:
# Also check the CompositeButton since IconButton likely extends it
cat packages/react-sdk/src/components/Button/CompositeButton.tsxRepository: GetStream/stream-video-js
Length of output: 3102
Add an explicit accessible name to the dismiss button.
Line 34 renders an icon-only IconButton without an accessible name. The icon="close" prop only controls visual rendering and does not provide an accessible label. Screen readers will expose this as an unnamed button, and users will not know its purpose. Pass a translated title or aria-label explicitly (e.g., title={t('Dismiss')}).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@sample-apps/react/react-dogfood/components/VisionAgent/AIAgentStatusPanel.tsx`
around lines 34 - 38, The IconButton rendered in AIAgentStatusPanel is icon-only
(icon="close") and lacks an accessible name; update the IconButton props to
include an explicit accessible label (e.g., title or aria-label) using the
translation function (for example title={t('Dismiss')} or
aria-label={t('Dismiss')}) where the component is created (the IconButton with
onClick={() => setDismissed(true)}), so screen readers get a clear label for the
dismiss action.
Comment on lines
+54
to
+60
| const response = await fetch( | ||
| `${VISION_AGENTS_API_BASE}/calls/${call.id}/sessions`, | ||
| { | ||
| method: 'POST', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| body: JSON.stringify({ call_type: call.type ?? 'default' }), | ||
| }, |
Contributor
There was a problem hiding this comment.
Encode dynamic path segments before calling the Vision Agents API.
Lines 55 and 84 interpolate call.id and sessionId directly into the URL path. Any reserved character in either value, such as / or ?, changes the target endpoint and can make invite/remove hit the wrong resource.
Suggested fix
- `${VISION_AGENTS_API_BASE}/calls/${call.id}/sessions`,
+ `${VISION_AGENTS_API_BASE}/calls/${encodeURIComponent(call.id)}/sessions`,
@@
- `${VISION_AGENTS_API_BASE}/calls/${call.id}/sessions/${sessionId}`,
+ `${VISION_AGENTS_API_BASE}/calls/${encodeURIComponent(call.id)}/sessions/${encodeURIComponent(sessionId)}`,Also applies to: 83-85
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sample-apps/react/react-dogfood/components/VisionAgent/AskAIAgentButton.tsx`
around lines 54 - 60, The URL path interpolations use unescaped dynamic segments
(call.id and sessionId) which can break routing; update the fetch calls that
build `${VISION_AGENTS_API_BASE}/calls/${call.id}/sessions` and any other places
using `${sessionId}` (e.g., the invite/remove calls around lines 83–85) to wrap
dynamic path segments with encodeURIComponent (encodeURIComponent(call.id) and
encodeURIComponent(sessionId)) before interpolating so reserved characters
cannot alter the endpoint.
…ls button The "AI agent in call" panel now floats centered above the Ask/Remove Agent pill in the call controls bar instead of in the bottom-right of the stage, making the link between the panel and the button it controls visually obvious.
flex-center the IconButton, drop the SDK button's default padding / min-width, and shrink the inner icon to 14px so it sits in the middle of the 24x24 close hit area.
oliverlaz
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 Overview
Adds a one-click Vision Agents flow to the React dogfood sample app:
https://api.demo.visionagents.ai/calls/{call_id}/sessionsto invite a vision agent. Once a participant whoseuserIdstarts withagentis detected, the same button flips to Remove Agent, switches to a red→purple gradient, and on click DELETEs the session (DELETE /calls/{call_id}/sessions/{session_id}) with acall.kickUserfallback when the session id is unknown (e.g. after a page refresh) or DELETE fails. ShowsInviting…/Removing…while in flight. The pill has a fixed min-width so it does not jump as labels change, and the gradient does not move on hover.📝 Implementation notes
sample-apps/react/react-dogfood/components/VisionAgent/{AskAIAgentButton,AIAgentStatusPanel,index}.tsx.ActiveCall.tsxand passed to the button (used for the DELETE call). The panel is propless and detects the agent on its own viauseParticipants().console.error; no toast surface.InviteVisionAgentButton) shipped earlier on this branch — the v1 file, its SCSS block, and its translation key are removed.🎫 Ticket: https://linear.app/stream/issue/AI-557/add-a-button-to-invite-an-agent-to-a-demo-call
Summary by CodeRabbit
New Features
Style