refactor: dev to main (refactor Streaming Chat API for Structured Responses and Citation Handling)

[Pavan-Microsoft]

Purpose

This pull request refactors the streaming chat API to yield structured (role, content) tuples instead of raw strings, aligning backend message formats with frontend expectations and improving citation handling. It also updates the associated TypeScript types and test cases to reflect the new streaming protocol.

API streaming protocol refactor:

• The stream_openai_text method in chat_service.py now yields (role, content) tuples, where role is either "assistant" for answer text or "tool" for citations, instead of yielding raw JSON or strings. This makes the streaming interface more explicit and easier to consume. [1] [2] [3]
• The stream_chat_request method is updated to consume these tuples and emit streaming responses with a choices array containing a delta object for each chunk, matching the frontend protocol.

Citation and marker handling improvements:

• Citation markers in streamed text are now replaced for both markers with and without the † character, ensuring all citation formats are handled consistently. [1] [2]
• Citations are collected and sent as a separate "tool" role message in JSON array format, decoupling them from the assistant's main response text. [1] [2]

Type and test updates:

• The ParsedChunk TypeScript type in AppTypes.ts is updated to support the new streaming format, with choices as an optional array containing either messages or a delta.
• All relevant tests in test_chat_service.py are updated to expect and validate the new (role, content) tuple protocol, including checks for correct role assignment, content, and citation handling. Additional test cases are added for marker replacement and fallback scenarios. [1] [2] [3] [4] [5]

These changes make the streaming API more robust, explicit, and easier to test and maintain.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

[Copilot]

Pull request overview

This PR refactors the backend chat streaming implementation to emit explicit (role, content) chunks (assistant answer text vs tool/citations), and updates the frontend to consume the new choices[0].delta streaming protocol while improving citation-marker replacement behavior.

Changes:

• Backend: stream_openai_text now yields (role, content) tuples and stream_chat_request emits choices[0].delta { role, content } JSON-lines.
• Frontend: streaming parsing updated to accumulate assistant deltas and handle citations via tool-role deltas; citation parsing supports both legacy and new formats.
• Types/tests: ParsedChunk updated for the new stream shape; unit tests updated and expanded for marker replacement and role separation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/api/services/chat_service.py	Emits structured (role, content) chunks; replaces citation markers with/without dagger; sends citations as a tool chunk.
src/tests/api/services/test_chat_service.py	Updates streaming tests to validate role-separated chunks and new streaming deltas.
src/App/src/types/AppTypes.ts	Extends ParsedChunk typing to support delta chunks (and legacy messages).
src/App/src/components/Chat/Chat.tsx	Refactors stream parsing to consume delta chunks; updates citation parsing and chart streaming handling.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.