Skip to content

Comments

Add streaming tool result events#20613

Open
ai-fanatic wants to merge 3 commits intorun-llama:mainfrom
ai-fanatic:feature/native-structured-output-18412
Open

Add streaming tool result events#20613
ai-fanatic wants to merge 3 commits intorun-llama:mainfrom
ai-fanatic:feature/native-structured-output-18412

Conversation

@ai-fanatic
Copy link

Summary

  • add streaming tool output support (acall_stream) with async generator handling in FunctionTool
  • emit preliminary ToolCallResult events and ignore them during aggregation to avoid early completion
  • add test coverage for streaming tool outputs

Testing

  • uv run pre-commit run --show-diff-on-failure --files <changed files> (Windows; equivalent to make lint)
  • uv run -- pytest llama-index-core/tests/agent/workflow/test_tool_streaming.py

AI Assistance

I used ChatGPT/Codex to help draft the changes. I reviewed the code and verified behavior via the lint and test commands above.

Issue

Relates to #20409

@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Feb 2, 2026
default="force",
description="Method to handle max iterations. 'force' raises an error (default). 'generate' makes one final LLM call to generate a response.",
)
use_native_structured_output: bool = Field(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some of these changes weren't intended to be in this PR?

@@ -342,42 +347,69 @@ def call(self, *args: Any, **kwargs: Any) -> ToolOutput:

async def acall(self, *args: Any, **kwargs: Any) -> ToolOutput:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only acall? Why not the sync version as well? Had a similar comment on another user that attempted this PR
#20552

@ai-fanatic ai-fanatic force-pushed the feature/native-structured-output-18412 branch from 92474d2 to fca65af Compare February 2, 2026 17:26
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Feb 2, 2026
@ai-fanatic
Copy link
Author

Thanks for the review.

  • Unintended changes: I rebased the branch onto upstream/main to drop unrelated structured‑output commits, then force‑pushed. The PR now only includes streaming tool result changes.
  • Sync version: added sync streaming support (FunctionTool.call_stream, BaseTool.call_stream, BaseToolAsyncAdapter.acall_stream) and treat sync generator functions as streaming; also added a sync generator test alongside the async one.

Checks run:

  • uv run pre-commit run --show-diff-on-failure --files <changed files>
  • uv run -- pytest llama-index-core/tests/agent/workflow/test_tool_streaming.py

@logan-markewich
Copy link
Collaborator

llama_index/core/tools/function_tool.py:437: error: "Awaitable[Any]" has no attribute "__aiter__"; maybe "__await__"? (not async iterable)  [attr-defined]
llama_index/core/tools/function_tool.py:440: error: No overload variant of "iter" matches argument type "Awaitable[Any]"  [call-overload]
llama_index/core/tools/function_tool.py:440: note: Possible overload variants:
llama_index/core/tools/function_tool.py:440: note:     def [_SupportsNextT: SupportsNext[Any]] iter(SupportsIter[_SupportsNextT], /) -> _SupportsNextT
llama_index/core/tools/function_tool.py:440: note:     def [_T] iter(_GetItemIterable[_T], /) -> Iterator[_T]
llama_index/core/tools/function_tool.py:440: note:     def [_T] iter(Callable[[], _T | None], None, /) -> Iterator[_T]
llama_index/core/tools/function_tool.py:440: note:     def [_T] iter(Callable[[], _T], object, /) -> Iterator[_T]

@ai-fanatic
Copy link
Author

Fixed the mypy errors in FunctionTool.acall_stream by explicitly typing raw_output as Any before checking awaitable/iterable branches. Pushed commit 4a3a178 to feature/native-structured-output-18412.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants