test: improve test_end_to_end.py

[ishymko]

Description

• Test artifacts.
• Add more assertions for streaming: validate all events.
• Fix non-streaming tests which were actually streaming.

[gemini-code-assist]

Summary of Changes

Hello @ishymko, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the robustness and maintainability of the integration test suite. By centralizing client setup through a ClientFactory and consolidating transport-specific tests into parametrized fixtures, it ensures consistent behavior verification across all supported communication protocols. The changes also introduce more detailed assertions for streaming events, including artifact handling, and correct the execution of non-streaming tests, leading to a more reliable and comprehensive testing environment.

Highlights

• Test Refactoring and Consolidation: The integration tests have been significantly refactored to use a unified ClientSetup fixture and a parametrized client_setups fixture, allowing tests to run consistently across REST, JSON-RPC, and gRPC transports with a single test function.
• ClientFactory Integration: All client initialization in tests now leverages the ClientFactory for creating BaseClient instances, simplifying setup and ensuring consistent client configuration across different transport protocols.
• Improved Streaming Assertions: The streaming tests now include more comprehensive assertions, validating all events received during a stream, including status updates and newly introduced artifact updates.
• Corrected Non-Streaming Test Behavior: Non-streaming tests (test_end_to_end_send_message_blocking and test_end_to_end_send_message_non_blocking) were updated to explicitly disable streaming via client._config.streaming = False, ensuring they correctly test blocking/non-streaming client behavior.
• Artifact Testing: Added a mock artifact creation in the MockAgentExecutor and corresponding assertions in streaming and blocking tests to verify artifact handling.
• Simplified Transport Management: Direct Transport objects and their explicit close() calls have been removed from test fixtures, with client lifecycle managed implicitly by the ClientFactory and ClientSetup.

Changelog

• tests/integration/test_client_server_integration.py
  • Updated imports to align with new client architecture.
  • Added JSONRPC protocol binding to the agent_card fixture.
  • Renamed TransportSetup to ClientSetup to reflect its broader scope.
  • Refactored jsonrpc_setup, rest_setup, and grpc_setup fixtures to utilize ClientFactory for client creation.
  • Introduced a client_setups parametrized fixture to run tests across all transport types.
  • Consolidated multiple transport-specific test functions (e.g., test_http_transport_sends_message_streaming, test_grpc_transport_sends_message_streaming) into single, parametrized tests (e.g., test_client_sends_message_streaming).
  • Removed explicit transport.close() calls, relying on fixture teardown for client resource management.
  • Updated signed card tests to use ClientFactory.connect or factory.create for client instantiation and card resolution.
• tests/integration/test_end_to_end.py
  • Removed unused imports, including cast and SendMessageRequest.
  • Added add_artifact call within MockAgentExecutor.execute to simulate artifact generation.
  • Renamed TransportSetup to ClientSetup for consistency.
  • Updated rest_setup, jsonrpc_setup, and grpc_setup fixtures to return ClientSetup and use ClientFactory.
  • Modified test_end_to_end_send_message_blocking and test_end_to_end_send_message_non_blocking to explicitly set client._config.streaming = False.
  • Added assertions for task artifacts in test_end_to_end_send_message_blocking.
  • Adjusted assertions in test_end_to_end_send_message_non_blocking to check for TASK_STATE_SUBMITTED.
  • Enhanced test_end_to_end_send_message_streaming with detailed checks for all expected event types (status updates, artifact updates) and their content.
  • Refined test_end_to_end_list_tasks to store and verify all collected task IDs, ensuring complete pagination coverage.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request significantly refactors and improves the integration and end-to-end tests. The introduction of ClientFactory and parametrized fixtures for different transports greatly enhances test maintainability and reduces code duplication. The fixes for non-streaming tests and the addition of more thorough assertions for streaming events, including artifact validation, improve test correctness and robustness. Overall, these are excellent changes that increase the quality of the test suite. I've added a few minor suggestions for consistency and simplification.

I am having trouble creating individual review comments. Click here to see my feedback.

tests/integration/test_client_server_integration.py (249)

The lambda function here is unnecessary. You can directly pass grpc.aio.insecure_channel as the grpc_channel_factory.

            grpc_channel_factory=grpc.aio.insecure_channel,

tests/integration/test_client_server_integration.py (555)

The lambda function here is unnecessary. You can directly pass grpc.aio.insecure_channel as the grpc_channel_factory.

            grpc_channel_factory=grpc.aio.insecure_channel,

tests/integration/test_client_server_integration.py (869)

The lambda function here is unnecessary. You can directly pass grpc.aio.insecure_channel as the grpc_channel_factory.

                grpc_channel_factory=grpc.aio.insecure_channel,

tests/integration/test_end_to_end.py (202)

For consistency with test_client_server_integration.py, consider disabling streaming by modifying the agent card's capabilities instead of the client config. This more accurately simulates an agent that doesn't support streaming.

    client._card.capabilities.streaming = False

tests/integration/test_end_to_end.py (229)

For consistency with test_client_server_integration.py, consider disabling streaming by modifying the agent card's capabilities instead of the client config. This more accurately simulates an agent that doesn't support streaming.

    client._card.capabilities.streaming = False