Skip to content

Fix ordering of Gemini inlined batch responses#2124

Open
neo-con wants to merge 1 commit intogoogleapis:mainfrom
neo-con:fix-inlined-batch-order-1909
Open

Fix ordering of Gemini inlined batch responses#2124
neo-con wants to merge 1 commit intogoogleapis:mainfrom
neo-con:fix-inlined-batch-order-1909

Conversation

@neo-con
Copy link

@neo-con neo-con commented Mar 5, 2026

Summary

Fixes #1909.

The Gemini API can return inlinedResponses out of request order for batch jobs that use inlined_requests. This change preserves the original request order by stamping an SDK-reserved order marker on each inlined request and reordering returned inlined responses by that marker when the full marker set is present and valid.

This change keeps the behavior conservative:

  • if any response is missing the reserved order marker, preserve backend order
  • if any response has a non-numeric order marker, preserve backend order
  • caller-provided metadata cannot override the reserved SDK order marker

Changes

  • add a reserved internal metadata key for inlined request ordering
  • stamp deterministic request-order metadata onto Gemini inlined batch requests
  • reorder Gemini inlined batch responses by that metadata before returning them
  • strip the internal order marker from returned response metadata after reordering
  • add regression coverage for:
    • internal order metadata injection
    • reserved-key override behavior
    • deterministic response reordering
    • missing-metadata fallback
    • non-numeric fallback

Testing

env -u GOOGLE_API_KEY -u GEMINI_API_KEY python -m py_compile google/genai/batches.py google/genai/tests/conftest.py google/genai/tests/batches/test_create_with_inlined_requests.py
env -u GOOGLE_API_KEY -u GEMINI_API_KEY GOOGLE_GENAI_REPLAYS_DIRECTORY=/tmp/google-genai-replays pytest google/genai/tests/batches/test_create_with_inlined_requests.py --mode replay -q
env -u GOOGLE_API_KEY -u GEMINI_API_KEY GOOGLE_GENAI_REPLAYS_DIRECTORY=/tmp/google-genai-replays pytest google/genai/tests/batches/test_create_with_inlined_requests.py -k internal_order_metadata --mode replay -q

@google-cla
Copy link

google-cla bot commented Mar 5, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Venkaiahbabuneelam Venkaiahbabuneelam self-assigned this Mar 6, 2026
@Venkaiahbabuneelam Venkaiahbabuneelam added the size:L Code changes between 40-100 lines label Mar 6, 2026
@Venkaiahbabuneelam
Copy link

Hi @neo-con, Thanks for reaching out us!

Looks like the branch is out of date, Can you please update the branch.
Thanks

@neo-con neo-con force-pushed the fix-inlined-batch-order-1909 branch from f6da1f8 to 7f21787 Compare March 9, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L Code changes between 40-100 lines status:awaiting user response

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Batch API InlinedResponses are not returned in order

2 participants