feat: add exponential backoff retry for transient provider errors#9
Open
ramparte wants to merge 2 commits intomicrosoft:mainfrom
Open
feat: add exponential backoff retry for transient provider errors#9ramparte wants to merge 2 commits intomicrosoft:mainfrom
ramparte wants to merge 2 commits intomicrosoft:mainfrom
Conversation
…imiting test markers (microsoft#8) * fix: read HookResult modify action from tool:post events (2 sites) Both _execute_tool_only and _execute_tool_with_result now detect when a hook modifies tool output via action='modify' on tool:post, and use the modified data instead of the original get_serialized_output(). This enables truncation and transformation hooks to work correctly. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> * fix: add missing @pytest.mark.asyncio to TestRateLimitDelay class The 4 async test methods in TestRateLimitDelay were missing the asyncio marker. With asyncio_mode = "strict" in pyproject.toml, explicit markers are required. Added class-level @pytest.mark.asyncio decorator. --------- Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Wrap all 3 provider call sites in the streaming orchestrator with configurable exponential backoff retry logic. Retries only on LLMError with retryable=True (RateLimitError, ProviderUnavailableError, LLMTimeoutError). Honors retry_after from provider responses and emits provider:retry events for observability. Config: retry_max_attempts (default 3), retry_base_delay_seconds (1.0), retry_max_delay_seconds (30.0). 18 new tests covering all retry behaviors, zero regressions. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Provider rate limit errors (HTTP 429) and transient failures cause sessions to crash immediately, even though these errors are inherently temporary and resolve on retry.
Solution
Added a
_call_provider_with_retry()method to the streaming orchestrator that wraps all 3 provider call sites with configurable exponential backoff:LLMError.retryableflag (True forRateLimitError,ProviderUnavailableError,LLMTimeoutError)base_delay * 2^attempt, capped atmax_delayretry_after: Uses server-provided delay when available (e.g., from 429 responses)provider:retryevents on each retry attemptConfiguration
retry_max_attemptsretry_base_delay_secondsretry_max_delay_secondsCall sites modified
provider.complete()) - main execution loopprovider.complete()) - graceful degradation pathprovider.stream()) - primary streaming pathTest coverage
18 new tests covering all retry behaviors. All 53 tests pass (18 new + 35 existing, zero regressions).
🤖 Generated with Amplifier