Skip to content

Feat: add request.extensions["retry"]#75

Open
vgavro wants to merge 3 commits into
will-ockmore:mainfrom
vgavro:feat/request-extensions
Open

Feat: add request.extensions["retry"]#75
vgavro wants to merge 3 commits into
will-ockmore:mainfrom
vgavro:feat/request-extensions

Conversation

@vgavro
Copy link
Copy Markdown

@vgavro vgavro commented May 12, 2026

This feature allows to:

  1. Override Retry configuration per request, the same way as Timeout configuration may be overriden per request, see https://www.python-httpx.org/advanced/extensions/
client.get("https://example.com", extensions={"retry": Retry(...)})
  1. Allows introspection on each request of actual retries made and time spent , because response.request.extensions["retry"] is always set (the same way as response.request.extensions["timeout"] is always set, so it's consistent across httpx api expectations)

  2. This is also pretty useful with feat: Add validate_response option #74 - considering you decide what request responses you want to pre-validate (for example, to read body in RetryTransport to retry timeouts), and what requests you will use with streaming.
    NOTE - these two merge requests are in conflict, some of them should be merged first, and then i'll adopt other IF you're willing to merge both hopefully.

NOTES ON SOME DESIGN DECISIONS:

  1. Retry.copy_with was implemented for easier use to copy global retry configuration with per-request retry configuration - naming like httpx.URL.copy_with, but instead of **kwargs in signature we're using full typing with _UNSET sentinel -for better type checker autocompletions - same pattern used in httpx._config

vgavro added 2 commits May 12, 2026 16:34
…uest, also set this object on request for introspection purposes
…xtensions["retry"] - allows more easily override Retry configuration parameters per request to pass it like Client.get(extensions={"retry": transport.retry.copy_with(...)}) (signature like httpx.URL.copy_with, typing approach with _UNSET from httpx._config)
@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (999e2aa) to head (c4433b2).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##              main       #75   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            6         6           
  Lines         1001      1065   +64     
=========================================
+ Hits          1001      1065   +64     
Files with missing lines Coverage Δ
httpx_retries/retry.py 100.00% <100.00%> (ø)
httpx_retries/transport.py 100.00% <100.00%> (ø)
tests/test_retry.py 100.00% <100.00%> (ø)
tests/test_transport.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Owner

@will-ockmore will-ockmore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @vgavro for this, appreciate the though that you've taken here. Splitting my feedback by the two halves:

copy_with(): This is great. It's the right fix for the field-list duplication in increment(). Two things I'd like:

  1. add a test that round-trips every __init__ field through copy_with() so the two signatures can't silently drift
  2. we'll need to thread validate_response through once #74/#75 ordering is sorted.

request.extensions["retry"]: I'd like to take this as a separate PR so we can get the semantics right. The blocker for me is that the transport writes the incremented retry back into request.extensions. Request objects can be reused, so a reused request comes back with a depleted budget. Can we

  1. read the override in from request.extensions["retry"] but not mutate it, and
  2. if we want post-hoc introspection of attempts_made, put the final retry on response.extensions instead?
    That keeps input config and output state on the right-lifetime objects.

Additionally, as it's squatting httpx's namespace here in the extensions dict, we should proceed with caution. It's a valid use case, but we may want to guard against a future version where this key is set externally.

Comment thread httpx_retries/retry.py
Comment on lines +303 to +317
total=self.total if isinstance(total, _UnsetType) else total,
allowed_methods=self.allowed_methods if isinstance(allowed_methods, _UnsetType) else allowed_methods,
status_forcelist=self.status_forcelist if isinstance(status_forcelist, _UnsetType) else status_forcelist,
retry_on_exceptions=self.retryable_exceptions
if isinstance(retry_on_exceptions, _UnsetType)
else retry_on_exceptions,
backoff_factor=self.backoff_factor if isinstance(backoff_factor, _UnsetType) else backoff_factor,
respect_retry_after_header=self.respect_retry_after_header
if isinstance(respect_retry_after_header, _UnsetType)
else respect_retry_after_header,
max_backoff_wait=self.max_backoff_wait if isinstance(max_backoff_wait, _UnsetType) else max_backoff_wait,
backoff_jitter=self.backoff_jitter if isinstance(backoff_jitter, _UnsetType) else backoff_jitter,
attempts_made=self.attempts_made if isinstance(attempts_made, _UnsetType) else attempts_made,
total_timeout=self.total_timeout if isinstance(total_timeout, _UnsetType) else total_timeout,
elapsed_sleep=self.elapsed_sleep if isinstance(elapsed_sleep, _UnsetType) else elapsed_sleep,
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we're using the sentinel pattern here, the more performant check is x is _UNSET instead of isinstance(x, _UnsetType)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants