feat: Add validate_response option#74
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #74 +/- ##
==========================================
Coverage 100.00% 100.00%
==========================================
Files 6 6
Lines 1001 1132 +131
==========================================
+ Hits 1001 1132 +131
🚀 New features to boost your workflow:
|
will-ockmore
left a comment
There was a problem hiding this comment.
Thanks for this @vgavro . It's a well thought out addition.
I'd like to take validate_response, but as a response validation hook, not the fix for #29. Your second example is a real problem that we should aim to solve, and validate_response gives users the tool to solve it. The implementation is
Why I don't think this is the right fix in general for #29 (and I'll give more detail there; apologies for my slow response on that thread, I've been moving countries) is twofold:
- At the general level: it breaks the layer boundary. Setting the
requestattribute is a symptom of this, but the transport should not be handling these aspects outside its remit. The first example would break streaming; the whole body would be read into memory, and this would not be clear to the user. This is another symptom of the same issue. - It requires users to understand the library internals to fix something that should be solved by default. They should not need to know the incantation to solve body-phase retries, it should be transparent to them.
So to get this PR ready to merge:
- Drop the faq.md edit that presents this as the solution to body-phase retries.
- Document it for content validation; mention that calling .read() inside it will buffer stream=True responses, so it shouldn't be used that way.
- Consider moving the async/sync-transport-mismatch check to RetryTransport.init so it fails fast
| if not retry.is_retryable_status_code(response.status_code): | ||
| if self.retry.validate_response is not None: | ||
| # normally set by httpx _after_ calling this function, but we want the request in the validator | ||
| response.request = request |
There was a problem hiding this comment.
this is the only slightly iffy operation. We should not be changing the behaviour of httpx in general. This is probably fine here.
Adding
Retry.validate_responsecallback to Retry configuration.Inspired by
pydantic_ai.retriesapproach https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/retries.pyThis should effectively solve two cases:
response.readandresponse.areadcan't be handled insideRetryTransport#29 - this shouldn't be default, but we're allowing user to do:While
ReadTimeoutis already in defaultRetry.retry_on_exceptions, it will work as expected. httpx will not re-read body on second.read()invocation, body is cached once readed anyway.