Show retries for HTTP status code errors (#13897)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | pyodide on ubuntu (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | free-threaded python on github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | graalpy on ubuntu (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks | walltime aarch64 linux (push) Blocked by required conditions
CI / benchmarks | instrumented (push) Blocked by required conditions

Using a companion change in the middleware
(https://github.com/TrueLayer/reqwest-middleware/pull/235, forked&tagged
pending review), we can check and show retries for HTTP status core
errors, to consistently report retries again.

We fix two cases:
* Show retries for status code errors for cache client requests
* Show retries for status code errors for Python download requests

Not handled:
* Show previous retries when a distribution download fails mid-streaming
* Perform retries when a distribution download fails mid-streaming
* Show previous retries when a Python download fails mid-streaming
* Perform retries when a Python download fails mid-streaming
This commit is contained in:
konsti 2025-06-16 12:14:00 +02:00 committed by GitHub
parent 7c90c5be02
commit cd71ad1672
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 205 additions and 146 deletions

View file

@ -53,6 +53,12 @@ pub enum Error {
TooManyParts(String),
#[error("Failed to download {0}")]
NetworkError(DisplaySafeUrl, #[source] WrappedReqwestError),
#[error("Request failed after {retries} retries")]
NetworkErrorWithRetries {
#[source]
err: Box<Error>,
retries: u32,
},
#[error("Failed to download {0}")]
NetworkMiddlewareError(DisplaySafeUrl, #[source] anyhow::Error),
#[error("Failed to extract archive: {0}")]
@ -1143,8 +1149,20 @@ fn parse_json_downloads(
}
impl Error {
pub(crate) fn from_reqwest(url: DisplaySafeUrl, err: reqwest::Error) -> Self {
Self::NetworkError(url, WrappedReqwestError::from(err))
pub(crate) fn from_reqwest(
url: DisplaySafeUrl,
err: reqwest::Error,
retries: Option<u32>,
) -> Self {
let err = Self::NetworkError(url, WrappedReqwestError::from(err));
if let Some(retries) = retries {
Self::NetworkErrorWithRetries {
err: Box::new(err),
retries,
}
} else {
err
}
}
pub(crate) fn from_reqwest_middleware(
@ -1260,10 +1278,15 @@ async fn read_url(
.await
.map_err(|err| Error::from_reqwest_middleware(url.clone(), err))?;
// Ensure the request was successful.
response
.error_for_status_ref()
.map_err(|err| Error::from_reqwest(url, err))?;
let retry_count = response
.extensions()
.get::<reqwest_retry::RetryCount>()
.map(|retries| retries.value());
// Check the status code.
let response = response
.error_for_status()
.map_err(|err| Error::from_reqwest(url, err, retry_count))?;
let size = response.content_length();
let stream = response