mirror of
https://github.com/astral-sh/uv.git
synced 2025-07-07 21:35:00 +00:00
Show retries for HTTP status code errors (#13897)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | pyodide on ubuntu (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | free-threaded python on github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | graalpy on ubuntu (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks | walltime aarch64 linux (push) Blocked by required conditions
CI / benchmarks | instrumented (push) Blocked by required conditions
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | pyodide on ubuntu (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | free-threaded python on github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | graalpy on ubuntu (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks | walltime aarch64 linux (push) Blocked by required conditions
CI / benchmarks | instrumented (push) Blocked by required conditions
Using a companion change in the middleware (https://github.com/TrueLayer/reqwest-middleware/pull/235, forked&tagged pending review), we can check and show retries for HTTP status core errors, to consistently report retries again. We fix two cases: * Show retries for status code errors for cache client requests * Show retries for status code errors for Python download requests Not handled: * Show previous retries when a distribution download fails mid-streaming * Perform retries when a distribution download fails mid-streaming * Show previous retries when a Python download fails mid-streaming * Perform retries when a Python download fails mid-streaming
This commit is contained in:
parent
7c90c5be02
commit
cd71ad1672
12 changed files with 205 additions and 146 deletions
3
.gitignore
vendored
3
.gitignore
vendored
|
@ -3,9 +3,10 @@
|
|||
|
||||
# Generated by Cargo
|
||||
# will have compiled files and executables
|
||||
/vendor/
|
||||
debug/
|
||||
target/
|
||||
target-alpine/
|
||||
target/
|
||||
|
||||
# Bootstrapped Python versions
|
||||
/bin/
|
||||
|
|
80
Cargo.lock
generated
80
Cargo.lock
generated
|
@ -933,7 +933,7 @@ dependencies = [
|
|||
"hashbrown 0.14.5",
|
||||
"lock_api",
|
||||
"once_cell",
|
||||
"parking_lot_core 0.9.10",
|
||||
"parking_lot_core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -1916,18 +1916,6 @@ dependencies = [
|
|||
"similar",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "instant"
|
||||
version = "0.1.13"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"js-sys",
|
||||
"wasm-bindgen",
|
||||
"web-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ipnet"
|
||||
version = "2.11.0"
|
||||
|
@ -2114,7 +2102,7 @@ checksum = "c0ff37bd590ca25063e35af745c343cb7a0271906fb7b37e4813e8f79f00268d"
|
|||
dependencies = [
|
||||
"bitflags 2.9.1",
|
||||
"libc",
|
||||
"redox_syscall 0.5.8",
|
||||
"redox_syscall",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -2496,17 +2484,6 @@ version = "2.2.1"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba"
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot"
|
||||
version = "0.11.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7d17b78036a60663b797adeaee46f5c9dfebb86948d1255007a1d6be0271ff99"
|
||||
dependencies = [
|
||||
"instant",
|
||||
"lock_api",
|
||||
"parking_lot_core 0.8.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot"
|
||||
version = "0.12.3"
|
||||
|
@ -2514,21 +2491,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
|||
checksum = "f1bf18183cf54e8d6059647fc3063646a1801cf30896933ec2311622cc4b9a27"
|
||||
dependencies = [
|
||||
"lock_api",
|
||||
"parking_lot_core 0.9.10",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot_core"
|
||||
version = "0.8.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "60a2cfe6f0ad2bfc16aefa463b497d5c7a5ecd44a23efa72aa342d90177356dc"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"instant",
|
||||
"libc",
|
||||
"redox_syscall 0.2.16",
|
||||
"smallvec",
|
||||
"winapi",
|
||||
"parking_lot_core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -2539,7 +2502,7 @@ checksum = "1e401f977ab385c9e4e3ab30627d6f26d00e2c73eef317493c4ec6d468726cf8"
|
|||
dependencies = [
|
||||
"cfg-if",
|
||||
"libc",
|
||||
"redox_syscall 0.5.8",
|
||||
"redox_syscall",
|
||||
"smallvec",
|
||||
"windows-targets 0.52.6",
|
||||
]
|
||||
|
@ -2969,15 +2932,6 @@ version = "0.5.0"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3b42e27ef78c35d3998403c1d26f3efd9e135d3e5121b0a4845cc5cc27547f4f"
|
||||
|
||||
[[package]]
|
||||
name = "redox_syscall"
|
||||
version = "0.2.16"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fb5a58c1855b4b6819d59012155603f0b22ad30cad752600aadfcb695265519a"
|
||||
dependencies = [
|
||||
"bitflags 1.3.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redox_syscall"
|
||||
version = "0.5.8"
|
||||
|
@ -3137,8 +3091,7 @@ dependencies = [
|
|||
[[package]]
|
||||
name = "reqwest-middleware"
|
||||
version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "57f17d28a6e6acfe1733fe24bcd30774d13bffa4b8a22535b4c8c98423088d4e"
|
||||
source = "git+https://github.com/astral-sh/reqwest-middleware?rev=ad8b9d332d1773fde8b4cd008486de5973e0a3f8#ad8b9d332d1773fde8b4cd008486de5973e0a3f8"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
|
@ -3152,8 +3105,7 @@ dependencies = [
|
|||
[[package]]
|
||||
name = "reqwest-retry"
|
||||
version = "0.7.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "29c73e4195a6bfbcb174b790d9b3407ab90646976c55de58a6515da25d851178"
|
||||
source = "git+https://github.com/astral-sh/reqwest-middleware?rev=ad8b9d332d1773fde8b4cd008486de5973e0a3f8#ad8b9d332d1773fde8b4cd008486de5973e0a3f8"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
|
@ -3161,14 +3113,13 @@ dependencies = [
|
|||
"getrandom 0.2.15",
|
||||
"http",
|
||||
"hyper",
|
||||
"parking_lot 0.11.2",
|
||||
"reqwest",
|
||||
"reqwest-middleware",
|
||||
"retry-policies",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"wasm-timer",
|
||||
"wasmtimer",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -3916,7 +3867,7 @@ version = "0.3.6"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "96374855068f47402c3121c6eed88d29cb1de8f3ab27090e273e420bdabcf050"
|
||||
dependencies = [
|
||||
"parking_lot 0.12.3",
|
||||
"parking_lot",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -4158,7 +4109,7 @@ dependencies = [
|
|||
"bytes",
|
||||
"libc",
|
||||
"mio",
|
||||
"parking_lot 0.12.3",
|
||||
"parking_lot",
|
||||
"pin-project-lite",
|
||||
"signal-hook-registry",
|
||||
"socket2",
|
||||
|
@ -6183,18 +6134,17 @@ dependencies = [
|
|||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-timer"
|
||||
version = "0.2.5"
|
||||
name = "wasmtimer"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "be0ecb0db480561e9a7642b5d3e4187c128914e58aa84330b9493e3eb68c5e7f"
|
||||
checksum = "0048ad49a55b9deb3953841fa1fc5858f0efbcb7a18868c899a360269fac1b23"
|
||||
dependencies = [
|
||||
"futures",
|
||||
"js-sys",
|
||||
"parking_lot 0.11.2",
|
||||
"parking_lot",
|
||||
"pin-utils",
|
||||
"slab",
|
||||
"wasm-bindgen",
|
||||
"wasm-bindgen-futures",
|
||||
"web-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -6250,7 +6200,7 @@ version = "1.6.0"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6994d13118ab492c3c80c1f81928718159254c53c472bf9ce36f8dae4add02a7"
|
||||
dependencies = [
|
||||
"redox_syscall 0.5.8",
|
||||
"redox_syscall",
|
||||
"wasite",
|
||||
"web-sys",
|
||||
]
|
||||
|
|
|
@ -143,8 +143,8 @@ reflink-copy = { version = "0.1.19" }
|
|||
regex = { version = "1.10.6" }
|
||||
regex-automata = { version = "0.4.8", default-features = false, features = ["dfa-build", "dfa-search", "perf", "std", "syntax"] }
|
||||
reqwest = { version = "=0.12.15", default-features = false, features = ["json", "gzip", "deflate", "zstd", "stream", "rustls-tls", "rustls-tls-native-roots", "socks", "multipart", "http2", "blocking"] }
|
||||
reqwest-middleware = { version = "0.4.0", features = ["multipart"] }
|
||||
reqwest-retry = { version = "0.7.0" }
|
||||
reqwest-middleware = { git = "https://github.com/astral-sh/reqwest-middleware", rev = "ad8b9d332d1773fde8b4cd008486de5973e0a3f8", features = ["multipart"] }
|
||||
reqwest-retry = { git = "https://github.com/astral-sh/reqwest-middleware", rev = "ad8b9d332d1773fde8b4cd008486de5973e0a3f8" }
|
||||
rkyv = { version = "0.8.8", features = ["bytecheck"] }
|
||||
rmp-serde = { version = "1.3.0" }
|
||||
rust-netrc = { version = "0.1.2" }
|
||||
|
@ -372,6 +372,10 @@ riscv64gc-unknown-linux-gnu = "2.31"
|
|||
"actions/download-artifact" = "d3f86a106a0bac45b974a628896c90dbdf5c8093" # v4.3.0
|
||||
"actions/attest-build-provenance" = "c074443f1aee8d4aeeae555aebba3282517141b2" #v2.2.3
|
||||
|
||||
[patch.crates-io]
|
||||
reqwest-middleware = { git = "https://github.com/astral-sh/reqwest-middleware", rev = "ad8b9d332d1773fde8b4cd008486de5973e0a3f8" }
|
||||
reqwest-retry = { git = "https://github.com/astral-sh/reqwest-middleware", rev = "ad8b9d332d1773fde8b4cd008486de5973e0a3f8" }
|
||||
|
||||
[workspace.metadata.dist.binaries]
|
||||
"*" = ["uv", "uvx"]
|
||||
# Add "uvw" binary for Windows targets
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
use std::fmt::{Debug, Display, Formatter};
|
||||
use std::time::{Duration, SystemTime};
|
||||
use std::{borrow::Cow, path::Path};
|
||||
|
||||
|
@ -100,44 +99,62 @@ where
|
|||
}
|
||||
}
|
||||
|
||||
/// Either a cached client error or a (user specified) error from the callback
|
||||
/// Dispatch type: Either a cached client error or a (user specified) error from the callback
|
||||
pub enum CachedClientError<CallbackError: std::error::Error + 'static> {
|
||||
Client(Error),
|
||||
Callback(CallbackError),
|
||||
Client {
|
||||
retries: Option<u32>,
|
||||
err: Error,
|
||||
},
|
||||
Callback {
|
||||
retries: Option<u32>,
|
||||
err: CallbackError,
|
||||
},
|
||||
}
|
||||
|
||||
impl<CallbackError: std::error::Error + 'static> Display for CachedClientError<CallbackError> {
|
||||
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
|
||||
impl<CallbackError: std::error::Error + 'static> CachedClientError<CallbackError> {
|
||||
/// Attach the number of retries to the error context.
|
||||
///
|
||||
/// Adds to existing errors if any, in case different layers retried.
|
||||
fn with_retries(self, retries: u32) -> Self {
|
||||
match self {
|
||||
CachedClientError::Client(err) => write!(f, "{err}"),
|
||||
CachedClientError::Callback(err) => write!(f, "{err}"),
|
||||
}
|
||||
CachedClientError::Client {
|
||||
retries: existing_retries,
|
||||
err,
|
||||
} => CachedClientError::Client {
|
||||
retries: Some(existing_retries.unwrap_or_default() + retries),
|
||||
err,
|
||||
},
|
||||
CachedClientError::Callback {
|
||||
retries: existing_retries,
|
||||
err,
|
||||
} => CachedClientError::Callback {
|
||||
retries: Some(existing_retries.unwrap_or_default() + retries),
|
||||
err,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
impl<CallbackError: std::error::Error + 'static> Debug for CachedClientError<CallbackError> {
|
||||
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
|
||||
fn retries(&self) -> Option<u32> {
|
||||
match self {
|
||||
CachedClientError::Client(err) => write!(f, "{err:?}"),
|
||||
CachedClientError::Callback(err) => write!(f, "{err:?}"),
|
||||
}
|
||||
CachedClientError::Client { retries, .. } => *retries,
|
||||
CachedClientError::Callback { retries, .. } => *retries,
|
||||
}
|
||||
}
|
||||
|
||||
impl<CallbackError: std::error::Error + 'static> std::error::Error
|
||||
for CachedClientError<CallbackError>
|
||||
{
|
||||
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
|
||||
fn error(&self) -> &dyn std::error::Error {
|
||||
match self {
|
||||
CachedClientError::Client(err) => Some(err),
|
||||
CachedClientError::Callback(err) => Some(err),
|
||||
CachedClientError::Client { err, .. } => err,
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<CallbackError: std::error::Error + 'static> From<Error> for CachedClientError<CallbackError> {
|
||||
fn from(error: Error) -> Self {
|
||||
Self::Client(error)
|
||||
Self::Client {
|
||||
retries: None,
|
||||
err: error,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -145,15 +162,35 @@ impl<CallbackError: std::error::Error + 'static> From<ErrorKind>
|
|||
for CachedClientError<CallbackError>
|
||||
{
|
||||
fn from(error: ErrorKind) -> Self {
|
||||
Self::Client(error.into())
|
||||
Self::Client {
|
||||
retries: None,
|
||||
err: error.into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<E: Into<Self> + std::error::Error + 'static> From<CachedClientError<E>> for Error {
|
||||
/// Attach retry error context, if there were retries.
|
||||
fn from(error: CachedClientError<E>) -> Self {
|
||||
match error {
|
||||
CachedClientError::Client(error) => error,
|
||||
CachedClientError::Callback(error) => error.into(),
|
||||
CachedClientError::Client {
|
||||
retries: Some(retries),
|
||||
err,
|
||||
} => ErrorKind::RequestWithRetries {
|
||||
source: Box::new(err.into_kind()),
|
||||
retries,
|
||||
}
|
||||
.into(),
|
||||
CachedClientError::Client { retries: None, err } => err,
|
||||
CachedClientError::Callback {
|
||||
retries: Some(retries),
|
||||
err,
|
||||
} => ErrorKind::RequestWithRetries {
|
||||
source: Box::new(err.into().into_kind()),
|
||||
retries,
|
||||
}
|
||||
.into(),
|
||||
CachedClientError::Callback { retries: None, err } => err.into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -385,7 +422,7 @@ impl CachedClient {
|
|||
let data = response_callback(response)
|
||||
.boxed_local()
|
||||
.await
|
||||
.map_err(|err| CachedClientError::Callback(err))?;
|
||||
.map_err(|err| CachedClientError::Callback { retries: None, err })?;
|
||||
let Some(cache_policy) = cache_policy else {
|
||||
return Ok(data.into_target());
|
||||
};
|
||||
|
@ -530,9 +567,21 @@ impl CachedClient {
|
|||
.for_host(&url)
|
||||
.execute(req)
|
||||
.await
|
||||
.map_err(|err| ErrorKind::from_reqwest_middleware(url.clone(), err))?
|
||||
.error_for_status()
|
||||
.map_err(|err| ErrorKind::from_reqwest(url.clone(), err))?;
|
||||
.map_err(|err| ErrorKind::from_reqwest_middleware(url.clone(), err))?;
|
||||
|
||||
let retry_count = response
|
||||
.extensions()
|
||||
.get::<reqwest_retry::RetryCount>()
|
||||
.map(|retries| retries.value());
|
||||
|
||||
if let Err(status_error) = response.error_for_status_ref() {
|
||||
return Err(CachedClientError::<Error>::Client {
|
||||
retries: retry_count,
|
||||
err: ErrorKind::from_reqwest(url, status_error).into(),
|
||||
}
|
||||
.into());
|
||||
}
|
||||
|
||||
let cache_policy = cache_policy_builder.build(&response);
|
||||
let cache_policy = if cache_policy.to_archived().is_storable() {
|
||||
Some(Box::new(cache_policy))
|
||||
|
@ -579,7 +628,7 @@ impl CachedClient {
|
|||
cache_control: CacheControl,
|
||||
response_callback: Callback,
|
||||
) -> Result<Payload::Target, CachedClientError<CallBackError>> {
|
||||
let mut n_past_retries = 0;
|
||||
let mut past_retries = 0;
|
||||
let start_time = SystemTime::now();
|
||||
let retry_policy = self.uncached().retry_policy();
|
||||
loop {
|
||||
|
@ -587,11 +636,20 @@ impl CachedClient {
|
|||
let result = self
|
||||
.get_cacheable(fresh_req, cache_entry, cache_control, &response_callback)
|
||||
.await;
|
||||
|
||||
// Check if the middleware already performed retries
|
||||
let middleware_retries = match &result {
|
||||
Err(err) => err.retries().unwrap_or_default(),
|
||||
Ok(_) => 0,
|
||||
};
|
||||
|
||||
if result
|
||||
.as_ref()
|
||||
.is_err_and(|err| is_extended_transient_error(err))
|
||||
.is_err_and(|err| is_extended_transient_error(err.error()))
|
||||
{
|
||||
let retry_decision = retry_policy.should_retry(start_time, n_past_retries);
|
||||
// If middleware already retried, consider that in our retry budget
|
||||
let total_retries = past_retries + middleware_retries;
|
||||
let retry_decision = retry_policy.should_retry(start_time, total_retries);
|
||||
if let reqwest_retry::RetryDecision::Retry { execute_after } = retry_decision {
|
||||
debug!(
|
||||
"Transient failure while handling response from {}; retrying...",
|
||||
|
@ -601,10 +659,15 @@ impl CachedClient {
|
|||
.duration_since(SystemTime::now())
|
||||
.unwrap_or_else(|_| Duration::default());
|
||||
tokio::time::sleep(duration).await;
|
||||
n_past_retries += 1;
|
||||
past_retries += 1;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if past_retries > 0 {
|
||||
return result.map_err(|err| err.with_retries(past_retries));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
@ -622,7 +685,7 @@ impl CachedClient {
|
|||
cache_entry: &CacheEntry,
|
||||
response_callback: Callback,
|
||||
) -> Result<Payload, CachedClientError<CallBackError>> {
|
||||
let mut n_past_retries = 0;
|
||||
let mut past_retries = 0;
|
||||
let start_time = SystemTime::now();
|
||||
let retry_policy = self.uncached().retry_policy();
|
||||
loop {
|
||||
|
@ -630,12 +693,20 @@ impl CachedClient {
|
|||
let result = self
|
||||
.skip_cache(fresh_req, cache_entry, &response_callback)
|
||||
.await;
|
||||
|
||||
// Check if the middleware already performed retries
|
||||
let middleware_retries = match &result {
|
||||
Err(err) => err.retries().unwrap_or_default(),
|
||||
_ => 0,
|
||||
};
|
||||
|
||||
if result
|
||||
.as_ref()
|
||||
.err()
|
||||
.is_some_and(|err| is_extended_transient_error(err))
|
||||
.is_some_and(|err| is_extended_transient_error(err.error()))
|
||||
{
|
||||
let retry_decision = retry_policy.should_retry(start_time, n_past_retries);
|
||||
let total_retries = past_retries + middleware_retries;
|
||||
let retry_decision = retry_policy.should_retry(start_time, total_retries);
|
||||
if let reqwest_retry::RetryDecision::Retry { execute_after } = retry_decision {
|
||||
debug!(
|
||||
"Transient failure while handling response from {}; retrying...",
|
||||
|
@ -645,10 +716,15 @@ impl CachedClient {
|
|||
.duration_since(SystemTime::now())
|
||||
.unwrap_or_else(|_| Duration::default());
|
||||
tokio::time::sleep(duration).await;
|
||||
n_past_retries += 1;
|
||||
past_retries += 1;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if past_retries > 0 {
|
||||
return result.map_err(|err| err.with_retries(past_retries));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
|
|
@ -197,6 +197,13 @@ pub enum ErrorKind {
|
|||
#[error("Failed to fetch: `{0}`")]
|
||||
WrappedReqwestError(DisplaySafeUrl, #[source] WrappedReqwestError),
|
||||
|
||||
/// Add the number of failed retries to the error.
|
||||
#[error("Request failed after {retries} retries")]
|
||||
RequestWithRetries {
|
||||
source: Box<ErrorKind>,
|
||||
retries: u32,
|
||||
},
|
||||
|
||||
#[error("Received some unexpected JSON from {}", url)]
|
||||
BadJson {
|
||||
source: serde_json::Error,
|
||||
|
|
|
@ -246,7 +246,7 @@ impl<'a> FlatIndexClient<'a> {
|
|||
.collect();
|
||||
Ok(FlatIndexEntries::from_entries(files))
|
||||
}
|
||||
Err(CachedClientError::Client(err)) if err.is_offline() => {
|
||||
Err(CachedClientError::Client { err, .. }) if err.is_offline() => {
|
||||
Ok(FlatIndexEntries::offline())
|
||||
}
|
||||
Err(err) => Err(err.into()),
|
||||
|
|
|
@ -41,10 +41,7 @@ use crate::flat_index::FlatIndexEntry;
|
|||
use crate::html::SimpleHtml;
|
||||
use crate::remote_metadata::wheel_metadata_from_remote_zip;
|
||||
use crate::rkyvutil::OwnedArchive;
|
||||
use crate::{
|
||||
BaseClient, CachedClient, CachedClientError, Error, ErrorKind, FlatIndexClient,
|
||||
FlatIndexEntries,
|
||||
};
|
||||
use crate::{BaseClient, CachedClient, Error, ErrorKind, FlatIndexClient, FlatIndexEntries};
|
||||
|
||||
/// A builder for an [`RegistryClient`].
|
||||
#[derive(Debug, Clone)]
|
||||
|
@ -607,18 +604,16 @@ impl RegistryClient {
|
|||
.boxed_local()
|
||||
.instrument(info_span!("parse_simple_api", package = %package_name))
|
||||
};
|
||||
self.cached_client()
|
||||
let simple = self
|
||||
.cached_client()
|
||||
.get_cacheable_with_retry(
|
||||
simple_request,
|
||||
cache_entry,
|
||||
cache_control,
|
||||
parse_simple_response,
|
||||
)
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Client(err) => err,
|
||||
CachedClientError::Callback(err) => err,
|
||||
})
|
||||
.await?;
|
||||
Ok(simple)
|
||||
}
|
||||
|
||||
/// Fetch the [`SimpleMetadata`] from a local file, using a PEP 503-compatible directory
|
||||
|
@ -900,15 +895,13 @@ impl RegistryClient {
|
|||
.map_err(|err| ErrorKind::AsyncHttpRangeReader(url.clone(), err))?;
|
||||
trace!("Getting metadata for {filename} by range request");
|
||||
let text = wheel_metadata_from_remote_zip(filename, url, &mut reader).await?;
|
||||
let metadata =
|
||||
ResolutionMetadata::parse_metadata(text.as_bytes()).map_err(|err| {
|
||||
Error::from(ErrorKind::MetadataParseError(
|
||||
filename.clone(),
|
||||
url.to_string(),
|
||||
Box::new(err),
|
||||
))
|
||||
})?;
|
||||
Ok::<ResolutionMetadata, CachedClientError<Error>>(metadata)
|
||||
})
|
||||
}
|
||||
.boxed_local()
|
||||
.instrument(info_span!("read_metadata_range_request", wheel = %filename))
|
||||
|
|
|
@ -644,8 +644,8 @@ impl<'a, Context: BuildContext> DistributionDatabase<'a, Context> {
|
|||
})
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})?;
|
||||
|
||||
// If the archive is missing the required hashes, or has since been removed, force a refresh.
|
||||
|
@ -663,8 +663,8 @@ impl<'a, Context: BuildContext> DistributionDatabase<'a, Context> {
|
|||
.skip_cache_with_retry(self.request(url)?, &http_entry, download)
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})
|
||||
})
|
||||
.await?
|
||||
|
@ -811,8 +811,8 @@ impl<'a, Context: BuildContext> DistributionDatabase<'a, Context> {
|
|||
})
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})?;
|
||||
|
||||
// If the archive is missing the required hashes, or has since been removed, force a refresh.
|
||||
|
@ -830,8 +830,8 @@ impl<'a, Context: BuildContext> DistributionDatabase<'a, Context> {
|
|||
.skip_cache_with_retry(self.request(url)?, &http_entry, download)
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})
|
||||
})
|
||||
.await?
|
||||
|
|
|
@ -728,8 +728,8 @@ impl<'a, T: BuildContext> SourceDistributionBuilder<'a, T> {
|
|||
})
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})?;
|
||||
|
||||
// If the archive is missing the required hashes, force a refresh.
|
||||
|
@ -747,8 +747,8 @@ impl<'a, T: BuildContext> SourceDistributionBuilder<'a, T> {
|
|||
)
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})
|
||||
})
|
||||
.await
|
||||
|
@ -2084,8 +2084,8 @@ impl<'a, T: BuildContext> SourceDistributionBuilder<'a, T> {
|
|||
)
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
CachedClientError::Callback(err) => err,
|
||||
CachedClientError::Client(err) => Error::Client(err),
|
||||
CachedClientError::Callback { err, .. } => err,
|
||||
CachedClientError::Client { err, .. } => Error::Client(err),
|
||||
})
|
||||
})
|
||||
.await
|
||||
|
|
|
@ -53,6 +53,12 @@ pub enum Error {
|
|||
TooManyParts(String),
|
||||
#[error("Failed to download {0}")]
|
||||
NetworkError(DisplaySafeUrl, #[source] WrappedReqwestError),
|
||||
#[error("Request failed after {retries} retries")]
|
||||
NetworkErrorWithRetries {
|
||||
#[source]
|
||||
err: Box<Error>,
|
||||
retries: u32,
|
||||
},
|
||||
#[error("Failed to download {0}")]
|
||||
NetworkMiddlewareError(DisplaySafeUrl, #[source] anyhow::Error),
|
||||
#[error("Failed to extract archive: {0}")]
|
||||
|
@ -1143,8 +1149,20 @@ fn parse_json_downloads(
|
|||
}
|
||||
|
||||
impl Error {
|
||||
pub(crate) fn from_reqwest(url: DisplaySafeUrl, err: reqwest::Error) -> Self {
|
||||
Self::NetworkError(url, WrappedReqwestError::from(err))
|
||||
pub(crate) fn from_reqwest(
|
||||
url: DisplaySafeUrl,
|
||||
err: reqwest::Error,
|
||||
retries: Option<u32>,
|
||||
) -> Self {
|
||||
let err = Self::NetworkError(url, WrappedReqwestError::from(err));
|
||||
if let Some(retries) = retries {
|
||||
Self::NetworkErrorWithRetries {
|
||||
err: Box::new(err),
|
||||
retries,
|
||||
}
|
||||
} else {
|
||||
err
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn from_reqwest_middleware(
|
||||
|
@ -1260,10 +1278,15 @@ async fn read_url(
|
|||
.await
|
||||
.map_err(|err| Error::from_reqwest_middleware(url.clone(), err))?;
|
||||
|
||||
// Ensure the request was successful.
|
||||
response
|
||||
.error_for_status_ref()
|
||||
.map_err(|err| Error::from_reqwest(url, err))?;
|
||||
let retry_count = response
|
||||
.extensions()
|
||||
.get::<reqwest_retry::RetryCount>()
|
||||
.map(|retries| retries.value());
|
||||
|
||||
// Check the status code.
|
||||
let response = response
|
||||
.error_for_status()
|
||||
.map_err(|err| Error::from_reqwest(url, err, retry_count))?;
|
||||
|
||||
let size = response.content_length();
|
||||
let stream = response
|
||||
|
|
|
@ -11477,7 +11477,8 @@ async fn add_unexpected_error_code() -> Result<()> {
|
|||
----- stdout -----
|
||||
|
||||
----- stderr -----
|
||||
error: Failed to fetch: `http://[LOCALHOST]/anyio/`
|
||||
error: Request failed after 3 retries
|
||||
Caused by: Failed to fetch: `http://[LOCALHOST]/anyio/`
|
||||
Caused by: HTTP status server error (503 Service Unavailable) for url (http://[LOCALHOST]/anyio/)
|
||||
"
|
||||
);
|
||||
|
|
|
@ -54,7 +54,8 @@ async fn simple_http_500() {
|
|||
----- stdout -----
|
||||
|
||||
----- stderr -----
|
||||
error: Failed to fetch: `[SERVER]/tqdm/`
|
||||
error: Request failed after 3 retries
|
||||
Caused by: Failed to fetch: `[SERVER]/tqdm/`
|
||||
Caused by: HTTP status server error (500 Internal Server Error) for url ([SERVER]/tqdm/)
|
||||
");
|
||||
}
|
||||
|
@ -105,6 +106,7 @@ async fn find_links_http_500() {
|
|||
|
||||
----- stderr -----
|
||||
error: Failed to read `--find-links` URL: [SERVER]/
|
||||
Caused by: Request failed after 3 retries
|
||||
Caused by: Failed to fetch: `[SERVER]/`
|
||||
Caused by: HTTP status server error (500 Internal Server Error) for url ([SERVER]/)
|
||||
");
|
||||
|
@ -159,6 +161,7 @@ async fn direct_url_http_500() {
|
|||
|
||||
----- stderr -----
|
||||
× Failed to download `tqdm @ [SERVER]/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl`
|
||||
├─▶ Request failed after 3 retries
|
||||
├─▶ Failed to fetch: `[SERVER]/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl`
|
||||
╰─▶ HTTP status server error (500 Internal Server Error) for url ([SERVER]/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl)
|
||||
");
|
||||
|
@ -243,6 +246,7 @@ async fn python_install_http_500() {
|
|||
|
||||
----- stderr -----
|
||||
error: Failed to install cpython-3.10.0-macos-aarch64-none
|
||||
Caused by: Request failed after 3 retries
|
||||
Caused by: Failed to download [SERVER]/astral-sh/python-build-standalone/releases/download/20211017/cpython-3.10.0-aarch64-apple-darwin-pgo%2Blto-20211017T1616.tar.zst
|
||||
Caused by: HTTP status server error (500 Internal Server Error) for url ([SERVER]/astral-sh/python-build-standalone/releases/download/20211017/cpython-3.10.0-aarch64-apple-darwin-pgo%2Blto-20211017T1616.tar.zst)
|
||||
");
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue