Add support for SSL_CERT_DIR (#16473)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | aarch64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux aarch64 (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / build binary | msrv (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux aarch64 (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | activate nushell venv (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | windows python install manager (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | pyodide on ubuntu (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | registries (push) Blocked by required conditions
CI / integration test | aarch64 windows implicit (push) Blocked by required conditions
CI / integration test | aarch64 windows explicit (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | free-threaded python on github actions (push) Blocked by required conditions
CI / integration test | pyenv on wsl x86-64 (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on rocky linux 10 (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | graalpy on ubuntu (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | x86-64 python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | aarch64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks | walltime aarch64 linux (push) Blocked by required conditions
CI / benchmarks | instrumented (push) Blocked by required conditions
zizmor / Run zizmor (push) Waiting to run

## Summary

Closes https://github.com/astral-sh/uv/issues/16414

Adds support for the standard
[SSL_CERT_DIR](https://docs.openssl.org/3.6/man3/SSL_CTX_load_verify_locations)
which has gained recent proper support from
[rustls-native-certs](https://github.com/rustls/rustls-native-certs/pull/187)
in v0.8.2.

In addition, this PR clarifies documentation around `SSL_CERT_FILE` and
`SSL_CERT_DIR` when used in combination with `UV_NATIVE_TLS` as
mentioned in
https://github.com/astral-sh/uv/issues/16412#issuecomment-3434927201

## Test Plan

Manually tested with custom cert chains in multiple directories and
loading them via SSL_CERT_DIR. We didn't have tests for `SSL_CERT_FILE`
or `SSL_CERT_DIR` environment variables so I added a basic one using our
own test-only certificate generation and dummy https server. I also
moved some things around for better reuse.
This commit is contained in:
samypr100 2025-11-16 12:48:31 -05:00 committed by GitHub
parent b9826778b9
commit bf99f0a195
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 821 additions and 79 deletions

7
Cargo.lock generated
View file

@ -3844,9 +3844,9 @@ dependencies = [
[[package]]
name = "rustls-native-certs"
version = "0.8.1"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7fcff2dd52b58a8d98a70243663a0d234c4e2b79235637849d15913394a247d3"
checksum = "9980d917ebb0c0536119ba501e90834767bffc3d60641457fd84a1f3fd337923"
dependencies = [
"openssl-probe",
"rustls-pki-types",
@ -5678,16 +5678,19 @@ dependencies = [
"itertools 0.14.0",
"jiff",
"percent-encoding",
"rcgen",
"reqwest",
"rkyv",
"rmp-serde",
"rustc-hash",
"rustls",
"serde",
"serde_json",
"sys-info",
"tempfile",
"thiserror 2.0.17",
"tokio",
"tokio-rustls",
"tokio-util",
"tracing",
"url",

View file

@ -211,15 +211,17 @@ byteorder = { version = "1.5.0" }
filetime = { version = "0.2.25" }
http-body-util = { version = "0.1.2" }
hyper = { version = "1.4.1", features = ["server", "http1"] }
hyper-util = { version = "0.1.8", features = ["tokio"] }
hyper-util = { version = "0.1.8", features = ["tokio", "server", "http1"] }
ignore = { version = "0.4.23" }
insta = { version = "1.40.0", features = ["json", "filters", "redactions"] }
predicates = { version = "3.1.2" }
rcgen = { version = "0.14.5", features = ["crypto", "pem", "ring"], default-features = false }
rustls = { version = "0.23.29", default-features = false }
similar = { version = "2.6.0" }
temp-env = { version = "0.3.6" }
test-case = { version = "3.3.1" }
test-log = { version = "0.2.16", features = ["trace"], default-features = false }
tokio-rustls = { version = "0.26.2", default-features = false }
whoami = { version = "1.6.0" }
[workspace.metadata.cargo-shear]

View file

@ -72,6 +72,9 @@ http-body-util = { workspace = true }
hyper = { workspace = true }
hyper-util = { workspace = true }
insta = { workspace = true }
rcgen = { workspace = true }
rustls = { workspace = true }
tokio = { workspace = true }
tokio-rustls = { workspace = true }
wiremock = { workspace = true }
tempfile = { workspace = true }

View file

@ -363,7 +363,9 @@ impl<'a> BaseClientBuilder<'a> {
let _ = write!(user_agent_string, " {output}");
}
// Check for the presence of an `SSL_CERT_FILE`.
// Checks for the presence of `SSL_CERT_FILE`.
// Certificate loading support is delegated to `rustls-native-certs`.
// See https://github.com/rustls/rustls-native-certs/blob/813790a297ad4399efe70a8e5264ca1b420acbec/src/lib.rs#L118-L125
let ssl_cert_file_exists = env::var_os(EnvVars::SSL_CERT_FILE).is_some_and(|path| {
let path_exists = Path::new(&path).exists();
if !path_exists {
@ -375,11 +377,61 @@ impl<'a> BaseClientBuilder<'a> {
path_exists
});
// Checks for the presence of `SSL_CERT_DIR`.
// Certificate loading support is delegated to `rustls-native-certs`.
// See https://github.com/rustls/rustls-native-certs/blob/813790a297ad4399efe70a8e5264ca1b420acbec/src/lib.rs#L118-L125
let ssl_cert_dir_exists = env::var_os(EnvVars::SSL_CERT_DIR)
.filter(|v| !v.is_empty())
.is_some_and(|dirs| {
// Parse `SSL_CERT_DIR`, with support for multiple entries using
// a platform-specific delimiter (`:` on Unix, `;` on Windows)
let (existing, missing): (Vec<_>, Vec<_>) =
env::split_paths(&dirs).partition(|p| p.exists());
if existing.is_empty() {
let end_note = if missing.len() == 1 {
"The directory does not exist."
} else {
"The entries do not exist."
};
warn_user_once!(
"Ignoring invalid `SSL_CERT_DIR`. {end_note}: {}.",
missing
.iter()
.map(Simplified::simplified_display)
.join(", ")
.cyan()
);
return false;
}
// Warn on any missing entries
if !missing.is_empty() {
let end_note = if missing.len() == 1 {
"The following directory does not exist:"
} else {
"The following entries do not exist:"
};
warn_user_once!(
"Invalid entries in `SSL_CERT_DIR`. {end_note}: {}.",
missing
.iter()
.map(Simplified::simplified_display)
.join(", ")
.cyan()
);
}
// Proceed while ignoring missing entries
true
});
// Create a secure client that validates certificates.
let raw_client = self.create_client(
&user_agent_string,
timeout,
ssl_cert_file_exists,
ssl_cert_dir_exists,
Security::Secure,
self.redirect_policy,
);
@ -389,6 +441,7 @@ impl<'a> BaseClientBuilder<'a> {
&user_agent_string,
timeout,
ssl_cert_file_exists,
ssl_cert_dir_exists,
Security::Insecure,
self.redirect_policy,
);
@ -401,6 +454,7 @@ impl<'a> BaseClientBuilder<'a> {
user_agent: &str,
timeout: Duration,
ssl_cert_file_exists: bool,
ssl_cert_dir_exists: bool,
security: Security,
redirect_policy: RedirectPolicy,
) -> Client {
@ -419,7 +473,7 @@ impl<'a> BaseClientBuilder<'a> {
Security::Insecure => client_builder.danger_accept_invalid_certs(true),
};
let client_builder = if self.native_tls || ssl_cert_file_exists {
let client_builder = if self.native_tls || ssl_cert_file_exists || ssl_cert_dir_exists {
client_builder.tls_built_in_native_certs(true)
} else {
client_builder.tls_built_in_webpki_certs(true)

View file

@ -0,0 +1,382 @@
use std::net::SocketAddr;
use std::path::PathBuf;
use std::sync::Arc;
use anyhow::{Context, Result};
use futures::future;
use http_body_util::combinators::BoxBody;
use http_body_util::{BodyExt, Full};
use hyper::body::{Bytes, Incoming};
use hyper::header::USER_AGENT;
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::{TokioExecutor, TokioIo};
use hyper_util::server::conn::auto::Builder;
use rcgen::{
BasicConstraints, Certificate, CertificateParams, DnType, ExtendedKeyUsagePurpose, IsCa,
Issuer, KeyPair, KeyUsagePurpose, SanType, date_time_ymd,
};
use rustls::pki_types::{CertificateDer, PrivateKeyDer};
use rustls::server::WebPkiClientVerifier;
use rustls::{RootCertStore, ServerConfig};
use tokio::net::TcpListener;
use tokio::task::JoinHandle;
use tokio_rustls::TlsAcceptor;
use uv_fs::Simplified;
/// An issued certificate, together with the subject keypair.
#[derive(Debug)]
pub(crate) struct SelfSigned {
/// An issued certificate.
pub public: Certificate,
/// The certificate's subject signing key.
pub private: KeyPair,
}
/// Defines the base location for temporary generated certs.
///
/// See [`TestContext::test_bucket_dir`] for implementation rationale.
pub(crate) fn test_cert_dir() -> PathBuf {
std::env::temp_dir()
.simple_canonicalize()
.expect("failed to canonicalize temp dir")
.join("uv")
.join("tests")
.join("certs")
}
/// Generates a self-signed server certificate for `uv-test-server`, `localhost` and `127.0.0.1`.
/// This certificate is standalone and not issued by a self-signed Root CA.
///
/// Use sparingly as generation of certs is a slow operation.
pub(crate) fn generate_self_signed_certs() -> Result<SelfSigned> {
let mut params = CertificateParams::default();
params.is_ca = IsCa::NoCa;
params.not_before = date_time_ymd(1975, 1, 1);
params.not_after = date_time_ymd(4096, 1, 1);
params.key_usages.push(KeyUsagePurpose::DigitalSignature);
params.key_usages.push(KeyUsagePurpose::KeyEncipherment);
params
.extended_key_usages
.push(ExtendedKeyUsagePurpose::ServerAuth);
params
.distinguished_name
.push(DnType::OrganizationName, "Astral Software Inc.");
params
.distinguished_name
.push(DnType::CommonName, "uv-test-server");
params
.subject_alt_names
.push(SanType::DnsName("uv-test-server".try_into()?));
params
.subject_alt_names
.push(SanType::DnsName("localhost".try_into()?));
params
.subject_alt_names
.push(SanType::IpAddress("127.0.0.1".parse()?));
let private = KeyPair::generate()?;
let public = params.self_signed(&private)?;
Ok(SelfSigned { public, private })
}
/// Generates a self-signed root CA, server certificate, and client certificate.
/// There are no intermediate certs generated as part of this function.
/// The server certificate is for `uv-test-server`, `localhost` and `127.0.0.1` issued by this CA.
/// The client certificate is for `uv-test-client` issued by this CA.
///
/// Use sparingly as generation of these certs is a very slow operation.
pub(crate) fn generate_self_signed_certs_with_ca() -> Result<(SelfSigned, SelfSigned, SelfSigned)> {
// Generate the CA
let mut ca_params = CertificateParams::default();
ca_params.is_ca = IsCa::Ca(BasicConstraints::Unconstrained); // root cert
ca_params.not_before = date_time_ymd(1975, 1, 1);
ca_params.not_after = date_time_ymd(4096, 1, 1);
ca_params.key_usages.push(KeyUsagePurpose::DigitalSignature);
ca_params.key_usages.push(KeyUsagePurpose::KeyCertSign);
ca_params.key_usages.push(KeyUsagePurpose::CrlSign);
ca_params
.distinguished_name
.push(DnType::OrganizationName, "Astral Software Inc.");
ca_params
.distinguished_name
.push(DnType::CommonName, "uv-test-ca");
ca_params
.subject_alt_names
.push(SanType::DnsName("uv-test-ca".try_into()?));
let ca_private_key = KeyPair::generate()?;
let ca_public_cert = ca_params.self_signed(&ca_private_key)?;
let ca_cert_issuer = Issuer::new(ca_params, &ca_private_key);
// Generate server cert issued by this CA
let mut server_params = CertificateParams::default();
server_params.is_ca = IsCa::NoCa;
server_params.not_before = date_time_ymd(1975, 1, 1);
server_params.not_after = date_time_ymd(4096, 1, 1);
server_params.use_authority_key_identifier_extension = true;
server_params
.key_usages
.push(KeyUsagePurpose::DigitalSignature);
server_params
.key_usages
.push(KeyUsagePurpose::KeyEncipherment);
server_params
.extended_key_usages
.push(ExtendedKeyUsagePurpose::ServerAuth);
server_params
.distinguished_name
.push(DnType::OrganizationName, "Astral Software Inc.");
server_params
.distinguished_name
.push(DnType::CommonName, "uv-test-server");
server_params
.subject_alt_names
.push(SanType::DnsName("uv-test-server".try_into()?));
server_params
.subject_alt_names
.push(SanType::DnsName("localhost".try_into()?));
server_params
.subject_alt_names
.push(SanType::IpAddress("127.0.0.1".parse()?));
let server_private_key = KeyPair::generate()?;
let server_public_cert = server_params.signed_by(&server_private_key, &ca_cert_issuer)?;
// Generate client cert issued by this CA
let mut client_params = CertificateParams::default();
client_params.is_ca = IsCa::NoCa;
client_params.not_before = date_time_ymd(1975, 1, 1);
client_params.not_after = date_time_ymd(4096, 1, 1);
client_params.use_authority_key_identifier_extension = true;
client_params
.key_usages
.push(KeyUsagePurpose::DigitalSignature);
client_params
.extended_key_usages
.push(ExtendedKeyUsagePurpose::ClientAuth);
client_params
.distinguished_name
.push(DnType::OrganizationName, "Astral Software Inc.");
client_params
.distinguished_name
.push(DnType::CommonName, "uv-test-client");
client_params
.subject_alt_names
.push(SanType::DnsName("uv-test-client".try_into()?));
let client_private_key = KeyPair::generate()?;
let client_public_cert = client_params.signed_by(&client_private_key, &ca_cert_issuer)?;
let ca_self_signed = SelfSigned {
public: ca_public_cert,
private: ca_private_key,
};
let server_self_signed = SelfSigned {
public: server_public_cert,
private: server_private_key,
};
let client_self_signed = SelfSigned {
public: client_public_cert,
private: client_private_key,
};
Ok((ca_self_signed, server_self_signed, client_self_signed))
}
// Plain is fine for now; Arc/Box could be used later if we need to support move.
type ServerSvcFn =
fn(
Request<Incoming>,
) -> future::Ready<Result<Response<BoxBody<Bytes, hyper::Error>>, hyper::Error>>;
#[derive(Default)]
pub(crate) struct TestServerBuilder<'a> {
// Custom server response function
svc_fn: Option<ServerSvcFn>,
// CA certificate
ca_cert: Option<&'a SelfSigned>,
// Server certificate
server_cert: Option<&'a SelfSigned>,
// Enable mTLS Verification
mutual_tls: bool,
}
impl<'a> TestServerBuilder<'a> {
pub(crate) fn new() -> Self {
Self {
svc_fn: None,
server_cert: None,
ca_cert: None,
mutual_tls: false,
}
}
#[expect(unused)]
/// Provide a custom server response function.
pub(crate) fn with_svc_fn(mut self, svc_fn: ServerSvcFn) -> Self {
self.svc_fn = Some(svc_fn);
self
}
/// Provide the server certificate. This will enable TLS (HTTPS).
pub(crate) fn with_server_cert(mut self, server_cert: &'a SelfSigned) -> Self {
self.server_cert = Some(server_cert);
self
}
/// CA certificate used to build the `RootCertStore` for client verification.
/// Requires `with_server_cert`.
pub(crate) fn with_ca_cert(mut self, ca_cert: &'a SelfSigned) -> Self {
self.ca_cert = Some(ca_cert);
self
}
/// Enforce mutual TLS (client cert auth).
/// Requires `with_server_cert` and `with_ca_cert`.
pub(crate) fn with_mutual_tls(mut self, mutual: bool) -> Self {
self.mutual_tls = mutual;
self
}
/// Starts the HTTP(S) server with optional mTLS enforcement.
pub(crate) async fn start(self) -> Result<(JoinHandle<Result<()>>, SocketAddr)> {
// Validate builder input combinations
if self.ca_cert.is_some() && self.server_cert.is_none() {
anyhow::bail!("server certificate is required when CA certificate is provided");
}
if self.mutual_tls && (self.ca_cert.is_none() || self.server_cert.is_none()) {
anyhow::bail!("ca certificate is required for mTLS");
}
// Set up the TCP listener on a random available port
let listener = TcpListener::bind("127.0.0.1:0").await?;
let addr = listener.local_addr()?;
// Setup TLS Config (if any)
let tls_acceptor = if let Some(server_cert) = self.server_cert {
// Prepare Server Cert and KeyPair
let server_key = PrivateKeyDer::try_from(server_cert.private.serialize_der()).unwrap();
let server_cert = vec![CertificateDer::from(server_cert.public.der().to_vec())];
// Setup CA Verifier
let client_verifier = if let Some(ca_cert) = self.ca_cert {
let mut root_store = RootCertStore::empty();
root_store
.add(CertificateDer::from(ca_cert.public.der().to_vec()))
.expect("failed to add CA cert");
if self.mutual_tls {
// Setup mTLS CA config
WebPkiClientVerifier::builder(root_store.into())
.build()
.expect("failed to setup client verifier")
} else {
// Only load the CA roots
WebPkiClientVerifier::builder(root_store.into())
.allow_unauthenticated()
.build()
.expect("failed to setup client verifier")
}
} else {
WebPkiClientVerifier::no_client_auth()
};
let mut tls_config = ServerConfig::builder()
.with_client_cert_verifier(client_verifier)
.with_single_cert(server_cert, server_key)?;
tls_config.alpn_protocols = vec![b"http/1.1".to_vec(), b"http/1.0".to_vec()];
Some(TlsAcceptor::from(Arc::new(tls_config)))
} else {
None
};
// Setup Response Handler
let svc_fn = if let Some(custom_svc_fn) = self.svc_fn {
custom_svc_fn
} else {
|req: Request<Incoming>| {
// Get User Agent Header and send it back in the response
let user_agent = req
.headers()
.get(USER_AGENT)
.and_then(|v| v.to_str().ok())
.map(ToString::to_string)
.unwrap_or_default(); // Empty Default
let response_content = Full::new(Bytes::from(user_agent))
.map_err(|_| unreachable!())
.boxed();
// If we ever want a true echo server, we can use instead
// let response_content = req.into_body().boxed();
// although uv-client doesn't expose post currently.
future::ok::<_, hyper::Error>(Response::new(response_content))
}
};
// Spawn the server loop in a background task
let server_task = tokio::spawn(async move {
let svc = service_fn(move |req: Request<Incoming>| svc_fn(req));
let (tcp_stream, _remote_addr) = listener
.accept()
.await
.context("Failed to accept TCP connection")?;
// Start Server (not wrapped in loop {} since we want a single response server)
// If we want server to accept multiple connections, we can wrap it in loop {}
// but we'll need to ensure to handle termination signals in the tests otherwise
// it may never stop.
if let Some(tls_acceptor) = tls_acceptor {
let tls_stream = tls_acceptor
.accept(tcp_stream)
.await
.context("Failed to accept TLS connection")?;
let socket = TokioIo::new(tls_stream);
tokio::task::spawn(async move {
Builder::new(TokioExecutor::new())
.serve_connection(socket, svc)
.await
.expect("HTTPS Server Started");
});
} else {
let socket = TokioIo::new(tcp_stream);
tokio::task::spawn(async move {
Builder::new(TokioExecutor::new())
.serve_connection(socket, svc)
.await
.expect("HTTP Server Started");
});
}
Ok(())
});
Ok((server_task, addr))
}
}
/// Single Request HTTP server that echoes the User Agent Header.
pub(crate) async fn start_http_user_agent_server() -> Result<(JoinHandle<Result<()>>, SocketAddr)> {
TestServerBuilder::new().start().await
}
/// Single Request HTTPS server that echoes the User Agent Header.
pub(crate) async fn start_https_user_agent_server(
server_cert: &SelfSigned,
) -> Result<(JoinHandle<Result<()>>, SocketAddr)> {
TestServerBuilder::new()
.with_server_cert(server_cert)
.start()
.await
}
/// Single Request HTTPS mTLS server that echoes the User Agent Header.
pub(crate) async fn start_https_mtls_user_agent_server(
ca_cert: &SelfSigned,
server_cert: &SelfSigned,
) -> Result<(JoinHandle<Result<()>>, SocketAddr)> {
TestServerBuilder::new()
.with_ca_cert(ca_cert)
.with_server_cert(server_cert)
.with_mutual_tls(true)
.start()
.await
}

View file

@ -1,2 +1,4 @@
mod http_util;
mod remote_metadata;
mod ssl_certs;
mod user_agent_version;

View file

@ -21,11 +21,11 @@ async fn remote_metadata_with_and_without_cache() -> Result<()> {
let filename = WheelFilename::from_str(url.rsplit_once('/').unwrap().1)?;
let dist = BuiltDist::DirectUrl(DirectUrlBuiltDist {
filename,
location: Box::new(DisplaySafeUrl::parse(url).unwrap()),
url: VerbatimUrl::from_str(url).unwrap(),
location: Box::new(DisplaySafeUrl::parse(url)?),
url: VerbatimUrl::from_str(url)?,
});
let capabilities = IndexCapabilities::default();
let metadata = client.wheel_metadata(&dist, &capabilities).await.unwrap();
let metadata = client.wheel_metadata(&dist, &capabilities).await?;
assert_eq!(metadata.version.to_string(), "4.66.1");
}

View file

@ -0,0 +1,333 @@
use std::str::FromStr;
use anyhow::Result;
use rustls::AlertDescription;
use url::Url;
use uv_cache::Cache;
use uv_client::BaseClientBuilder;
use uv_client::RegistryClientBuilder;
use uv_redacted::DisplaySafeUrl;
use uv_static::EnvVars;
use crate::http_util::{
generate_self_signed_certs, generate_self_signed_certs_with_ca,
start_https_mtls_user_agent_server, start_https_user_agent_server, test_cert_dir,
};
// SAFETY: This test is meant to run with single thread configuration
#[tokio::test]
#[allow(unsafe_code)]
async fn ssl_env_vars() -> Result<()> {
// Ensure our environment is not polluted with anything that may affect `rustls-native-certs`
unsafe {
std::env::remove_var(EnvVars::UV_NATIVE_TLS);
std::env::remove_var(EnvVars::SSL_CERT_FILE);
std::env::remove_var(EnvVars::SSL_CERT_DIR);
std::env::remove_var(EnvVars::SSL_CLIENT_CERT);
}
// Create temporary cert dirs
let cert_dir = test_cert_dir();
fs_err::create_dir_all(&cert_dir).expect("Failed to create test cert bucket");
let cert_dir =
tempfile::TempDir::new_in(cert_dir).expect("Failed to create test cert directory");
let does_not_exist_cert_dir = cert_dir.path().join("does_not_exist");
// Generate self-signed standalone cert
let standalone_server_cert = generate_self_signed_certs()?;
let standalone_public_pem_path = cert_dir.path().join("standalone_public.pem");
let standalone_private_pem_path = cert_dir.path().join("standalone_private.pem");
// Generate self-signed CA, server, and client certs
let (ca_cert, server_cert, client_cert) = generate_self_signed_certs_with_ca()?;
let ca_public_pem_path = cert_dir.path().join("ca_public.pem");
let ca_private_pem_path = cert_dir.path().join("ca_private.pem");
let server_public_pem_path = cert_dir.path().join("server_public.pem");
let server_private_pem_path = cert_dir.path().join("server_private.pem");
let client_combined_pem_path = cert_dir.path().join("client_combined.pem");
// Persist the certs in PKCS8 format as the env vars expect a path on disk
fs_err::write(
standalone_public_pem_path.as_path(),
standalone_server_cert.public.pem(),
)?;
fs_err::write(
standalone_private_pem_path.as_path(),
standalone_server_cert.private.serialize_pem(),
)?;
fs_err::write(ca_public_pem_path.as_path(), ca_cert.public.pem())?;
fs_err::write(
ca_private_pem_path.as_path(),
ca_cert.private.serialize_pem(),
)?;
fs_err::write(server_public_pem_path.as_path(), server_cert.public.pem())?;
fs_err::write(
server_private_pem_path.as_path(),
server_cert.private.serialize_pem(),
)?;
fs_err::write(
client_combined_pem_path.as_path(),
// SSL_CLIENT_CERT expects a "combined" cert with the public and private key.
format!(
"{}\n{}",
client_cert.public.pem(),
client_cert.private.serialize_pem()
),
)?;
// ** Set SSL_CERT_FILE to non-existent location
// ** Then verify our request fails to establish a connection
unsafe {
std::env::set_var(EnvVars::SSL_CERT_FILE, does_not_exist_cert_dir.as_os_str());
}
let (server_task, addr) = start_https_user_agent_server(&standalone_server_cert).await?;
let url = DisplaySafeUrl::from_str(&format!("https://{addr}"))?;
let cache = Cache::temp()?.init()?;
let client = RegistryClientBuilder::new(BaseClientBuilder::default(), cache).build();
let res = client
.cached_client()
.uncached()
.for_host(&url)
.get(Url::from(url))
.send()
.await;
unsafe {
std::env::remove_var(EnvVars::SSL_CERT_FILE);
}
// Validate the client error
let Some(reqwest_middleware::Error::Middleware(middleware_error)) = res.err() else {
panic!("expected middleware error");
};
let reqwest_error = middleware_error
.chain()
.find_map(|err| {
err.downcast_ref::<reqwest_middleware::Error>().map(|err| {
if let reqwest_middleware::Error::Reqwest(inner) = err {
inner
} else {
panic!("expected reqwest error")
}
})
})
.expect("expected reqwest error");
assert!(reqwest_error.is_connect());
// Validate the server error
let server_res = server_task.await?;
let expected_err = if let Err(anyhow_err) = server_res
&& let Some(io_err) = anyhow_err.downcast_ref::<std::io::Error>()
&& let Some(wrapped_err) = io_err.get_ref()
&& let Some(tls_err) = wrapped_err.downcast_ref::<rustls::Error>()
&& matches!(
tls_err,
rustls::Error::AlertReceived(AlertDescription::UnknownCA)
) {
true
} else {
false
};
assert!(expected_err);
// ** Set SSL_CERT_FILE to our public certificate
// ** Then verify our request successfully establishes a connection
unsafe {
std::env::set_var(
EnvVars::SSL_CERT_FILE,
standalone_public_pem_path.as_os_str(),
);
}
let (server_task, addr) = start_https_user_agent_server(&standalone_server_cert).await?;
let url = DisplaySafeUrl::from_str(&format!("https://{addr}"))?;
let cache = Cache::temp()?.init()?;
let client = RegistryClientBuilder::new(BaseClientBuilder::default(), cache).build();
let res = client
.cached_client()
.uncached()
.for_host(&url)
.get(Url::from(url))
.send()
.await;
assert!(res.is_ok());
let _ = server_task.await?; // wait for server shutdown
unsafe {
std::env::remove_var(EnvVars::SSL_CERT_FILE);
}
// ** Set SSL_CERT_DIR to our cert dir as well as some other dir that does not exist
// ** Then verify our request still successfully establishes a connection
unsafe {
std::env::set_var(
EnvVars::SSL_CERT_DIR,
std::env::join_paths(vec![
cert_dir.path().as_os_str(),
does_not_exist_cert_dir.as_os_str(),
])?,
);
}
let (server_task, addr) = start_https_user_agent_server(&standalone_server_cert).await?;
let url = DisplaySafeUrl::from_str(&format!("https://{addr}"))?;
let cache = Cache::temp()?.init()?;
let client = RegistryClientBuilder::new(BaseClientBuilder::default(), cache).build();
let res = client
.cached_client()
.uncached()
.for_host(&url)
.get(Url::from(url))
.send()
.await;
assert!(res.is_ok());
let _ = server_task.await?; // wait for server shutdown
unsafe {
std::env::remove_var(EnvVars::SSL_CERT_DIR);
}
// ** Set SSL_CERT_DIR to only the dir that does not exist
// ** Then verify our request fails to establish a connection
unsafe {
std::env::set_var(EnvVars::SSL_CERT_DIR, does_not_exist_cert_dir.as_os_str());
}
let (server_task, addr) = start_https_user_agent_server(&standalone_server_cert).await?;
let url = DisplaySafeUrl::from_str(&format!("https://{addr}"))?;
let cache = Cache::temp()?.init()?;
let client = RegistryClientBuilder::new(BaseClientBuilder::default(), cache).build();
let res = client
.cached_client()
.uncached()
.for_host(&url)
.get(Url::from(url))
.send()
.await;
unsafe {
std::env::remove_var(EnvVars::SSL_CERT_DIR);
}
// Validate the client error
let Some(reqwest_middleware::Error::Middleware(middleware_error)) = res.err() else {
panic!("expected middleware error");
};
let reqwest_error = middleware_error
.chain()
.find_map(|err| {
err.downcast_ref::<reqwest_middleware::Error>().map(|err| {
if let reqwest_middleware::Error::Reqwest(inner) = err {
inner
} else {
panic!("expected reqwest error")
}
})
})
.expect("expected reqwest error");
assert!(reqwest_error.is_connect());
// Validate the server error
let server_res = server_task.await?;
let expected_err = if let Err(anyhow_err) = server_res
&& let Some(io_err) = anyhow_err.downcast_ref::<std::io::Error>()
&& let Some(wrapped_err) = io_err.get_ref()
&& let Some(tls_err) = wrapped_err.downcast_ref::<rustls::Error>()
&& matches!(
tls_err,
rustls::Error::AlertReceived(AlertDescription::UnknownCA)
) {
true
} else {
false
};
assert!(expected_err);
// *** mTLS Tests
// ** Set SSL_CERT_FILE to our CA and SSL_CLIENT_CERT to our client cert
// ** Then verify our request still successfully establishes a connection
// We need to set SSL_CERT_FILE or SSL_CERT_DIR to our CA as we need to tell
// our HTTP client that we trust certificates issued by our self-signed CA.
// This inherently also tests that our server cert is also validated as part
// of the certificate path validation algorithm.
unsafe {
std::env::set_var(EnvVars::SSL_CERT_FILE, ca_public_pem_path.as_os_str());
std::env::set_var(
EnvVars::SSL_CLIENT_CERT,
client_combined_pem_path.as_os_str(),
);
}
let (server_task, addr) = start_https_mtls_user_agent_server(&ca_cert, &server_cert).await?;
let url = DisplaySafeUrl::from_str(&format!("https://{addr}"))?;
let cache = Cache::temp()?.init()?;
let client = RegistryClientBuilder::new(BaseClientBuilder::default(), cache).build();
let res = client
.cached_client()
.uncached()
.for_host(&url)
.get(Url::from(url))
.send()
.await;
assert!(res.is_ok());
let _ = server_task.await?; // wait for server shutdown
unsafe {
std::env::remove_var(EnvVars::SSL_CERT_FILE);
std::env::remove_var(EnvVars::SSL_CLIENT_CERT);
}
// ** Set SSL_CERT_FILE to our CA and unset SSL_CLIENT_CERT
// ** Then verify our request fails to establish a connection
unsafe {
std::env::set_var(EnvVars::SSL_CERT_FILE, ca_public_pem_path.as_os_str());
}
let (server_task, addr) = start_https_mtls_user_agent_server(&ca_cert, &server_cert).await?;
let url = DisplaySafeUrl::from_str(&format!("https://{addr}"))?;
let cache = Cache::temp()?.init()?;
let client = RegistryClientBuilder::new(BaseClientBuilder::default(), cache).build();
let res = client
.cached_client()
.uncached()
.for_host(&url)
.get(Url::from(url))
.send()
.await;
unsafe {
std::env::remove_var(EnvVars::SSL_CERT_FILE);
}
// Validate the client error
let Some(reqwest_middleware::Error::Middleware(middleware_error)) = res.err() else {
panic!("expected middleware error");
};
let reqwest_error = middleware_error
.chain()
.find_map(|err| {
err.downcast_ref::<reqwest_middleware::Error>().map(|err| {
if let reqwest_middleware::Error::Reqwest(inner) = err {
inner
} else {
panic!("expected reqwest error")
}
})
})
.expect("expected reqwest error");
assert!(reqwest_error.is_connect());
// Validate the server error
let server_res = server_task.await?;
let expected_err = if let Err(anyhow_err) = server_res
&& let Some(io_err) = anyhow_err.downcast_ref::<std::io::Error>()
&& let Some(wrapped_err) = io_err.get_ref()
&& let Some(tls_err) = wrapped_err.downcast_ref::<rustls::Error>()
&& matches!(tls_err, rustls::Error::NoCertificatesPresented)
{
true
} else {
false
};
assert!(expected_err);
// Fin.
Ok(())
}

View file

@ -1,16 +1,9 @@
use anyhow::Result;
use futures::future;
use http_body_util::Full;
use hyper::body::Bytes;
use hyper::header::USER_AGENT;
use hyper::server::conn::http1;
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::TokioIo;
use insta::{assert_json_snapshot, assert_snapshot, with_settings};
use std::str::FromStr;
use tokio::net::TcpListener;
use anyhow::Result;
use insta::{assert_json_snapshot, assert_snapshot, with_settings};
use url::Url;
use uv_cache::Cache;
use uv_client::RegistryClientBuilder;
use uv_client::{BaseClientBuilder, LineHaul};
@ -19,36 +12,12 @@ use uv_platform_tags::{Arch, Os, Platform};
use uv_redacted::DisplaySafeUrl;
use uv_version::version;
use crate::http_util::start_http_user_agent_server;
#[tokio::test]
async fn test_user_agent_has_version() -> Result<()> {
// Set up the TCP listener on a random available port
let listener = TcpListener::bind("127.0.0.1:0").await?;
let addr = listener.local_addr()?;
// Spawn the server loop in a background task
let server_task = tokio::spawn(async move {
let svc = service_fn(move |req: Request<hyper::body::Incoming>| {
// Get User Agent Header and send it back in the response
let user_agent = req
.headers()
.get(USER_AGENT)
.and_then(|v| v.to_str().ok())
.map(ToString::to_string)
.unwrap_or_default(); // Empty Default
future::ok::<_, hyper::Error>(Response::new(Full::new(Bytes::from(user_agent))))
});
// Start Server (not wrapped in loop {} since we want a single response server)
// If you want server to accept multiple connections, wrap it in loop {}
let (socket, _) = listener.accept().await.unwrap();
let socket = TokioIo::new(socket);
tokio::task::spawn(async move {
http1::Builder::new()
.serve_connection(socket, svc)
.with_upgrades()
.await
.expect("Server Started");
});
});
// Initialize dummy http server
let (server_task, addr) = start_http_user_agent_server().await?;
// Initialize uv-client
let cache = Cache::temp()?.init()?;
@ -94,41 +63,15 @@ async fn test_user_agent_has_version() -> Result<()> {
});
// Wait for the server task to complete, to be a good citizen.
server_task.await?;
let _ = server_task.await?;
Ok(())
}
#[tokio::test]
async fn test_user_agent_has_linehaul() -> Result<()> {
// Set up the TCP listener on a random available port
let listener = TcpListener::bind("127.0.0.1:0").await?;
let addr = listener.local_addr()?;
// Spawn the server loop in a background task
let server_task = tokio::spawn(async move {
let svc = service_fn(move |req: Request<hyper::body::Incoming>| {
// Get User Agent Header and send it back in the response
let user_agent = req
.headers()
.get(USER_AGENT)
.and_then(|v| v.to_str().ok())
.map(ToString::to_string)
.unwrap_or_default(); // Empty Default
future::ok::<_, hyper::Error>(Response::new(Full::new(Bytes::from(user_agent))))
});
// Start Server (not wrapped in loop {} since we want a single response server)
// If you want server to accept multiple connections, wrap it in loop {}
let (socket, _) = listener.accept().await.unwrap();
let socket = TokioIo::new(socket);
tokio::task::spawn(async move {
http1::Builder::new()
.serve_connection(socket, svc)
.with_upgrades()
.await
.expect("Server Started");
});
});
// Initialize dummy http server
let (server_task, addr) = start_http_user_agent_server().await?;
// Add some representative markers for an Ubuntu CI runner
let markers = MarkerEnvironment::try_from(MarkerEnvironmentBuilder {
@ -143,8 +86,7 @@ async fn test_user_agent_has_linehaul() -> Result<()> {
python_full_version: "3.12.2",
python_version: "3.12",
sys_platform: "linux",
})
.unwrap();
})?;
// Initialize uv-client
let cache = Cache::temp()?.init()?;
@ -189,7 +131,7 @@ async fn test_user_agent_has_linehaul() -> Result<()> {
let body = res.text().await?;
// Wait for the server task to complete, to be a good citizen.
server_task.await?;
let _ = server_task.await?;
// Unpack User-Agent with linehaul
let (uv_version, uv_linehaul) = body

View file

@ -586,9 +586,19 @@ impl EnvVars {
pub const XDG_BIN_HOME: &'static str = "XDG_BIN_HOME";
/// Custom certificate bundle file path for SSL connections.
///
/// Takes precedence over `UV_NATIVE_TLS` when set.
#[attr_added_in("0.1.14")]
pub const SSL_CERT_FILE: &'static str = "SSL_CERT_FILE";
/// Custom path for certificate bundles for SSL connections.
/// Multiple entries are supported separated using a platform-specific
/// delimiter (`:` on Unix, `;` on Windows).
///
/// Takes precedence over `UV_NATIVE_TLS` when set.
#[attr_added_in("next release")]
pub const SSL_CERT_DIR: &'static str = "SSL_CERT_DIR";
/// If set, uv will use this file for mTLS authentication.
/// This should be a single file containing both the certificate and the private key in PEM format.
#[attr_added_in("0.2.11")]

View file

@ -1004,11 +1004,22 @@ the fact that Windows' real main thread is only 1MB. That thread has size
The standard `SHELL` posix env var.
### `SSL_CERT_DIR`
<small class="added-in">added in `next release`</small>
Custom path for certificate bundles for SSL connections.
Multiple entries are supported separated using a platform-specific
delimiter (`:` on Unix, `;` on Windows).
Takes precedence over `UV_NATIVE_TLS` when set.
### `SSL_CERT_FILE`
<small class="added-in">added in `0.1.14`</small>
Custom certificate bundle file path for SSL connections.
Takes precedence over `UV_NATIVE_TLS` when set.
### `SSL_CLIENT_CERT`
<small class="added-in">added in `0.2.11`</small>