Redact credentials when displaying URLs (#13333)
Some checks are pending
CI / integration test | free-threaded on linux (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | free-threaded python on github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | graalpy on ubuntu (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions

This PR redacts credentials in displayed URLs. 

It mostly relies on a `redacted_url` function (and where possible
`IndexUrl::redacted`). This is a quick way to prevent leaked credentials
but it's prone to programmer error when adding new trace statements. A
better follow-on would use a `RedactedUrl` type with the appropriate
`Display` implementation. This would allow us to still extract
credentials from the URL while displaying it securely. On the plus side,
the sites where the `redacted_url` function are used serve as easy
signposts for where to use the new type in a future PR.

Closes #1714.
This commit is contained in:
John Mumm 2025-05-12 17:58:25 +01:00 committed by GitHub
parent 1afadda819
commit 6df588bb00
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 153 additions and 8 deletions

11
Cargo.lock generated
View file

@ -4664,6 +4664,7 @@ dependencies = [
"uv-publish",
"uv-pypi-types",
"uv-python",
"uv-redacted",
"uv-requirements",
"uv-requirements-txt",
"uv-resolver",
@ -4950,6 +4951,7 @@ dependencies = [
"uv-pep508",
"uv-platform-tags",
"uv-pypi-types",
"uv-redacted",
"uv-small-str",
"uv-static",
"uv-torch",
@ -5250,6 +5252,7 @@ dependencies = [
"uv-cache-key",
"uv-fs",
"uv-git-types",
"uv-redacted",
"uv-static",
"uv-version",
"which",
@ -5263,6 +5266,7 @@ dependencies = [
"thiserror 2.0.12",
"tracing",
"url",
"uv-redacted",
]
[[package]]
@ -5597,6 +5601,13 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "uv-redacted"
version = "0.0.1"
dependencies = [
"url",
]
[[package]]
name = "uv-requirements"
version = "0.1.0"

View file

@ -53,6 +53,7 @@ uv-platform-tags = { path = "crates/uv-platform-tags" }
uv-publish = { path = "crates/uv-publish" }
uv-pypi-types = { path = "crates/uv-pypi-types" }
uv-python = { path = "crates/uv-python" }
uv-redacted = { path = "crates/uv-redacted" }
uv-requirements = { path = "crates/uv-requirements" }
uv-requirements-txt = { path = "crates/uv-requirements-txt" }
uv-resolver = { path = "crates/uv-resolver" }

View file

@ -259,6 +259,8 @@ impl From<(Realm, Username)> for RealmUsername {
#[cfg(test)]
mod tests {
use crate::credentials::Password;
use super::*;
#[test]
@ -331,4 +333,23 @@ mod tests {
let url = Url::parse("https://example.com/foobar").unwrap();
assert_eq!(trie.get(&url), None);
}
#[test]
fn test_url_with_credentials() {
let username = Username::new(Some(String::from("username")));
let password = Password::new(String::from("password"));
let credentials = Arc::new(Credentials::Basic {
username: username.clone(),
password: Some(password),
});
let cache = CredentialsCache::default();
// Insert with URL with credentials and get with redacted URL.
let url = Url::parse("https://username:password@example.com/foobar").unwrap();
cache.insert(&url, credentials.clone());
assert_eq!(cache.get_url(&url, &username), Some(credentials.clone()));
// Insert with redacted URL and get with URL with credentials.
let url = Url::parse("https://username:password@second-example.com/foobar").unwrap();
cache.insert(&url, credentials.clone());
assert_eq!(cache.get_url(&url, &username), Some(credentials.clone()));
}
}

View file

@ -24,6 +24,7 @@ uv-pep508 = { workspace = true }
uv-platform-tags = { workspace = true }
uv-pypi-types = { workspace = true }
uv-small-str = { workspace = true }
uv-redacted = { workspace = true }
uv-static = { workspace = true }
uv-torch = { workspace = true }
uv-version = { workspace = true }

View file

@ -7,6 +7,7 @@ use url::Url;
use uv_distribution_filename::{WheelFilename, WheelFilenameError};
use uv_normalize::PackageName;
use uv_redacted::redacted_url;
use crate::middleware::OfflineError;
use crate::{html, FlatIndexError};
@ -197,10 +198,10 @@ pub enum ErrorKind {
#[error("Failed to fetch: `{0}`")]
WrappedReqwestError(Url, #[source] WrappedReqwestError),
#[error("Received some unexpected JSON from {url}")]
#[error("Received some unexpected JSON from {}", redacted_url(url))]
BadJson { source: serde_json::Error, url: Url },
#[error("Received some unexpected HTML from {url}")]
#[error("Received some unexpected HTML from {}", redacted_url(url))]
BadHtml { source: html::Error, url: Url },
#[error("Failed to read zip with range requests: `{0}`")]

View file

@ -10,6 +10,7 @@ use uv_cache_key::cache_digest;
use uv_distribution_filename::DistFilename;
use uv_distribution_types::{File, FileLocation, IndexUrl, UrlString};
use uv_pypi_types::HashDigests;
use uv_redacted::redacted_url;
use uv_small_str::SmallString;
use crate::cached_client::{CacheControl, CachedClientError};
@ -207,7 +208,7 @@ impl<'a> FlatIndexClient<'a> {
Ok(file) => Some(file),
Err(err) => {
// Ignore files with unparsable version specifiers.
warn!("Skipping file in {url}: {err}");
warn!("Skipping file in {}: {err}", redacted_url(&url));
None
}
}

View file

@ -31,6 +31,7 @@ use uv_pep440::Version;
use uv_pep508::MarkerEnvironment;
use uv_platform_tags::Platform;
use uv_pypi_types::{ResolutionMetadata, SimpleJson};
use uv_redacted::redacted_url;
use uv_small_str::SmallString;
use uv_torch::TorchStrategy;
@ -484,7 +485,10 @@ impl RegistryClient {
// ref https://github.com/servo/rust-url/issues/333
.push("");
trace!("Fetching metadata for {package_name} from {url}");
trace!(
"Fetching metadata for {package_name} from {}",
redacted_url(&url)
);
let cache_entry = self.cache.entry(
CacheBucket::Simple,

View file

@ -20,3 +20,5 @@ serde = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
url = { workspace = true }
uv-redacted = { workspace = true }

View file

@ -5,6 +5,8 @@ pub use crate::reference::GitReference;
use thiserror::Error;
use url::Url;
use uv_redacted::redacted_url;
mod github;
mod oid;
mod reference;
@ -151,6 +153,6 @@ impl From<GitUrl> for Url {
impl std::fmt::Display for GitUrl {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.repository)
write!(f, "{}", redacted_url(&self.repository))
}
}

View file

@ -20,6 +20,7 @@ uv-auth = { workspace = true }
uv-cache-key = { workspace = true }
uv-fs = { workspace = true, features = ["tokio"] }
uv-git-types = { workspace = true }
uv-redacted = { workspace = true }
uv-static = { workspace = true }
uv-version = { workspace = true }

View file

@ -4,6 +4,7 @@ use tracing::trace;
use url::Url;
use uv_auth::Credentials;
use uv_cache_key::RepositoryUrl;
use uv_redacted::redacted_url;
/// Global authentication cache for a uv invocation.
///
@ -31,7 +32,7 @@ impl GitStore {
/// Returns `true` if the store was updated.
pub fn store_credentials_from_url(url: &Url) -> bool {
if let Some(credentials) = Credentials::from_url(url) {
trace!("Caching credentials for {url}");
trace!("Caching credentials for {}", redacted_url(url));
GIT_STORE.insert(RepositoryUrl::new(url), credentials);
true
} else {

View file

@ -13,6 +13,7 @@ use url::Url;
use uv_cache_key::{cache_digest, RepositoryUrl};
use uv_git_types::GitUrl;
use uv_redacted::redacted_url;
use crate::git::GitRemote;
use crate::GIT_STORE;
@ -100,7 +101,10 @@ impl GitSource {
// situation that we have a locked revision but the database
// doesn't have it.
(locked_rev, db) => {
debug!("Updating Git source `{}`", self.git.repository());
debug!(
"Updating Git source `{}`",
redacted_url(self.git.repository())
);
// Report the checkout operation to the reporter.
let task = self.reporter.as_ref().map(|reporter| {

View file

@ -0,0 +1,19 @@
[package]
name = "uv-redacted"
version = "0.0.1"
edition = { workspace = true }
rust-version = { workspace = true }
homepage = { workspace = true }
documentation = { workspace = true }
repository = { workspace = true }
authors = { workspace = true }
license = { workspace = true }
[lib]
doctest = false
[lints]
workspace = true
[dependencies]
url = { workspace = true }

View file

@ -0,0 +1,72 @@
use std::borrow::Cow;
use url::Url;
/// Return a version of the URL with redacted credentials, allowing the generic `git` username (without a password)
/// in SSH URLs, as in, `ssh://git@github.com/...`.
pub fn redacted_url(url: &Url) -> Cow<'_, Url> {
if url.username().is_empty() && url.password().is_none() {
return Cow::Borrowed(url);
}
if url.scheme() == "ssh" && url.username() == "git" && url.password().is_none() {
return Cow::Borrowed(url);
}
let mut url = url.clone();
let _ = url.set_username("");
let _ = url.set_password(None);
Cow::Owned(url)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn from_url_no_credentials() {
let url = Url::parse("https://pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn from_url_username_and_password() {
let url = Url::parse("https://user:pass@pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn from_url_just_password() {
let url = Url::parse("https://:pass@pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn from_url_just_username() {
let url = Url::parse("https://user@pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
);
}
}

View file

@ -1515,7 +1515,7 @@ impl std::fmt::Display for PubGrubHint {
"hint".bold().cyan(),
":".bold(),
name.cyan(),
found_index.cyan(),
found_index.redacted().cyan(),
PackageRange::compatibility(&PubGrubPackage::base(name), range, None).cyan(),
next_index.cyan(),
"--index-strategy unsafe-best-match".green(),

View file

@ -43,6 +43,7 @@ uv-platform-tags = { workspace = true }
uv-publish = { workspace = true }
uv-pypi-types = { workspace = true }
uv-python = { workspace = true, features = ["schemars"] }
uv-redacted = { workspace = true }
uv-requirements = { workspace = true }
uv-requirements-txt = { workspace = true }
uv-resolver = { workspace = true }

View file

@ -8,6 +8,7 @@ use indicatif::{MultiProgress, ProgressBar, ProgressStyle};
use owo_colors::OwoColorize;
use rustc_hash::FxHashMap;
use url::Url;
use uv_redacted::redacted_url;
use crate::commands::human_readable_bytes;
use crate::printer::Printer;
@ -300,6 +301,7 @@ impl ProgressReporter {
}
fn on_checkout_start(&self, url: &Url, rev: &str) -> usize {
let url = redacted_url(url);
let ProgressMode::Multi {
multi_progress,
state,
@ -330,6 +332,7 @@ impl ProgressReporter {
}
fn on_checkout_complete(&self, url: &Url, rev: &str, id: usize) {
let url = redacted_url(url);
let ProgressMode::Multi {
state,
multi_progress,