mirror of
https://github.com/astral-sh/uv.git
synced 2025-08-04 10:58:28 +00:00

Prior to this PR, there were numerous places where uv would leak credentials in logs. We had a way to mask credentials by calling methods or a recently-added `redact_url` function, but this was not secure by default. There were a number of other types (like `GitUrl`) that would leak credentials on display. This PR adds a `DisplaySafeUrl` newtype to prevent leaking credentials when logging by default. It takes a maximalist approach, replacing the use of `Url` almost everywhere. This includes when first parsing config files, when storing URLs in types like `GitUrl`, and also when storing URLs in types that in practice will never contain credentials (like `DirectorySourceUrl`). The idea is to make it easy for developers to do the right thing and for the compiler to support this (and to minimize ever having to manually convert back and forth). Displaying credentials now requires an active step. Note that despite this maximalist approach, the use of the newtype should be zero cost. One conspicuous place this PR does not use `DisplaySafeUrl` is in the `uv-auth` crate. That would require new clones since there are calls to `request.url()` that return a `&Url`. One option would have been to make `DisplaySafeUrl` wrap a `Cow`, but this would lead to lifetime annotations all over the codebase. I've created a separate PR based on this one (#13576) that updates `uv-auth` to use `DisplaySafeUrl` with one new clone. We can discuss the tradeoffs there. Most of this PR just replaces `Url` with `DisplaySafeUrl`. The core is `uv_redacted/lib.rs`, where the newtype is implemented. To make it easier to review the rest, here are some points of note: * `DisplaySafeUrl` has a `Display` implementation that masks credentials. Currently, it will still display the username when there is both a username and password. If we think is the wrong choice, it can now be changed in one place. * `DisplaySafeUrl` has a `remove_credentials()` method and also a `.to_string_with_credentials()` method. This allows us to use it in a variety of scenarios. * `IndexUrl::redacted()` was renamed to `IndexUrl::removed_credentials()` to make it clearer that we are not masking. * We convert from a `DisplaySafeUrl` to a `Url` when calling `reqwest` methods like `.get()` and `.head()`. * We convert from a `DisplaySafeUrl` to a `Url` when creating a `uv_auth::Index`. That is because, as mentioned above, I will be updating the `uv_auth` crate to use this newtype in a separate PR. * A number of tests (e.g., in `pip_install.rs`) that formerly used filters to mask tokens in the test output no longer need those filters since tokens in URLs are now masked automatically. * The one place we are still knowingly writing credentials to `pyproject.toml` is when a URL with credentials is passed to `uv add` with `--raw`. Since displaying credentials is no longer automatic, I have added a `to_string_with_credentials()` method to the `Pep508Url` trait. This is used when `--raw` is passed. Adding it to that trait is a bit weird, but it's the simplest way to achieve the goal. I'm open to suggestions on how to improve this, but note that because of the way we're using generic bounds, it's not as simple as just creating a separate trait for that method.
399 lines
14 KiB
Rust
399 lines
14 KiB
Rust
use std::{io::Write, process::Stdio};
|
|
use tokio::process::Command;
|
|
use tracing::{instrument, trace, warn};
|
|
use uv_redacted::DisplaySafeUrlRef;
|
|
use uv_warnings::warn_user_once;
|
|
|
|
use crate::credentials::Credentials;
|
|
|
|
/// A backend for retrieving credentials from a keyring.
|
|
///
|
|
/// See pip's implementation for reference
|
|
/// <https://github.com/pypa/pip/blob/ae5fff36b0aad6e5e0037884927eaa29163c0611/src/pip/_internal/network/auth.py#L102>
|
|
#[derive(Debug)]
|
|
pub struct KeyringProvider {
|
|
backend: KeyringProviderBackend,
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
pub(crate) enum KeyringProviderBackend {
|
|
/// Use the `keyring` command to fetch credentials.
|
|
Subprocess,
|
|
#[cfg(test)]
|
|
Dummy(Vec<(String, &'static str, &'static str)>),
|
|
}
|
|
|
|
impl KeyringProvider {
|
|
/// Create a new [`KeyringProvider::Subprocess`].
|
|
pub fn subprocess() -> Self {
|
|
Self {
|
|
backend: KeyringProviderBackend::Subprocess,
|
|
}
|
|
}
|
|
|
|
/// Fetch credentials for the given [`Url`] from the keyring.
|
|
///
|
|
/// Returns [`None`] if no password was found for the username or if any errors
|
|
/// are encountered in the keyring backend.
|
|
#[instrument(skip_all, fields(url = % url.to_string(), username))]
|
|
pub async fn fetch(
|
|
&self,
|
|
url: &DisplaySafeUrlRef<'_>,
|
|
username: Option<&str>,
|
|
) -> Option<Credentials> {
|
|
// Validate the request
|
|
debug_assert!(
|
|
url.host_str().is_some(),
|
|
"Should only use keyring for urls with host"
|
|
);
|
|
debug_assert!(
|
|
url.password().is_none(),
|
|
"Should only use keyring for urls without a password"
|
|
);
|
|
debug_assert!(
|
|
!username.map(str::is_empty).unwrap_or(false),
|
|
"Should only use keyring with a non-empty username"
|
|
);
|
|
|
|
// Check the full URL first
|
|
// <https://github.com/pypa/pip/blob/ae5fff36b0aad6e5e0037884927eaa29163c0611/src/pip/_internal/network/auth.py#L376C1-L379C14>
|
|
trace!("Checking keyring for URL {url}");
|
|
let mut credentials = match self.backend {
|
|
KeyringProviderBackend::Subprocess => {
|
|
self.fetch_subprocess(url.as_str(), username).await
|
|
}
|
|
#[cfg(test)]
|
|
KeyringProviderBackend::Dummy(ref store) => {
|
|
Self::fetch_dummy(store, url.as_str(), username)
|
|
}
|
|
};
|
|
// And fallback to a check for the host
|
|
if credentials.is_none() {
|
|
let host = if let Some(port) = url.port() {
|
|
format!("{}:{}", url.host_str()?, port)
|
|
} else {
|
|
url.host_str()?.to_string()
|
|
};
|
|
trace!("Checking keyring for host {host}");
|
|
credentials = match self.backend {
|
|
KeyringProviderBackend::Subprocess => self.fetch_subprocess(&host, username).await,
|
|
#[cfg(test)]
|
|
KeyringProviderBackend::Dummy(ref store) => {
|
|
Self::fetch_dummy(store, &host, username)
|
|
}
|
|
};
|
|
}
|
|
|
|
credentials.map(|(username, password)| Credentials::basic(Some(username), Some(password)))
|
|
}
|
|
|
|
#[instrument(skip(self))]
|
|
async fn fetch_subprocess(
|
|
&self,
|
|
service_name: &str,
|
|
username: Option<&str>,
|
|
) -> Option<(String, String)> {
|
|
// https://github.com/pypa/pip/blob/24.0/src/pip/_internal/network/auth.py#L136-L141
|
|
let mut command = Command::new("keyring");
|
|
command.arg("get").arg(service_name);
|
|
|
|
if let Some(username) = username {
|
|
command.arg(username);
|
|
} else {
|
|
command.arg("--mode").arg("creds");
|
|
}
|
|
|
|
let child = command
|
|
.stdin(Stdio::null())
|
|
.stdout(Stdio::piped())
|
|
// If we're using `--mode creds`, we need to capture the output in order to avoid
|
|
// showing users an "unrecognized arguments: --mode" error; otherwise, we stream stderr
|
|
// so the user has visibility into keyring's behavior if it's doing something slow
|
|
.stderr(if username.is_some() {
|
|
Stdio::inherit()
|
|
} else {
|
|
Stdio::piped()
|
|
})
|
|
.spawn()
|
|
.inspect_err(|err| warn!("Failure running `keyring` command: {err}"))
|
|
.ok()?;
|
|
|
|
let output = child
|
|
.wait_with_output()
|
|
.await
|
|
.inspect_err(|err| warn!("Failed to wait for `keyring` output: {err}"))
|
|
.ok()?;
|
|
|
|
if output.status.success() {
|
|
// If we captured stderr, display it in case it's helpful to the user
|
|
// TODO(zanieb): This was done when we added `--mode creds` support for parity with the
|
|
// existing behavior, but it might be a better UX to hide this on success? It also
|
|
// might be problematic that we're not streaming it. We could change this given some
|
|
// user feedback.
|
|
std::io::stderr().write_all(&output.stderr).ok();
|
|
|
|
// On success, parse the newline terminated credentials
|
|
let output = String::from_utf8(output.stdout)
|
|
.inspect_err(|err| warn!("Failed to parse response from `keyring` command: {err}"))
|
|
.ok()?;
|
|
|
|
let (username, password) = if let Some(username) = username {
|
|
// We're only expecting a password
|
|
let password = output.trim_end();
|
|
(username, password)
|
|
} else {
|
|
// We're expecting a username and password
|
|
let mut lines = output.lines();
|
|
let username = lines.next()?;
|
|
let Some(password) = lines.next() else {
|
|
warn!(
|
|
"Got username without password for `{service_name}` from `keyring` command"
|
|
);
|
|
return None;
|
|
};
|
|
(username, password)
|
|
};
|
|
|
|
if password.is_empty() {
|
|
// We allow this for backwards compatibility, but it might be better to return
|
|
// `None` instead if there's confusion from users — we haven't seen this in practice
|
|
// yet.
|
|
warn!("Got empty password for `{username}@{service_name}` from `keyring` command");
|
|
}
|
|
|
|
Some((username.to_string(), password.to_string()))
|
|
} else {
|
|
// On failure, no password was available
|
|
let stderr = std::str::from_utf8(&output.stderr).ok()?;
|
|
if stderr.contains("unrecognized arguments: --mode") {
|
|
// N.B. We do not show the `service_name` here because we'll show the warning twice
|
|
// otherwise, once for the URL and once for the realm.
|
|
warn_user_once!(
|
|
"Attempted to fetch credentials using the `keyring` command, but it does not support `--mode creds`; upgrade to `keyring>=v25.2.1` for support or provide a username"
|
|
);
|
|
} else if username.is_none() {
|
|
// If we captured stderr, display it in case it's helpful to the user
|
|
std::io::stderr().write_all(&output.stderr).ok();
|
|
}
|
|
None
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
fn fetch_dummy(
|
|
store: &Vec<(String, &'static str, &'static str)>,
|
|
service_name: &str,
|
|
username: Option<&str>,
|
|
) -> Option<(String, String)> {
|
|
store.iter().find_map(|(service, user, password)| {
|
|
if service == service_name && username.is_none_or(|username| username == *user) {
|
|
Some(((*user).to_string(), (*password).to_string()))
|
|
} else {
|
|
None
|
|
}
|
|
})
|
|
}
|
|
|
|
/// Create a new provider with [`KeyringProviderBackend::Dummy`].
|
|
#[cfg(test)]
|
|
pub fn dummy<S: Into<String>, T: IntoIterator<Item = (S, &'static str, &'static str)>>(
|
|
iter: T,
|
|
) -> Self {
|
|
Self {
|
|
backend: KeyringProviderBackend::Dummy(
|
|
iter.into_iter()
|
|
.map(|(service, username, password)| (service.into(), username, password))
|
|
.collect(),
|
|
),
|
|
}
|
|
}
|
|
|
|
/// Create a new provider with no credentials available.
|
|
#[cfg(test)]
|
|
pub fn empty() -> Self {
|
|
Self {
|
|
backend: KeyringProviderBackend::Dummy(Vec::new()),
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use futures::FutureExt;
|
|
use url::Url;
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_no_host() {
|
|
let url = Url::parse("file:/etc/bin/").unwrap();
|
|
let keyring = KeyringProvider::empty();
|
|
// Panics due to debug assertion; returns `None` in production
|
|
let result = std::panic::AssertUnwindSafe(
|
|
keyring.fetch(&DisplaySafeUrlRef::from(&url), Some("user")),
|
|
)
|
|
.catch_unwind()
|
|
.await;
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_with_password() {
|
|
let url = Url::parse("https://user:password@example.com").unwrap();
|
|
let keyring = KeyringProvider::empty();
|
|
// Panics due to debug assertion; returns `None` in production
|
|
let result = std::panic::AssertUnwindSafe(
|
|
keyring.fetch(&DisplaySafeUrlRef::from(&url), Some(url.username())),
|
|
)
|
|
.catch_unwind()
|
|
.await;
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_with_empty_username() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let keyring = KeyringProvider::empty();
|
|
// Panics due to debug assertion; returns `None` in production
|
|
let result = std::panic::AssertUnwindSafe(
|
|
keyring.fetch(&DisplaySafeUrlRef::from(&url), Some(url.username())),
|
|
)
|
|
.catch_unwind()
|
|
.await;
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_no_auth() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let url = DisplaySafeUrlRef::from(&url);
|
|
let keyring = KeyringProvider::empty();
|
|
let credentials = keyring.fetch(&url, Some("user"));
|
|
assert!(credentials.await.is_none());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let keyring = KeyringProvider::dummy([(url.host_str().unwrap(), "user", "password")]);
|
|
assert_eq!(
|
|
keyring
|
|
.fetch(&DisplaySafeUrlRef::from(&url), Some("user"))
|
|
.await,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("password".to_string())
|
|
))
|
|
);
|
|
assert_eq!(
|
|
keyring
|
|
.fetch(
|
|
&DisplaySafeUrlRef::from(&url.join("test").unwrap()),
|
|
Some("user")
|
|
)
|
|
.await,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("password".to_string())
|
|
))
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_no_match() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let keyring = KeyringProvider::dummy([("other.com", "user", "password")]);
|
|
let credentials = keyring
|
|
.fetch(&DisplaySafeUrlRef::from(&url), Some("user"))
|
|
.await;
|
|
assert_eq!(credentials, None);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_prefers_url_to_host() {
|
|
let url = Url::parse("https://example.com/").unwrap();
|
|
let keyring = KeyringProvider::dummy([
|
|
(url.join("foo").unwrap().as_str(), "user", "password"),
|
|
(url.host_str().unwrap(), "user", "other-password"),
|
|
]);
|
|
assert_eq!(
|
|
keyring
|
|
.fetch(
|
|
&DisplaySafeUrlRef::from(&url.join("foo").unwrap()),
|
|
Some("user")
|
|
)
|
|
.await,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("password".to_string())
|
|
))
|
|
);
|
|
assert_eq!(
|
|
keyring
|
|
.fetch(&DisplaySafeUrlRef::from(&url), Some("user"))
|
|
.await,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("other-password".to_string())
|
|
))
|
|
);
|
|
assert_eq!(
|
|
keyring
|
|
.fetch(
|
|
&DisplaySafeUrlRef::from(&url.join("bar").unwrap()),
|
|
Some("user")
|
|
)
|
|
.await,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("other-password".to_string())
|
|
))
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_username() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let keyring = KeyringProvider::dummy([(url.host_str().unwrap(), "user", "password")]);
|
|
let credentials = keyring
|
|
.fetch(&DisplaySafeUrlRef::from(&url), Some("user"))
|
|
.await;
|
|
assert_eq!(
|
|
credentials,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("password".to_string())
|
|
))
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_no_username() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let keyring = KeyringProvider::dummy([(url.host_str().unwrap(), "user", "password")]);
|
|
let credentials = keyring.fetch(&DisplaySafeUrlRef::from(&url), None).await;
|
|
assert_eq!(
|
|
credentials,
|
|
Some(Credentials::basic(
|
|
Some("user".to_string()),
|
|
Some("password".to_string())
|
|
))
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn fetch_url_username_no_match() {
|
|
let url = Url::parse("https://example.com").unwrap();
|
|
let keyring = KeyringProvider::dummy([(url.host_str().unwrap(), "foo", "password")]);
|
|
let credentials = keyring
|
|
.fetch(&DisplaySafeUrlRef::from(&url), Some("bar"))
|
|
.await;
|
|
assert_eq!(credentials, None);
|
|
|
|
// Still fails if we have `foo` in the URL itself
|
|
let url = Url::parse("https://foo@example.com").unwrap();
|
|
let credentials = keyring
|
|
.fetch(&DisplaySafeUrlRef::from(&url), Some("bar"))
|
|
.await;
|
|
assert_eq!(credentials, None);
|
|
}
|
|
}
|