## Summary
Right now, if a workspace member is first created by way of being a dev
dependency on another member, we end up duplicating it in the graph.
Instead, we should create all the roots upfront; all subsequent node
creations are robust to existing nodes.
Closes
https://github.com/astral-sh/uv/issues/13673#issuecomment-2912196406.
Prior to this PR, there were numerous places where uv would leak
credentials in logs. We had a way to mask credentials by calling methods
or a recently-added `redact_url` function, but this was not secure by
default. There were a number of other types (like `GitUrl`) that would
leak credentials on display.
This PR adds a `DisplaySafeUrl` newtype to prevent leaking credentials
when logging by default. It takes a maximalist approach, replacing the
use of `Url` almost everywhere. This includes when first parsing config
files, when storing URLs in types like `GitUrl`, and also when storing
URLs in types that in practice will never contain credentials (like
`DirectorySourceUrl`). The idea is to make it easy for developers to do
the right thing and for the compiler to support this (and to minimize
ever having to manually convert back and forth). Displaying credentials
now requires an active step. Note that despite this maximalist approach,
the use of the newtype should be zero cost.
One conspicuous place this PR does not use `DisplaySafeUrl` is in the
`uv-auth` crate. That would require new clones since there are calls to
`request.url()` that return a `&Url`. One option would have been to make
`DisplaySafeUrl` wrap a `Cow`, but this would lead to lifetime
annotations all over the codebase. I've created a separate PR based on
this one (#13576) that updates `uv-auth` to use `DisplaySafeUrl` with
one new clone. We can discuss the tradeoffs there.
Most of this PR just replaces `Url` with `DisplaySafeUrl`. The core is
`uv_redacted/lib.rs`, where the newtype is implemented. To make it
easier to review the rest, here are some points of note:
* `DisplaySafeUrl` has a `Display` implementation that masks
credentials. Currently, it will still display the username when there is
both a username and password. If we think is the wrong choice, it can
now be changed in one place.
* `DisplaySafeUrl` has a `remove_credentials()` method and also a
`.to_string_with_credentials()` method. This allows us to use it in a
variety of scenarios.
* `IndexUrl::redacted()` was renamed to
`IndexUrl::removed_credentials()` to make it clearer that we are not
masking.
* We convert from a `DisplaySafeUrl` to a `Url` when calling `reqwest`
methods like `.get()` and `.head()`.
* We convert from a `DisplaySafeUrl` to a `Url` when creating a
`uv_auth::Index`. That is because, as mentioned above, I will be
updating the `uv_auth` crate to use this newtype in a separate PR.
* A number of tests (e.g., in `pip_install.rs`) that formerly used
filters to mask tokens in the test output no longer need those filters
since tokens in URLs are now masked automatically.
* The one place we are still knowingly writing credentials to
`pyproject.toml` is when a URL with credentials is passed to `uv add`
with `--raw`. Since displaying credentials is no longer automatic, I
have added a `to_string_with_credentials()` method to the `Pep508Url`
trait. This is used when `--raw` is passed. Adding it to that trait is a
bit weird, but it's the simplest way to achieve the goal. I'm open to
suggestions on how to improve this, but note that because of the way
we're using generic bounds, it's not as simple as just creating a
separate trait for that method.
PackageMetadata, for whatever reason, does not have a mirrored Wire type
so it was easy to not realize that it contains markers that need to be
complexified.
Fixes#13614
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.
Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
This PR redacts credentials in displayed URLs.
It mostly relies on a `redacted_url` function (and where possible
`IndexUrl::redacted`). This is a quick way to prevent leaked credentials
but it's prone to programmer error when adding new trace statements. A
better follow-on would use a `RedactedUrl` type with the appropriate
`Display` implementation. This would allow us to still extract
credentials from the URL while displaying it securely. On the plus side,
the sites where the `redacted_url` function are used serve as easy
signposts for where to use the new type in a future PR.
Closes#1714.
## Summary
If a script has some requirements, and you provide `--with`, we
currently ignore any constraints from those requirements. We might want
to treat them as hard constraints in the future. For now, though, we
just treat them as preferences -- so we _prefer_ those versions, but
don't require them to match and still run the `--with` resolution in
isolation.
Closes https://github.com/astral-sh/uv/issues/13173.
uv’s default index strategy was designed with dependency confusion
attacks in mind. [According to the
docs](https://docs.astral.sh/uv/configuration/indexes/#searching-across-multiple-indexes),
“if a package exists on an internal index, it should always be installed
from the internal index, and never from PyPI”. Unfortunately, this is
not true in the case where authentication fails on that internal index.
In that case, uv will simply try the next index (even on the
`first-index` strategy). This means that uv is not secure by default in
this common scenario.
This PR causes uv to stop searching for a package if it encounters an
authentication failure at an index. It is possible to opt out of this
behavior for an index with a new `pyproject.toml` option
`ignore-error-codes`. For example:
```
[[tool.uv.index]]
name = "my-index"
url = "<index-url>"
ignore-error-codes = [401, 403]
```
This will also enable users to handle idiosyncratic registries in a more
fine-grained way. For example, PyTorch registries return a 403 when a
package is not found. In this PR, we special-case PyTorch registries to
ignore 403s, but users can use `ignore-error-codes` to handle similar
behaviors if they encounter them on internal registries.
Depends on #12651Closes#9429Closes#12362
## Summary
If you use `--torch-backend=auto`, we want to avoid selecting (e.g.) a
`+cu124` build of `torch` alongside a `+cu126` build of `torchvision`.
## Summary
Part of #8607. This is a pure refactor aimed at paving the way for
supporting the `default-extras` configuration in the `pyproject.toml`
file.
The `ExtraSpecification` struct has been refactored to align more
closely with the
[`DependencyGroups`](256b100a9e/crates/uv-configuration/src/dependency_groups.rs (L9))
struct.
## Test Plan
Existing tests.
## Summary
In https://github.com/astral-sh/uv/pull/12968, we added support for
upload time to `uv.lock`, but stylized as `upload_time`. The other keys
in `uv.lock` use kebab casing, as in common in Python formats, so this
really should've been `upload-time`. I want to change it ASAP to
minimize churn for users. Any users that already upgraded will of course
experience churn in their files a second time. But if we don't change it
now, we'll only increase the surface area of affected users.
So, this PR uses `upload-time` instead, but continues reading
`upload_time` to make it non-breaking.
This PR restores the `bogus_redirect` test that was
non-deterministically hanging (reverting #13076).
Mismatched package and distribution names were causing uv to hang prior
to #12917 (which added the `bogus_redirect` test). But with that fix, uv
was only checking for mismatched package names on the main thread (and
not the resolver thread). This allowed for a race condition which would
prevent uv from ever doing the check, triggering the original hang
condition. This PR adds the check to the resolver thread to prevent this
race condition.
When an index performs a bogus redirect or otherwise returns a different
distribution name than expected, uv currently hangs.
In the example case, requesting the simple index page for any package
returns the page for anyio. This mean querying the sniffio version map
returns only anyio entries, and the version maps resolves to an anyio
version. When the resolver makes a query for sniffio and waits for it to
resolve, the main thread finds an anyio and resolves only that in the
wait map, causing the hang.
We fix this by checking the name of the returned distribution against
the name of the requested distribution. For good measure, we add the
same check in `Request::Dist` and `Request::Installed`. For performance
and complexity reasons, we don't perform this check in the version map
itself, but only after a candidate distribution has been selected.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
We accept `pylock.toml` as a requirements file (e.g., `uv sync
pylock.toml` or `uv pip install -r pylock.toml`). When you provide a
`pylock.toml` file, we don't allow you to provide other requirements, or
constraints, etc. And you can only provide one `pylock.toml` file, not
multiple.
We might want to remove this from `uv pip install` for now, since `pip`
may end up with a different interface (whereas `uv pip sync` is already
specific to uv), and most of the arguments aren't applicable (like
`--resolution`, etc.). Regardless, it's behind `--preview` for both
commands.
## Summary
This PR adds `uv export` support for [PEP
751](https://peps.python.org/pep-0751). We don't yet expose a way to
consume the generated lockfile, but it's a first step.
The logic to go from `uv.lock` to "flat set of packages to include, with
markers telling us when to include them" is all shared with the
`requirements.txt` export (and extracted in
https://github.com/astral-sh/uv/pull/12956). So most of the code is just
converting from our internal types to the PEP 751 schema.
## Summary
This is included in PEP 751, so we lose it when converting from
`uv.lock`. I think it's a good piece of information to include in the
`uv.lock` anyway.
## Summary
The overall strategy here is to make this code look more like
`requirements_txt.rs`: we seed the root members, then perform a DFS.
Previously, we created all nodes upfront, which caused problems when
using `--only-group`, since we'd omit "production" dependencies of
development dependencies.
Closes https://github.com/astral-sh/uv/issues/12526.
Adding dependency trace/parent comments ("via ...") to the export
command output.
This is a similar behavior to the pip compile output.
#### Note to the eager reviewer:
First of all - thanks!
Secondly, this is still a very rough draft. These are the first lines of
code I've ever written in Rust. This is still mostly an educational/fun
exercise for myself. If opening a Draft PR is creating too much noise -
I apologize and I will close it until it is ready.
## Summary
Resolves#7777
## Test Plan
- [X] manual command execution
- [x] update expected output in tests
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This PR extends `[[tool.uv.index]]` to support `--find-links`-style
"flat" indexes, so that users can point to such indexes without using
`--find-links` _and_ get access to the full functionality of
`[[tool.uv.index]]` (e.g., they can now pin packages to
`--find-links`-style indexes).
Note that, at present, `--find-links` indexes actually have some quirky
behavior, in that we combine them into a single entity and then merge
the discovered distributions into each Simple API-style index. The
motivation here, IIRC, was to match pip's behavior quite closely. I'm
interested in _removing_ that behavior, but it'd be breaking (and may
also be inconvenient for some use-cases). So, the behavior for indexes
passed in via `--find-links` remains completely unchanged. However,
`[[tool.uv.index]]` entries with `format = "flat"` are now treated
identically to those defined with `format = "simple"` (the default), in
that we stop after we find the first-matching index, etc.
Closes https://github.com/astral-sh/uv/issues/11634.
## Summary
I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
## Summary
This PR modifies the requirement source entities to store a (new)
container struct that wraps `IndexUrl`. This will allow us to store
user-defined metadata alongside `IndexUrl`, and propagate that metadata
throughout resolution.
Specifically, I need to store the "kind" of the index (Simple API vs.
`--find-links`), but I also ran into this problem when I tried to add
support for overriding `Cache-Control` headers on a per-index basis: at
present, we have no way to passing around metadata alongside an
`IndexUrl`.
## Summary
In general, we merge `--find-links` entries into each index. If a
package is pinned to an index, though, it seems surprising (and wrong)
that we'd ever select a distribution from `--find-links`. This PR
modifies the provider to ignore `--find-links` for any explicitly pinned
packages.
## Summary
This crate is for standards-compliant types, but this is explicitly a
type that's custom to uv. It's also strange because we kind of want to
reference `IndexUrl` on the registry type, but that's in a crate that
_depends_ on `uv-pypi-types`, which to me is a sign that this is off.
## Summary
This is a prototype that I'm considering shipping under `--preview`,
based on [`light-the-torch`](https://github.com/pmeier/light-the-torch).
`light-the-torch` patches pip to pull PyTorch packages from the PyTorch
indexes automatically. And, in particular, `light-the-torch` will query
the installed CUDA drivers to determine which indexes are compatible
with your system.
This PR implements equivalent behavior under `--torch-backend auto`,
though you can also set `--torch-backend cpu`, etc. for convenience.
When enabled, the registry client will fetch from the appropriate
PyTorch index when it sees a package from the PyTorch ecosystem (and
ignore any other configured indexes, _unless_ the package is explicitly
pinned to a different index).
Right now, this is only implemented in the `uv pip` CLI, since it
doesn't quite fit into the lockfile APIs given that it relies on feature
detection on the currently-running machine.
## Test Plan
On macOS, you can test this with (e.g.):
```shell
UV_TORCH_BACKEND=auto UV_CUDA_DRIVER_VERSION=450.80.2 cargo run \
pip install torch --python-platform linux --python-version 3.12
```
On a GPU-enabled EC2 machine:
```shell
ubuntu@ip-172-31-47-149:~/uv$ UV_TORCH_BACKEND=auto cargo run pip install torch -v
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.31s
Running `target/debug/uv pip install torch -v`
DEBUG uv 0.6.6 (e95ca063b 2025-03-14)
DEBUG Searching for default Python interpreter in virtual environments
DEBUG Found `cpython-3.13.0-linux-x86_64-gnu` at `/home/ubuntu/uv/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.13.0 environment at: .venv
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: torch
warning: The `--torch-backend` setting is experimental and may change without warning. Pass `--preview` to disable this warning.
DEBUG Detected CUDA driver version from `/sys/module/nvidia/version`: 550.144.3
...
```
Reject lockfiles where the package version and the wheel versions are
incoherent. This implicitly checks that all wheel files have the same
version.
It does not check for the source dist version, since a source dist may
not contain a version in the filename and attempting to deserialize
source dist filenames we may not need is a performance overhead for
something that's already slow in `uv run`.
Fixes#12164
## Summary
This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)
Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.
Closes#12038.
