uv was failing to authenticate on 302 redirects when credentials were
available. This was because it was relying on `reqwest_middleware`'s
default redirect behavior which bypasses the middleware pipeline when
trying the redirect request (and hence bypasses our authentication
middleware). This PR updates uv to retrigger the middleware pipeline
when handling a 302 redirect, correctly using credentials from the URL,
the keyring, or `.netrc`.
Closes#5595Closes#11097
## Summary
closes#12234
[fetch_with_cli](e0f81f0d4a/crates/uv-git/src/git.rs (L573))
doesn't respect the registry client's [connectivity
setting](e0f81f0d4a/crates/uv-client/src/registry_client.rs (L1009))
- this pr updates `fetch_with_cli` to set `GIT_ALLOW_PROTOCOL=file` when
the client's connectivity setting is `Connectivity::Offline`
## Test Plan
E2E
```sh
cargo run add "pycurl @ git+https://github.com/pycurl/pycurl.git" --directory ~/src/offline-test/ --offline
```
```sh
Compiling uv-cli v0.0.1 (/Users/justinchapman/src/uv/crates/uv-cli)
Compiling uv v0.6.11 (/Users/justinchapman/src/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.47s
Running `target/debug/uv add 'pycurl @ git+https://github.com/pycurl/pycurl.git' --directory /Users/justinchapman/src/offline-test/ --offline`
Updating https://github.com/pycurl/pycurl.git (HEAD) × Failed to download and build `pycurl @ git+https://github.com/pycurl/pycurl.git`
├─▶ Git operation failed
├─▶ failed to fetch into: /Users/justinchapman/.cache/uv/git-v0/db/9a596e5213c3162d
╰─▶ process didn't exit successfully: `/usr/bin/git fetch --force --update-head-ok 'https://github.com/pycurl/pycurl.git' '+HEAD:refs/remotes/origin/HEAD'` (exit status: 128)
--- stderr
fatal: transport 'https' not allowed
help: If you want to add the package regardless of the failed resolution, provide the `--frozen` flag to skip locking and syncing.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
This PR extends `[[tool.uv.index]]` to support `--find-links`-style
"flat" indexes, so that users can point to such indexes without using
`--find-links` _and_ get access to the full functionality of
`[[tool.uv.index]]` (e.g., they can now pin packages to
`--find-links`-style indexes).
Note that, at present, `--find-links` indexes actually have some quirky
behavior, in that we combine them into a single entity and then merge
the discovered distributions into each Simple API-style index. The
motivation here, IIRC, was to match pip's behavior quite closely. I'm
interested in _removing_ that behavior, but it'd be breaking (and may
also be inconvenient for some use-cases). So, the behavior for indexes
passed in via `--find-links` remains completely unchanged. However,
`[[tool.uv.index]]` entries with `format = "flat"` are now treated
identically to those defined with `format = "simple"` (the default), in
that we stop after we find the first-matching index, etc.
Closes https://github.com/astral-sh/uv/issues/11634.
## Summary
I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
## Summary
This PR modifies the requirement source entities to store a (new)
container struct that wraps `IndexUrl`. This will allow us to store
user-defined metadata alongside `IndexUrl`, and propagate that metadata
throughout resolution.
Specifically, I need to store the "kind" of the index (Simple API vs.
`--find-links`), but I also ran into this problem when I tried to add
support for overriding `Cache-Control` headers on a per-index basis: at
present, we have no way to passing around metadata alongside an
`IndexUrl`.
## Summary
This crate is for standards-compliant types, but this is explicitly a
type that's custom to uv. It's also strange because we kind of want to
reference `IndexUrl` on the registry type, but that's in a crate that
_depends_ on `uv-pypi-types`, which to me is a sign that this is off.
## Summary
This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)
Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.
Closes#12038.
Reduce the overhead of `uv run` in large workspaces. Instead of
re-discovering the entire workspace each time we resolve the metadata of
a member, we can the discovered set of workspace members. Care needs to
be taken to not cache the discovery for `uv init`, `uv add` and `uv
remove`, which change the definitions of workspace members.
Below is apache airflow e3fe06382df4b19f2c0de40ce7c0bdc726754c74 `uv run
python` with a minimal payload. With this change, we avoid a ~350ms
overhead of each `uv run` invocation.
```
$ hyperfine --warmup 2 \
"uv run --no-dev python -c \"print('hi')\"" \
"uv-profiling run --no-dev python -c \"print('hi')\""
Benchmark 1: uv run --no-dev python -c "print('hi')"
Time (mean ± σ): 492.6 ms ± 7.0 ms [User: 393.2 ms, System: 97.1 ms]
Range (min … max): 482.3 ms … 501.5 ms 10 runs
Benchmark 2: uv-profiling run --no-dev python -c "print('hi')"
Time (mean ± σ): 129.7 ms ± 2.5 ms [User: 105.4 ms, System: 23.2 ms]
Range (min … max): 126.0 ms … 136.1 ms 22 runs
Summary
uv-profiling run --no-dev python -c "print('hi')" ran
3.80 ± 0.09 times faster than uv run --no-dev python -c "print('hi')"
```
The profile after those change below. We still spend a large chunk in
toml parsing (both `uv.lock` and `pyproject.toml`), but it's not
excessive anymore.
## Summary
In https://github.com/astral-sh/uv/issues/11998, a user is attempting to
vendor `pydantic-core`. But when they add `pydantic-core = { path =
"src/foo/vendor/pydantic-core" } `, we're installing it as a virtual
package, since `pydantic-core/pyproject.toml` contains `package =
false`.
This PR allows users to mark dependencies as "explicitly a package" or
"explicitly not a package" (i.e., virtual), as a workaround.
Closes https://github.com/astral-sh/uv/issues/11998.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Closes#2410
<!-- What's the purpose of the change? What does it do, and why? -->
This changes the name of files in `wheels` bucket to use a hash instead
of the wheel name as to not exceed maximum file length limit on various
systems.
This only addresses the primary concern of #2410. It still does _not_
address:
- Path limit of 260 on windows:
https://github.com/astral-sh/uv/issues/2410#issuecomment-2062020882
To solve this we need to opt-in to longer path limits on windows
([ref](https://github.com/astral-sh/uv/issues/2410#issuecomment-2150532658)),
but I think that is a separate issue and should be a separate MR.
- Exceeding filename limit while building a wheel from source
distribution
As per my understanding, this is out of uv's control. Name of the output
wheel will be decided by build-backend used by the project. For wheels
built from source distribution, pip also uses the wheel names in cache.
So I have not touched `sdists` cache.
I have added a `filename: WheelFileName` field in `Archive`, so we can
use it while indexing instead of relying on the filename on disk.
Another way to do this was to read `.dist-info/WHEEL` and
`.dist-info/METADATA` and build `WheelFileName` but that seems less
robust and will be slower.
## Test Plan
<!-- How was it tested? -->
Tested by installing `yt-dlp`, `httpie` and `sqlalchemy` and verifying
that cache files in `wheels` bucket use hash.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
* Upgrade the rust toolchain to 1.85.0. This does not increase the MSRV.
* Update windows trampoline to 1.86 nightly beta (previously in 1.85
nightly beta).
## Test Plan
Existing tests
## Summary
I believe these are not necessary... They're currently used in two
places:
1. When building wheels. But that's already wrapped in an in-flight map,
which does the same thing.
2. When fetching source distribution metadata. But every route there
uses it's own `flock` to coordinate across processes, so this seems
redundant?
Initially, we were limiting Git schemes to HTTPS and SSH as only
supported schemes. We lost this validation in #3429. This incidentally
allowed file schemes, which apparently work with Git out of the box.
A caveat for this is that in tool.uv.sources, we parse the git field
always as URL. This caused a problem with #11425: repo = { git =
'c:\path\to\repo', rev = "xxxxx" } was parsed as a URL where c: is the
scheme, causing a bad error message down the line.
This PR:
* Puts Git URL validation back in place. It bans everything but HTTPS,
SSH, and file URLs. This could be a breaking change, if users were using
a git transport protocol were not aware of, even though never
intentionally supported.
* Allows file: URL in Git: This seems to be supported by Git and we were
supporting it albeit unintentionally, so it's reasonable to continue to
support it.
* It does not allow relative paths in the git field in tool.uv.sources.
Absolute file URLs are supported, whether we want relative file URLs for
Git too should be discussed separately.
Closes#3429: We reject the input with a proper error message, while
hinting the user towards file:. If there's still desire for relative
path support, we can keep it open.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
We want to build `uv-build` without depending on the network crates. In
preparation for that, we split uv-git into uv-git and uv-git-types,
where only uv-git depends on reqwest, so that uv-build can use
uv-git-types.
We added this to help with resolving some specific packages, and for
parity with Poetry. But in some cases, this metadata is just wrong, and
at the very least it's unreliable.
Closes https://github.com/astral-sh/uv/issues/8989.
Closes#10945.
Instead of using junctions, we can just write files that contain (as the
file contents) the target path. This requires a little more finesse in
that, as readers, we need to know where to expect these. But it also
means we get to avoid junctions, which have led to a variety of
confusing behaviors. Further, `replace_symlink` should now be on atomic
on Windows.
Closes#11263.
We've never bumped the version of this bucket, and we may never do so...
But it's still incorrect for us to omit it from these serialized structs
in the cache. Specifically, these structs include a pointer into the
archive bucket (namely, the ID). But we don't include the bucket
version! So, in theory, we could end up pointing to archives that don't
match the current bucket version expected in the code.
## Summary
Just a logic issue... If we see a dynamic field that isn't `"version"`,
we end up _not_ propagating the fact that `"version"` is also dynamic.
Closes https://github.com/astral-sh/uv/issues/11460.
## Summary
The environment is located at a stable path within the cache, based on
the script's absolute path.
If a lockfile exists for the script, then we use our standard lockfile
semantics (i.e., update the lockfile if necessary, etc.); if not, we
just do a `uv pip sync` (roughly).
Example usage:
```
❯ uv init --script hello.py
Initialized script at `hello.py`
❯ uv add --script hello.py requests
Updated `hello.py`
❯ cargo run sync --script hello.py
Using script environment at: /Users/crmarsh/.cache/uv/environments-v1/hello-84e289fe3f6241a0
Resolved 5 packages in 3ms
Installed 5 packages in 12ms
+ certifi==2025.1.31
+ charset-normalizer==3.4.1
+ idna==3.10
+ requests==2.32.3
+ urllib3==2.3.0
```
Closes https://github.com/astral-sh/uv/issues/6637.
## Summary
We currently enforce that if you do `uv pip install
./dist/iniconfig-1.0.0.tar.gz`, the build _must_ produce a wheel like
`iniconfig-1.0.0-py3-none-any.whl` (i.e., the name and version must
match). It turns out some packages produce a wheel that has a local
suffix on it, like `vllm`. This PR makes the check a little more
permissive in that we now accept `1.0.0` or that version with a local
suffix (e.g., `1.0.0+cpu`). I don't love this practice, but we already
relaxed this check when _installing_ a wheel, so this seems reasonable:
5e15881dcc/crates/uv-install-wheel/src/install.rs (L50-L52)
Note that this is _still_ stricter than pip. pip seems to only require
that the package name is the same (i.e., `iniconfig` matches
`iniconfig`; but they'll happily install a wheel like
`iniconfig-2.0.0-py3-none-any.whl` given
`./dist/iniconfig-1.0.0.tar.gz`).
Closes https://github.com/astral-sh/uv/issues/11038.
## Motivation
No-op `uv lock` in apache airflow
(891c67f210ab7c877d1f00ea6ea3d3cdbb0e96ef) is slow, which makes `uv run`
slow, too.
Reference project:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 16.3 ms ± 1.5 ms [User: 9.8 ms, System: 6.4 ms]
Range (min … max): 13.0 ms … 20.0 ms 186 runs
```
Apache airflow before:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 161.0 ms ± 5.2 ms [User: 135.3 ms, System: 24.1 ms]
Range (min … max): 155.0 ms … 176.3 ms 18 runs
```
## Optimization
`FlatRequiresDist::from_requirements` is taking 50% of main thread
runtime.
Before:
After both commits:
Apache airflow after the first commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 122.3 ms ± 5.4 ms [User: 96.1 ms, System: 24.7 ms]
Range (min … max): 114.0 ms … 133.2 ms 23 runs
```
Apache airflow after the second commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 108.5 ms ± 3.4 ms [User: 83.2 ms, System: 24.2 ms]
Range (min … max): 103.6 ms … 119.9 ms 28 runs
```
These are noisy relative to the effect they have on the user. It seems
better to prioritize hints on poor resolutions. Notably, it seems hard
to make these "not noisy" ref #11091.
Does not include the "lowest" resolution mode, in which lower bounds are
critical.
With the parallel simple index fetching, we would only acquire one
download concurrency token, meaning that we could in the worst case make
times the number of indexes more requests than the user requested limit.
We fix this by passing the semaphore down to the simple API method.
## Summary
If we fail to deserialize cached metadata in the cache, we should just
ignore it, rather than failing.
Ideally, this never happens. If it does, it means we missed a cache
version bump. But if it does happen, it should still be non-fatal.
Closes https://github.com/astral-sh/uv/issues/11043.
Closes https://github.com/astral-sh/uv/issues/11101.
## Test Plan
Prior to this PR, the following would fail:
- `uvx uv@0.5.25 venv --python 3.12 --cache-dir foo`
- `uvx uv@0.5.25 pip install ./scripts/packages/hatchling_dynamic
--no-deps --python 3.12 --cache-dir foo`
- `uvx uv@0.5.18 venv --python 3.12 --cache-dir foo`
- `uvx uv@0.5.18 pip install ./scripts/packages/hatchling_dynamic
--no-deps --python 3.12 --cache-dir foo`
We can't go back and fix 0.5.18, but this will prevent such regressions
in the future.
## Summary
The issue here boils down to: when we write metadata that came from
building the wheel itself, we aren't setting the `dynamic` field.
We now _always_ set the dynamic field when reading, even when we read
cached data.
Closes https://github.com/astral-sh/uv/issues/11047.
## Summary
On Windows, we have a lot of issues with atomic replacement and such.
There are a bunch of different failure modes, but they generally
involve: trying to persist a fail to a path at which the file already
exists, trying to replace or remove a file while someone else is reading
it, etc.
This PR adds locks to all of the relevant database paths. We already use
these advisory locks when building source distributions; now we use them
when unzipping wheels, storing metadata, etc.
Closes#11002.
## Test Plan
I ran the following script:
```shell
# Define the cache directory path
$cacheDir = "C:\Users\crmar\workspace\uv\cache"
# Clear the cache directory if it exists
if (Test-Path $cacheDir) {
Remove-Item -Recurse -Force $cacheDir
}
# Create the cache directory again
New-Item -ItemType Directory -Force -Path $cacheDir
# Define the command to run with --cache-dir flag
$command = {
param ($venvPath)
# Create a virtual environment in the specified path with --python
uv venv $venvPath
# Run the pip install command with --cache-dir flag
C:\Users\crmar\workspace\uv\target\profiling\uv.exe pip install flask==1.0.4 --no-binary flask --cache-dir C:\Users\crmar\workspace\uv\cache -v --python $venvPath
}
# Define the paths for the different virtual environments
$venv1 = "C:\Users\crmar\workspace\uv\venv1"
$venv2 = "C:\Users\crmar\workspace\uv\venv2"
$venv3 = "C:\Users\crmar\workspace\uv\venv3"
$venv4 = "C:\Users\crmar\workspace\uv\venv4"
$venv5 = "C:\Users\crmar\workspace\uv\venv5"
# Start the command in parallel five times using Start-Job, each with a different venv
$job1 = Start-Job -ScriptBlock $command -ArgumentList $venv1
$job2 = Start-Job -ScriptBlock $command -ArgumentList $venv2
$job3 = Start-Job -ScriptBlock $command -ArgumentList $venv3
$job4 = Start-Job -ScriptBlock $command -ArgumentList $venv4
$job5 = Start-Job -ScriptBlock $command -ArgumentList $venv5
# Wait for all jobs to complete
$jobs = @($job1, $job2, $job3, $job4, $job5)
$jobs | ForEach-Object { Wait-Job $_ }
# Retrieve the results (optional)
$jobs | ForEach-Object { Receive-Job -Job $_ }
# Clean up the jobs
$jobs | ForEach-Object { Remove-Job -Job $_ }
```
And ensured it succeeded in five straight invocations (whereas on
`main`, it consistently fails with a variety of different traces).
## Summary
We should only be ignoring changes in `version` for dynamic projects;
for static projects, it should still be enforced. We should also be
invalidating the lockfile if a project goes from static to dynamic or
vice versa.
Closes#10852.
## Summary
For example: in the linked issue, the user has a symlink at
`pyproject.toml`. The GitHub CDN doesn't give us any way to determine
whether a file is a symlink, so we should just log the error and move on
to the slow path.
Closes https://github.com/astral-sh/uv/issues/10857
## Summary
I noticed that we're only handling `Error::WheelMetadataNameMismatch`
here; but `Error::WheelMetadataVersionMismatch` should also be treated
as non-fatal.
## Summary
I needed this for https://github.com/astral-sh/uv/pull/10794, but it
makes sense as a standalone change, since it's much more testable. We
can also reuse this in at least one more place.
