Co-authored-by: Pradyun Gedam <pgedam@bloomberg.net>
When a venv is created, values such as `sys.path` and `site.getsitepackages()` are different from the base interpreter, which we query from the interpreter directly instead of trying to do the transformation ourselves.
Previously, uv was discovering packages only in purelib/platlib, which excluded system site packages if activated.
`site.getsitepackages()` is more stable than `sys.path`. `sys.path` can change in a number of ways such as `.pth` files and runtime modification, while `site.getsitepackages()` is generally stable for a Python environment. Using the `site`, we get the same paths that Python itself uses for `sys.path`.
This behavior was originally written for supporting `sys.path` instead of `site.getsitepackages()`: It's currently not load-bearing, but it is correct deduplication if we need to add more paths to the package discovery in the future.
With this change the behavior of `uv pip list` mirrors the behavior of `pip list`: It shows only the first package in `sys.path`, which is the package that Python will load, and ignores the others.
In the JSON output, we show all packages, but tagged with a `shadowed` bool. The exception is `--outdated`, in which case we only check if the main packages are outdated (updating shadowed packages is ineffective). The consumer is responsible for filtering on `shadowed` if applicable.
We show a hint (stderr, not stdout like the main output), if shadowed packages exist.
The logic includes handling of the fedora lib/lib64 venv split.
This better reflects the intended behavior of this object, which is to
serve as a database of InstalledPackages rather than a representation of
the relevant environment/interpreter path.
## Summary
Small omission I noticed last night. This was overly strict (so, didn't
lead to any incorrect behavior; more that we did unnecessary work in
some cases).
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Similar to https://github.com/astral-sh/uv/pull/11399
This adds `UV_NO_BUILD` and `UV_NO_BUILD_PACKAGE` environment variables
for non-pip commands.
## Test Plan
<!-- How was it tested? -->
Tested manually and with snapshot tests.
Fixes#11963
Signed-off-by: Alex Lowe <alex@lowe.dev>
## Summary
If we're looking at (e.g.) `python3.12`, and we have a `requires-python:
">=3.12.7, <3.13"`, then checking if the range includes `3.12` will
return `false`. Instead, we need to look at the lower- and upper-bound
major-minors of the `requires-python`.
Closes https://github.com/astral-sh/uv/issues/11825.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
To stay within quota, it now has just under 30 days of data, so the
filename has been updated. Both will be available for a while. See
https://github.com/hugovk/top-pypi-packages/pull/46.
## Test Plan
<!-- How was it tested? -->
```sh
curl https://hugovk.github.io/top-pypi-packages/top-pypi-packages-30-days.min.json | jq -r ❯
curl https://hugovk.github.io/top-pypi-packages/top-pypi-packages.min.json | jq -r ".rows | .
diff pypi_8k_downloads_original.txt pypi_8k_downloads_new.txt
```
We prepend the interpreter discovery in a temporary path to `sys.path`,
which we have to strip to avoid the `sys.path` value containing a
then-deleted temp dir.
I noticed that https://github.com/astral-sh/uv/pull/11936 did not run
the Docker builds, nor did #11934
We should run these when the relevant files change so there aren't
surprises at release time!
Updates the `build-binaries` workflow to include toolchain version
changes and `.cargo/config.toml` changes too.
If we see `uvx script.py`, we exit early, giving a hint to use `uv run
script.py` if the script exists. If it does not exist, we suggest
running `uv run` with a normalized package name.
This PR includes a snapshot test for each of these scenarios.
An alternative approach would be to wait until we encounter an error,
and then add the hint. But if there happens to be a malicious package
called `script-py`, this would be run unintentionally (a point raised by
@zanieb).
Closes#10784
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fix error message suggesting `--user` instead of `--username`:
```
> uv publish --publish-url ... ... --password $(cat ~/.token)
Publishing 1 file to ...
error: Attempted to publish with a password, but no username. Either provide a username with `--user` (`UV_PUBLISH_USERNAME`), or use `--token` (`UV_PUBLISH_TOKEN`) instead of a password.
> uv publish --publish-url ... ... --user lewis --password $(cat ~/.token)
error: unexpected argument '--user' found
tip: a similar argument exists: '--username'
Usage: uv publish <FILES|--index <INDEX>|--username <USERNAME>|--password <PASSWORD>|--token <TOKEN>|--trusted-publishing <TRUSTED_PUBLISHING>|--keyring-provider <KEYRING_PROVIDER>|--publish-url <PUBLISH_URL>|--check-url <CHECK_URL>|--skip-existing>
For more information, try '--help'.
```
## Test Plan
I have not tested manually, I'm hoping this isn't necessary and there
will be sufficient CI coverage.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
## Summary
I believe `zlib-rs` is now a better choice on ARM and x86, so I'm just
going to assume it's a better choice everywhere. It's much easier to
build (removes our CMake dependency), and in my benchmarking, it's
substantially faster on ARM and faster or ~exactly even on my x86
Windows machine.
We migrated to `zlib-rs` once before (#9184); however, I later reverted
it as I learned that they were only doing compile-time feature
detection, and so `zlib-rs` was meaningfully slower on x86. They now
perform runtime feature detection:
https://trifectatech.org/blog/zlib-rs-is-faster-than-c/.
To benchmark, I wrote a script to create a local Simple API-compliant
registry (see the commit history) for a single package. Then I ran the
`install-cold` benchmark against that registry to install NumPy.
On ARM:
```
❯ uv run resolver --uv-pip-path ../../zlib-ng --uv-pip-path ../../zlib-rs \
--benchmark install-cold \
req.txt --warmup 10 --min-runs 30
Benchmark 1: ../../zlib-ng (install-cold)
Time (mean ± σ): 165.7 ms ± 34.7 ms [User: 64.4 ms, System: 93.2 ms]
Range (min … max): 141.8 ms … 293.2 ms 30 runs
Benchmark 2: ../../zlib-rs (install-cold)
Time (mean ± σ): 150.9 ms ± 16.2 ms [User: 57.4 ms, System: 86.4 ms]
Range (min … max): 135.3 ms … 202.4 ms 30 runs
Summary
../../zlib-rs (install-cold) ran
1.10 ± 0.26 times faster than ../../zlib-ng (install-cold)
```
I benchmarked this about 100 times on my Windows machine and found it
difficult to conclude anything beyond "They're nearly the same". Here's
an example:
```
PS C:\Users\crmar\workspace\puffin> hyperfine --prepare "uv venv" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" --runs 10 --warmup 5
Benchmark 1: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 240.6 ms ± 10.8 ms [User: 6.1 ms, System: 92.2 ms]
Range (min … max): 229.4 ms … 267.9 ms 10 runs
Benchmark 2: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 241.3 ms ± 6.2 ms [User: 7.7 ms, System: 90.6 ms]
Range (min … max): 233.9 ms … 252.1 ms 10 runs
Benchmark 3: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 242.8 ms ± 7.7 ms [User: 6.2 ms, System: 23.4 ms]
Range (min … max): 236.1 ms … 262.8 ms 10 runs
Benchmark 4: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 245.9 ms ± 5.7 ms [User: 1.5 ms, System: 59.4 ms]
Range (min … max): 240.9 ms … 257.3 ms 10 runs
Summary
zlib-rs.exe pip sync ./scripts/benchmark/req.txt ran
1.00 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
1.01 ± 0.06 times faster than zlib-rs.exe pip sync ./scripts/benchmark/req.txt
1.02 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
```
Closes#11885.
At certain points in the code, dependency groups are represented by
`DevGroups*` naming, probably as a historical artifact. This PR updates
the naming.
This includes renaming `uv-configuration/src/dev.rs` to
`uv-configuration/src/dependency_groups.rs`.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Follow on to #11706. In the original PR, I tried to solve the issue by
getting rid of the `ctrlc::set_handler` call. Unfortunately, this didn't
work on windows due to an issue with the console crate. console 0.15.11
includes https://github.com/console-rs/console/pull/235, which resolves
the issue, so now we can get rid of the call.
<!-- What's the purpose of the change? What does it do, and why? -->
This change is not super important but I still think it's worthwhile.
For one, spinning up a background thread to handle `SIGINT`s when we're
going to be raising the `SIGINT` from within the function is more
technical complexity than needed, now that there's an easy way to
explicitly catch the Ctrl-C from the terminal input. Secondly,
`ctrlc::set_handler`'s
[docs](https://docs.rs/ctrlc/3.4.5/ctrlc/fn.set_handler.html) advise
that you set the handler just once, at the beginning of the program, so
this use seems somewhat error prone. In fact, uv already has a second
[callsite](461f4d9007/crates/uv/src/commands/project/add.rs (L596-L611))
for this function (though I'm not sure if the two callsites could
currently ever both occur on the same run of uv)
## Test Plan
I've tested this manually on linux (WSL ubuntu) and windows, though not
on aarch64-apple-darwin as I don't have a machine running that. I would
appreciate if someone would double-check that it works on such machines.
As discussed in the original PR, this change is pretty hard to test due
to the fact that the behavior only occurs if stderr is connected to a
tty. I experimented with using pseudoterminals to test this but it's
still quite tricky due to the lack of x-platform non-blocking reads on
the pty.
<!-- How was it tested? -->
Fixes#11217
By default, a 64-bit uv does not see a 32-bit global (HKLM) installation
of Python in the registry
(https://github.com/astral-sh/uv/issues/11217). To work around this, we
manually request both 32-bit and 64-bit access using registry access
flags (https://peps.python.org/pep-0514/#sample-code). The flags have no
effect on 32-bit (https://stackoverflow.com/a/12796797/3549270).
This effect is that there is an asymmetry between discovery modes: For
the registry-based discovery using PEP 514, we discover both 32-bit and
64-bit Pythons, while for managed installations, we are stricter and
only discover those matching in bit-ness.
I tested this manually with an additional 32-bit installation of CPython
on a 64-bit machine and windows with 32-bit and 64-bit (x86_64 and i686)
builds of uv.
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[reqwest-middleware](https://redirect.github.com/TrueLayer/reqwest-middleware)
| workspace.dependencies | patch | `0.4.0` -> `0.4.1` |
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/uv).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNzYuMiIsInVwZGF0ZWRJblZlciI6IjM5LjE3Ni4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: konstin <konstin@mailbox.org>
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.
In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
## Summary
Testing with `UV_PYTHON_INSTALL_DIR` environment variable has some
problems. This PR fix them.
- `UV_PYTHON_INSTALL_DIR` must be an absolute path.
- Cargo tries to find Python executables from each crates in test. If it
is relative path, cargo searches in different directories for each
tests.
- Skip the test asserting help messages.
- Clap shows the current value of the environment variables. If
`UV_PYTHON_INSTALL_DIR` is set, the test fails.
## Test Plan
<!-- How was it tested? -->
All tests pass with
`UV_PYTHON_INSTALL_DIR=/path/to/my/home/uv/target/testpython`.
## Summary
This is roughly equivalent, but gets the non-`+cpu` macOS build from the
PyTorch index rather than PyPI. It seems a bit simpler? Though up for
debate.
## Summary
The current wording on the [caching
page](https://docs.astral.sh/uv/concepts/cache/#dynamic-metadata) makes
it sounds like defining `cache-keys` in a project adds to the metadata
considered when caching. However it actually replaces the metadata. So
copying the example using the git commit results in only considering the
git commit, not the pyproject.toml, which is likely not what is
typically desired.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
