Alternative to #15105
Instead of building a `BaseClientBuilder` from `NetworkSettings` each
time we need a client, we instead build a single `BaseClientBuilder` and
pass it around. The `RegistryClientBuilder` then uses
`BaseClientBuilder` exclusively for configuration. This removes a chunk
of copy-and-paste code, and also moves the fallible `retries_from_env`
into a single place
Borrow vs. clone is mostly ad-hoc, we can change it in either direction
if it matters.
Closes#15105
Add support for `RUST_LOG` to the uv build backend. While we were
previously using logging statements in the uv build backend, they could
only be shown when when using the direct build fast path through uv, as
there was no tracing subscriber to write log messages out. This means no
debug logging when using the build backend through pip, `python -m
build`, an incompatible version of uv, or any other build frontend; No
option to figure why includes and excludes behave the way they do.
This PR closes this gap by adding a tracing subscriber. The only option
to enable it is `RUST_LOG`, as we don't have a CLI. The formatting style
is the same as for uv, and color is also support in the same way, albeit
only through anstream's support for TTYs and environment variables. We
recommend only `RUST_LOG=uv=debug` and `RUST_LOG=uv=verbose` in the
docs, but this can be used to debug into crates such as `glob`, too.
<img width="1008" height="325" alt="image"
src="https://github.com/user-attachments/assets/d33df219-750b-46a2-b3b4-8895aa137ab9"
/>
**Before**
```
$ pip wheel . -v [...]
Looking in links: /home/konsti/projects/uv/target/wheels/
Processing /home/konsti/projects/uv/scripts/packages/built-by-uv
Running command pip subprocess to install build dependencies
Looking in links: /home/konsti/projects/uv/target/wheels/
Processing /home/konsti/projects/uv/target/wheels/uv_build-0.8.13-py3-none-manylinux_2_39_x86_64.whl
Installing collected packages: uv_build
Successfully installed uv_build-0.8.13
Installing build dependencies ... done
Running command Getting requirements to build wheel
Getting requirements to build wheel ... done
Running command Preparing metadata (pyproject.toml)
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: built-by-uv
Running command Building wheel for built-by-uv (pyproject.toml)
Error: Unsupported glob expression in: `tool.uv.build-backend.*-exclude`
Caused by:
Invalid character `!` at position 10 in glob: `**/build-*!$§%!½¼²¼³¬!§%$§%.h`. hint: Characters can be escaped with a backslash
error: subprocess-exited-with-error
× Building wheel for built-by-uv (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /usr/bin/python3 /usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpow1illc9
cwd: /home/konsti/projects/uv/scripts/packages/built-by-uv
Building wheel for built-by-uv (pyproject.toml) ... error
ERROR: Failed building wheel for built-by-uv
Failed to build built-by-uv
ERROR: Failed to build one or more wheels
```
**After**
```
$ RUST_LOG=uv=debug pip wheel . -v [...]
Looking in links: /home/konsti/projects/uv/target/wheels/
Processing /home/konsti/projects/uv/scripts/packages/built-by-uv
Running command pip subprocess to install build dependencies
Looking in links: /home/konsti/projects/uv/target/wheels/
Processing /home/konsti/projects/uv/target/wheels/uv_build-0.8.13-py3-none-manylinux_2_39_x86_64.whl
Installing collected packages: uv_build
Successfully installed uv_build-0.8.13
Installing build dependencies ... done
Running command Getting requirements to build wheel
Getting requirements to build wheel ... done
Running command Preparing metadata (pyproject.toml)
DEBUG Writing metadata files to /tmp/pip-modern-metadata-l_kh78cj
DEBUG Found PEP 639 license declarations, using METADATA 2.4
DEBUG License files match: `LICENSE-APACHE`
DEBUG License files match: `LICENSE-MIT`
DEBUG License files match: `third-party-licenses/PEP-401.txt`
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: built-by-uv
Running command Building wheel for built-by-uv (pyproject.toml)
DEBUG Checking metadata directory /tmp/pip-modern-metadata-l_kh78cj/built_by_uv-0.1.0.dist-info
DEBUG Found PEP 639 license declarations, using METADATA 2.4
DEBUG License files match: `LICENSE-APACHE`
DEBUG License files match: `LICENSE-MIT`
DEBUG License files match: `third-party-licenses/PEP-401.txt`
DEBUG Writing wheel at /tmp/pip-wheel-bu6to9i7/built_by_uv-0.1.0-py3-none-any.whl
DEBUG Wheel excludes: ["__pycache__", "*.pyc", "*.pyo", "build-*!$§%!½¼²¼³¬!§%$§%.h", "/src/built_by_uv/not-packaged.txt"]
Error: Unsupported glob expression in: `tool.uv.build-backend.*-exclude`
Caused by:
Invalid character `!` at position 10 in glob: `**/build-*!$§%!½¼²¼³¬!§%$§%.h`. hint: Characters can be escaped with a backslash
error: subprocess-exited-with-error
× Building wheel for built-by-uv (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /usr/bin/python3 /usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpjrxou13a
cwd: /home/konsti/projects/uv/scripts/packages/built-by-uv
Building wheel for built-by-uv (pyproject.toml) ... error
ERROR: Failed building wheel for built-by-uv
Failed to build built-by-uv
ERROR: Failed to build one or more wheels
```
(There is no color in the above uv log statements, as pip doesn't
register as a TTY)
Fixes#12723
## Summary
This is causing some cyclic dependencies issues for me, because these
can be used in virtually _any_ crate (like `uv-install-wheel`), which
then means that all of `uv-configuration` becomes a dependency, etc. I
think this should be a leaf crate so that we can safely depend on it
anywhere.
## Summary
uv will now reject ZIP files that meet any of the following conditions:
- Multiple local header entries exist for the same file with different
contents.
- A local header entry exists for a file that isn't included in the
end-of-central directory record.
- An entry exists in the end-of-central directory record that does not
have a corresponding local header.
- The ZIP file contains contents after the first end-of-central
directory record.
- The CRC32 doesn't match between the local file header and the
end-of-central directory record.
- The compressed size doesn't match between the local file header and
the end-of-central directory record.
- The uncompressed size doesn't match between the local file header and
the end-of-central directory record.
- The reported central directory offset (in the end-of-central-directory
header) does not match the actual offset.
- The reported ZIP64 end of central directory locator offset does not
match the actual offset.
We also validate the above for files with data descriptors, which we
previously ignored.
Wheels from the most recent releases of the top 15,000 packages on PyPI
have been confirmed to pass these checks, and PyPI will also reject ZIPs
under many of the same conditions (at upload time) in the future.
In rare cases, this validation can be disabled by setting
`UV_INSECURE_NO_ZIP_VALIDATION=1`. Any validations should be reported to
the uv issue tracker and to the upstream package maintainer.
## Summary
Make the use of `Self` consistent. Mostly done by running `cargo clippy
--fix -- -A clippy::all -W clippy::use_self`.
## Test Plan
<!-- How was it tested? -->
No need.
This fixes a regression from 0.8.0 from
https://github.com/astral-sh/uv/pull/7934 and follows
https://github.com/astral-sh/uv/pull/15059
The regression is from [this
change](https://github.com/astral-sh/uv/pull/7934/files#diff-c7a660ac39628d5e12f388b0cacc7360affa3d7bb21191184d7ee78489675e83),
which was made because we'd otherwise (with the other changes in that
pull request) _filter out_ managed Python interpreters found in virtual
environments.
When `--system` is used we'll convert the default Python preference of
`managed` to `system` which avoids things like `uv pip install --system`
targeting a managed Python installation.
The basic test is
```
uv python install
uv pip install --system anyio
```
Prior to this change, we'd read a managed interpreter from our managed
installation directory and target that. After this change, without
#15059, we'd read a managed interpreter from the PATH and target that.
Both of those experiences are bad, because the managed interpreters are
marked as externally managed. After this change, with #15059, we
properly target the system interpreter.
Since we use `system` instead of `only-system`, if there is not a system
interpreter we'll still retain our existing behavior and use a managed
interpreter. This should limit breakage from the change. Given the
source of the regression, we could probably use `only-system` here. I
don't feel strongly. I think the main benefit of doing so would be that
we'd omit the check for managed installations in error messages when an
interpreter cannot be found?
We can't really add test coverage here because the test suite always has
externally managed interpreters :)
This means that CI tests fail in a way that is redundant with the
dedicated CI job which can obscure signal on whether actual tests are
failing
e.g.,
4703411653
I think this would give us better hygiene than a global flag. It makes
it easier for users to opt-in to overlapping features, such as Python
upgrades and Python bin installations and to disable warnings for
preview mode without opting in to a bunch of other features. In general,
I want to reduce the burden for putting something under preview.
The `--preview` and `--no-preview` flags are retained as global
overrides. A new `--preview-features` option is added which accepts
comma separated features or can be passed multiple times, e.g.,
`--preview-features add-bounds,pylock`. There's a `UV_PREVIEW_FEATURES`
environment variable for that option (I'm not sure if we should overload
`UV_PREVIEW`, but could be convinced).
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Update [schemars
0.9.0](https://github.com/GREsau/schemars/releases/tag/v0.9.0)
There are differences in the generated JSON Schema and I will [contact
the author](https://github.com/GREsau/schemars/issues/407).
## Test Plan
---------
Co-authored-by: konstin <konstin@mailbox.org>
> NOTE: The PRs that were merged into this feature branch have all been
independently reviewed. But it's also useful to see all of the changes
in their final form. I've added comments to significant changes
throughout the PR to aid discussion.
This PR introduces transparent Python version upgrades to uv, allowing
for a smoother experience when upgrading to new patch versions.
Previously, upgrading Python patch versions required manual updates to
each virtual environment. Now, virtual environments can transparently
upgrade to newer patch versions.
Due to significant changes in how uv installs and executes managed
Python executables, this functionality is initially available behind a
`--preview` flag. Once an installation has been made upgradeable through
`--preview`, subsequent operations (like `uv venv -p 3.10` or patch
upgrades) will work without requiring the flag again. This is
accomplished by checking for the existence of a minor version symlink
directory (or junction on Windows).
### Features
* New `uv python upgrade` command to upgrade installed Python versions
to the latest available patch release:
```
# Upgrade specific minor version
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview
```
* Transparent upgrades also occur when installing newer patch versions:
```
uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18
```
* Support for transparently upgradeable Python `bin` installations via
`--preview` flag
```
uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview
```
* Virtual environments can still be tied to a patch version if desired
(ignoring patch upgrades):
```
uv venv -p 3.10.8
```
### Implementation
Transparent upgrades are implemented using:
* Minor version symlink directories (Unix) or junctions (Windows)
* On Windows, trampolines simulate paths with junctions
* Symlink directory naming follows Python build standalone format: e.g.,
`cpython-3.10-macos-aarch64-none`
* Upgrades are scoped to the minor version key (as represented in the
naming format: implementation-minor version+variant-os-arch-libc)
* If the context does not provide a patch version request and the
interpreter is from a managed CPython installation, the `Interpreter`
used by `uv python run` will use the full symlink directory executable
path when available, enabling transparently upgradeable environments
created with the `venv` module (`uv run python -m venv`)
New types:
* `PythonMinorVersionLink`: in a sense, the core type for this PR, this
is a representation of a minor version symlink directory (or junction on
Windows) that points to the highest installed managed CPython patch
version for a minor version key.
* `PythonInstallationMinorVersionKey`: provides a view into a
`PythonInstallationKey` that excludes the patch and prerelease. This is
used for grouping installations by minor version key (e.g., to find the
highest available patch installation for that minor version key) and for
minor version directory naming.
### Compatibility
* Supports virtual environments created with:
* `uv venv`
* `uv run python -m venv` (using managed Python that was installed or
upgraded with `--preview`)
* Virtual environments created within these environments
* Existing virtual environments from before these changes continue to
work but aren't transparently upgradeable without being recreated
* Supports both standard Python (`python3.10`) and freethreaded Python
(`python3.10t`)
* Support for transparently upgrades is currently only available for
managed CPython installations
Closes#7287Closes#7325Closes#7892Closes#9031Closes#12977
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Extends https://github.com/astral-sh/uv/pull/13841 — I'll drop that
commit later after that pull request merges but it's small.
I find the split into a "Configuration" section awkward and don't think
it's helping us. Everything moved into the "Concepts" section, except
the "Environment variables" page which definitely belongs in the
reference and the "Installer" page which is fairly niche and seems
better in the reference.
Before / After
<img
src="https://github.com/user-attachments/assets/80d8304b-17da-4900-a5f4-c3ccac96fcc5"
width="400">
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [markdown](https://redirect.github.com/wooorm/markdown-rs) |
dependencies | major | `0.3.0` -> `1.0.0` |
---
### Release Notes
<details>
<summary>wooorm/markdown-rs (markdown)</summary>
###
[`v1.0.0`](https://redirect.github.com/wooorm/markdown-rs/releases/tag/1.0.0)
💯
Nothing changed since the last alpha.
It’s just that: this crate’s now being used a bunch and working well, so
it’s time to be stable!
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/uv).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yNTcuMyIsInVwZGF0ZWRJblZlciI6IjM5LjI1Ny4zIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: konstin <konstin@mailbox.org>
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.
Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
#5577 fixed a bug on macos due to dynamically linking lzma/xz through
static linking. In #7686, this feature was moved to the performance
category.
This PR moves the `xz2/static` back to the general default features,
and, inspired by https://github.com/Homebrew/homebrew-core/pull/222211,
it structures and documents the feature flags cleaner.
We need to take care that this feature does not accidentally disable
features we want.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
I believe `zlib-rs` is now a better choice on ARM and x86, so I'm just
going to assume it's a better choice everywhere. It's much easier to
build (removes our CMake dependency), and in my benchmarking, it's
substantially faster on ARM and faster or ~exactly even on my x86
Windows machine.
We migrated to `zlib-rs` once before (#9184); however, I later reverted
it as I learned that they were only doing compile-time feature
detection, and so `zlib-rs` was meaningfully slower on x86. They now
perform runtime feature detection:
https://trifectatech.org/blog/zlib-rs-is-faster-than-c/.
To benchmark, I wrote a script to create a local Simple API-compliant
registry (see the commit history) for a single package. Then I ran the
`install-cold` benchmark against that registry to install NumPy.
On ARM:
```
❯ uv run resolver --uv-pip-path ../../zlib-ng --uv-pip-path ../../zlib-rs \
--benchmark install-cold \
req.txt --warmup 10 --min-runs 30
Benchmark 1: ../../zlib-ng (install-cold)
Time (mean ± σ): 165.7 ms ± 34.7 ms [User: 64.4 ms, System: 93.2 ms]
Range (min … max): 141.8 ms … 293.2 ms 30 runs
Benchmark 2: ../../zlib-rs (install-cold)
Time (mean ± σ): 150.9 ms ± 16.2 ms [User: 57.4 ms, System: 86.4 ms]
Range (min … max): 135.3 ms … 202.4 ms 30 runs
Summary
../../zlib-rs (install-cold) ran
1.10 ± 0.26 times faster than ../../zlib-ng (install-cold)
```
I benchmarked this about 100 times on my Windows machine and found it
difficult to conclude anything beyond "They're nearly the same". Here's
an example:
```
PS C:\Users\crmar\workspace\puffin> hyperfine --prepare "uv venv" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" --runs 10 --warmup 5
Benchmark 1: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 240.6 ms ± 10.8 ms [User: 6.1 ms, System: 92.2 ms]
Range (min … max): 229.4 ms … 267.9 ms 10 runs
Benchmark 2: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 241.3 ms ± 6.2 ms [User: 7.7 ms, System: 90.6 ms]
Range (min … max): 233.9 ms … 252.1 ms 10 runs
Benchmark 3: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 242.8 ms ± 7.7 ms [User: 6.2 ms, System: 23.4 ms]
Range (min … max): 236.1 ms … 262.8 ms 10 runs
Benchmark 4: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 245.9 ms ± 5.7 ms [User: 1.5 ms, System: 59.4 ms]
Range (min … max): 240.9 ms … 257.3 ms 10 runs
Summary
zlib-rs.exe pip sync ./scripts/benchmark/req.txt ran
1.00 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
1.01 ± 0.06 times faster than zlib-rs.exe pip sync ./scripts/benchmark/req.txt
1.02 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
```
Closes#11885.
In #6827, we switched the uv-dev binary to not being built by default.
As an unintended side effect, we were also stopping to run the tests
that ensured the schema was up-to-date.
To fix this, we split uv-dev into an unconditional library, with only
the binary being a conditional build. This way, `cargo test` and `cargo
nextest` pick those tests up again.
An alternative would be running tests with the `dev` feature, with the
side effect of always building the uv-dev binary, too.
Instead of always using all available threads for bytecode compilation,
respect `UV_CONCURRENT_INSTALLS`, so the parallelism is configurable
instead of hardcoded. We reuse the install limit since bytecode
compilation only runs after install.
Enable `lzma-sys/static` through the performance feature not only in uv,
but in uv-dev and uv-bench too, to avoid the system dependency on
`liblzma-dev`.
Ref #9880
Add a preview option `uv init --build-backend uv --preview` that uses
the uv build backend when generating the project. The uv build backend
is in preview, so the option is also guarded by preview and hidden from
the help message and docs.
For https://github.com/astral-sh/uv/issues/3957#issuecomment-2518757563
- Adds a collapsible section for the project concept
- Splits the project concept document into several child documents.
- Moves the workspace and dependencies documents to under the project
section
- Adds a mkdocs plugin for redirects, so links to the moved documents
still work
I attempted to make the minimum required changes to the contents of the
documents here. There is a lot of room for improvement on the content of
each new child document. For review purposes, I want to do that work
separately. I'd prefer if the review focused on this structure and idea
rather than the content of the files.
I expect to do this to other documentation pages that would otherwise be
very nested.
The project concept landing page and nav (collapsed by default) looks
like this now:
<img width="1507" alt="Screenshot 2024-11-14 at 11 28 45 AM"
src="https://github.com/user-attachments/assets/88288b09-8463-49d4-84ba-ee27144b62a5">
## Summary
These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
## Summary
* Env docs now support anchors, which allows sending a link to someone
with a direct reference to an env var or cross-reference them in the
docs.
* Marked additional env vars as hidden from the docs due to their
internal use
* Updates some tests still using literals to use the static env vars
## Test Plan
<img width="1370" alt="env_var_anchors"
src="https://github.com/user-attachments/assets/52ae1caa-5199-4798-9eb5-81b8f5b57c24">
## Summary
Resolves#8417
I've just begun learning procedural macros, so this PR is more of a
proof of concept. It's still a work in progress, and I welcome any
assistance or feedback.
## Summary
For example, in:
```toml
[tool.uv]
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
We can just omit `[tool.uv]`.
## Summary
This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.
I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.
This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.
I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.
I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.
Did I miss any? Are their initial docs cohesive?
Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.
## Test Plan
Existing tests
