<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Update [schemars
0.9.0](https://github.com/GREsau/schemars/releases/tag/v0.9.0)
There are differences in the generated JSON Schema and I will [contact
the author](https://github.com/GREsau/schemars/issues/407).
## Test Plan
---------
Co-authored-by: konstin <konstin@mailbox.org>
We were checking whether a path was an executable in a virtual
environment or the base directory of a virtual environment in multiple
places in the codebase. This PR consolidates this logic into one place.
Closes#13947.
> NOTE: The PRs that were merged into this feature branch have all been
independently reviewed. But it's also useful to see all of the changes
in their final form. I've added comments to significant changes
throughout the PR to aid discussion.
This PR introduces transparent Python version upgrades to uv, allowing
for a smoother experience when upgrading to new patch versions.
Previously, upgrading Python patch versions required manual updates to
each virtual environment. Now, virtual environments can transparently
upgrade to newer patch versions.
Due to significant changes in how uv installs and executes managed
Python executables, this functionality is initially available behind a
`--preview` flag. Once an installation has been made upgradeable through
`--preview`, subsequent operations (like `uv venv -p 3.10` or patch
upgrades) will work without requiring the flag again. This is
accomplished by checking for the existence of a minor version symlink
directory (or junction on Windows).
### Features
* New `uv python upgrade` command to upgrade installed Python versions
to the latest available patch release:
```
# Upgrade specific minor version
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview
```
* Transparent upgrades also occur when installing newer patch versions:
```
uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18
```
* Support for transparently upgradeable Python `bin` installations via
`--preview` flag
```
uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview
```
* Virtual environments can still be tied to a patch version if desired
(ignoring patch upgrades):
```
uv venv -p 3.10.8
```
### Implementation
Transparent upgrades are implemented using:
* Minor version symlink directories (Unix) or junctions (Windows)
* On Windows, trampolines simulate paths with junctions
* Symlink directory naming follows Python build standalone format: e.g.,
`cpython-3.10-macos-aarch64-none`
* Upgrades are scoped to the minor version key (as represented in the
naming format: implementation-minor version+variant-os-arch-libc)
* If the context does not provide a patch version request and the
interpreter is from a managed CPython installation, the `Interpreter`
used by `uv python run` will use the full symlink directory executable
path when available, enabling transparently upgradeable environments
created with the `venv` module (`uv run python -m venv`)
New types:
* `PythonMinorVersionLink`: in a sense, the core type for this PR, this
is a representation of a minor version symlink directory (or junction on
Windows) that points to the highest installed managed CPython patch
version for a minor version key.
* `PythonInstallationMinorVersionKey`: provides a view into a
`PythonInstallationKey` that excludes the patch and prerelease. This is
used for grouping installations by minor version key (e.g., to find the
highest available patch installation for that minor version key) and for
minor version directory naming.
### Compatibility
* Supports virtual environments created with:
* `uv venv`
* `uv run python -m venv` (using managed Python that was installed or
upgraded with `--preview`)
* Virtual environments created within these environments
* Existing virtual environments from before these changes continue to
work but aren't transparently upgradeable without being recreated
* Supports both standard Python (`python3.10`) and freethreaded Python
(`python3.10t`)
* Support for transparently upgrades is currently only available for
managed CPython installations
Closes#7287Closes#7325Closes#7892Closes#9031Closes#12977
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
We've been using a number of different winapi crates. This PR removes
winsafe in favor of the official windows-* crates, so all of uv's own
winapi calls go through the official windows-* crates.
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
In #13302, there was an IO error without context. This error seems to be
caused by a symlink error. Switching as symlinking to `fs_err` ensures
these errors will carry context in the future.
## Summary
I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
This PR moves functions for finding user- and system-level config
directories to public functions in `uv_fs::config`. This will allow them
to be used in future work without duplicating code.
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.
In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
Instead of using junctions, we can just write files that contain (as the
file contents) the target path. This requires a little more finesse in
that, as readers, we need to know where to expect these. But it also
means we get to avoid junctions, which have led to a variety of
confusing behaviors. Further, `replace_symlink` should now be on atomic
on Windows.
Closes#11263.
## Summary
These are used for coordination across processes. If you run uv under,
e.g., the root user, then under a different user, I don't think we
should prevent you from acquiring the lock.
Closes https://github.com/astral-sh/uv/issues/11324.
## Summary
This lets us drop a dependency entirely. `percent-encoding` is used by
`url` and so is already in the graph, whereas `urlencoding` isn't used
by anything else.
## Summary
On Windows, we have a lot of issues with atomic replacement and such.
There are a bunch of different failure modes, but they generally
involve: trying to persist a fail to a path at which the file already
exists, trying to replace or remove a file while someone else is reading
it, etc.
This PR adds locks to all of the relevant database paths. We already use
these advisory locks when building source distributions; now we use them
when unzipping wheels, storing metadata, etc.
Closes#11002.
## Test Plan
I ran the following script:
```shell
# Define the cache directory path
$cacheDir = "C:\Users\crmar\workspace\uv\cache"
# Clear the cache directory if it exists
if (Test-Path $cacheDir) {
Remove-Item -Recurse -Force $cacheDir
}
# Create the cache directory again
New-Item -ItemType Directory -Force -Path $cacheDir
# Define the command to run with --cache-dir flag
$command = {
param ($venvPath)
# Create a virtual environment in the specified path with --python
uv venv $venvPath
# Run the pip install command with --cache-dir flag
C:\Users\crmar\workspace\uv\target\profiling\uv.exe pip install flask==1.0.4 --no-binary flask --cache-dir C:\Users\crmar\workspace\uv\cache -v --python $venvPath
}
# Define the paths for the different virtual environments
$venv1 = "C:\Users\crmar\workspace\uv\venv1"
$venv2 = "C:\Users\crmar\workspace\uv\venv2"
$venv3 = "C:\Users\crmar\workspace\uv\venv3"
$venv4 = "C:\Users\crmar\workspace\uv\venv4"
$venv5 = "C:\Users\crmar\workspace\uv\venv5"
# Start the command in parallel five times using Start-Job, each with a different venv
$job1 = Start-Job -ScriptBlock $command -ArgumentList $venv1
$job2 = Start-Job -ScriptBlock $command -ArgumentList $venv2
$job3 = Start-Job -ScriptBlock $command -ArgumentList $venv3
$job4 = Start-Job -ScriptBlock $command -ArgumentList $venv4
$job5 = Start-Job -ScriptBlock $command -ArgumentList $venv5
# Wait for all jobs to complete
$jobs = @($job1, $job2, $job3, $job4, $job5)
$jobs | ForEach-Object { Wait-Job $_ }
# Retrieve the results (optional)
$jobs | ForEach-Object { Receive-Job -Job $_ }
# Clean up the jobs
$jobs | ForEach-Object { Remove-Job -Job $_ }
```
And ensured it succeeded in five straight invocations (whereas on
`main`, it consistently fails with a variety of different traces).
This should be essentially the exact same behaviour, but backon is a
total API redesign, so things had to be expressed slightly differently.
Overall I think the code is more readable, which is nice.
Fixes#10001
## Summary
A few places where there are extra conversions to and from string that
seem unnecessary; a few places where we're using `PathBuf` instead of
`PortablePathBuf`.
Hello! 🙂
## Summary
After submitting retry mechanisms on scripts installation for windows:
#9543 , I noticed that some other functions were using the same
`persist` features of temporary files. This could lead to the same issue
spotted before (temporary lock by AV/EDR software). I validated that it
was possible.
So I updated them to go through the same function on Windows, which is
using the retry mechanisms if needed.
In order to do so, I add to add an async version of the
`persist_with_retry`.
There is a little trick to make the borrow-checker happy line 306,
curious of your opinion on it? This is just a pointer move so it should
not induce some performance regression if I'm not mistaking.
I also updated them to use `fs_err` on Unix for better error messages.
Also, one of the error messages I introduced was badly formatted, I
fixed it. 🙂
## Test Plan
The changes should be iso functional and covered with the existing
test-suite.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Since the `backoff` dependency is only *used* on Windows in practice,
this PR would ensure that it is only *compiled* on Windows, too. This is
helpful because it appears to be unmaintained upstream,
https://github.com/astral-sh/uv/issues/10001, and it would be nice to be
able to [drop it from
Fedora](https://bugzilla.redhat.com/show_bug.cgi?id=2329729).
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
```
$ cargo run python install
$ cargo test
```
## Summary
If the shell is currently in a directory that no longer exists, uv will
panic from any command. Panicking is a confusing behavior to those
unfamiliar with Rust and can sometimes make it hard to determine the
true issue.
Closes#9875
## Test Plan
The reproduction steps in the issue report were followed and uv no
longer panics. `uv version` can still successfully print the version if
the directory does exist.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
In the end, the problem is that `relative_to` has incorrect behavior if
either path is non-normalize (e.g., `foo/bar/../project`). So I've fixed
that method, but we _also_ now normalize `project` upfront, which _also_
fixes the issue.
Closes https://github.com/astral-sh/uv/issues/9692.
Fixes#9531
## Context
While working with [uv](https://github.com/astral-sh/uv), I encountered
issues with a python dependency, [httpx](https://www.python-httpx.org/)
unable to be installed because of a **os error 5 permission denied**.
The error occur when we try to persist a **.exe file** from a temporary
folder into a persistent one.
I only reproduce the issue in an enterprise **Windows** Jenkins Runner.
In my virtual machines, I don't have any issues. So I think this is most
probably coming from the system configuration. This windows runner
**contains an AV/EDR**. And the fact that the file locked occured only
once for an executable make me think that it's most probably the cause.
While doing some research and speaking with some colleagues (hi
@vmeurisse), it seems that the issue is a very recurrent one on Windows.
In the Javascript ecosystem, there is this package, created by the
@isaacs, `npm` inventor: https://www.npmjs.com/package/graceful-fs, used
inside `npm`, allowing its package installations to be more resilient to
filesystem errors:
> The improvements are meant to normalize behavior across different
platforms and environments, and to make filesystem access more resilient
to errors.
One of its core feature is this one:
> On Windows, it retries renaming a file for up to one second if EACCESS
or EPERM error occurs, likely because antivirus software has locked the
directory.
So I tried to implement the same algorithm on `uv`, **and it fixed my
issue**! I can finally install `httpx`.
Then, [as I mentionned in this
issue](https://github.com/astral-sh/uv/issues/9531#issuecomment-2508981316),
I saw that you already implemented exactly the same algorithm in an
asynchronous function for renames 😄22fd9f7ff1/crates/uv-fs/src/lib.rs (L221)
## Summary of changes
- I added a similar function for `persist` (was not easy to have the
benediction of the borrow checker 😄)
- I added a `sync` variant of `rename_with_retry`
- I edited `install_script` to use the function including retries on
Windows
Let me know if I should change anything 🙂
Thanks!!
## Test Plan
This pull-request should be totally iso-functional, so I think it should
be covered by existing tests in case of regression.
All tests are still passing on my side.
Also, of course validated that my windows machines (windows 10 & windows
11) containing AV/EDR software are now able to install `httpx.exe`
script.
However, if you think any additional test is needed, feel free to tell
me!
Currently, user display returns an empty path if the current dir is the
directory we are printing. This leads to odd messages such as
> Including project.license-files at `` with `LICENSE*`
or
> Not a license files match: ``
Instead, we display the current path as a dot.
## Summary
These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
Incorporating #8637 into #8458
- Adds `python-managed` feature selection to Windows CI for `python
install` tests
- Adds trampoline sniffing utilities to `uv-trampoline-builder`
- Uses a trampoline to install Python executables into the `PATH` on
Windows
A temporary fix for https://github.com/astral-sh/uv/issues/8298 while we
wait for my slower upstream fix at
https://github.com/indygreg/python-build-standalone/pull/373
I think we'll want this machinery anyway to ensure that the various
executable names are available? Otherwise we need to special-case all
the `python` names in `uv run`?
We don't have unit test coverage of managed downloads, so I added an
[integration
test](3170395680)
similar to what we have for Linux.
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html
Before that, there were 91 separate integration tests binary.
(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
## Summary
This PR adds the ability to list available scripts in the environment
when `uv run` is invoked without any arguments.
It somewhat mimics the behavior of `rye run` command
(See https://rye.astral.sh/guide/commands/run).
This is an attempt to fix#4024.
## Test Plan
I added test cases. The CI pipeline should pass.
### Manuel Tests
```shell
❯ uv run
Provide a command or script to invoke with `uv run <command>` or `uv run script.py`.
The following scripts are available:
normalizer
python
python3
python3.12
See `uv run --help` for more information.
```
---------
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
This PR modifies our parsing to allow spaces in URLs. I don't know if
this is a correct change... But we now parse URLs until we see:
- A newline.
- A semicolon (marker) or hash (comment), _preceded_ by a space. We
parse the URL until the last
non-whitespace character (inclusive).
- A semicolon (marker) or hash (comment) _followed_ by a space. We treat
this as an error, since
the end of the URL is ambiguous (e.g., `https://foo.com; marker`) would
be a URL that ends in `;`).
Closes https://github.com/astral-sh/uv/issues/6032.
## Summary
A few of these should use `absolute` instead of `canonicalize`; and
apparently we no longer need to strip the `CANONICAL_CWD` to get tests
passing.
## Summary
This PR addresses an issue on Windows where `std::fs::canonicalize` can
fail or panic when resolving paths on mapped network drives. By
replacing it with `Simplified::simple_canonicalize`, we aim to improve
the robustness and cross-platform compatibility of path resolution.
### Changes
* Updated `CANONICAL_CWD` in `path.rs` to use
`Simplified::simple_canonicalize` instead of `std::fs::canonicalize`.
### Why
* `std::fs::canonicalize` has known issues with resolving paths on
mapped network drives on Windows, which can lead to panics or incorrect
path resolution.
* `Simplified::simple_canonicalize` internally uses
`dunce::canonicalize`, which handles these cases more gracefully,
ensuring better stability and reliability.
## Test Plan
Since `simple_canonicalize` has already been tested in a prior PR, this
change is expected to work without introducing any new issues. No
additional tests are necessary beyond ensuring existing tests pass,
which will confirm the correctness of the change.
## Summary
Closes https://github.com/astral-sh/uv/issues/6840.
## Test Plan
```
❯ ~/workspace/uv/target/debug/uv pip list --verbose
DEBUG uv 0.4.0
DEBUG Searching for Python interpreter in system path
DEBUG Found `cpython-3.12.3-macos-aarch64-none` at `/Users/crmarsh/.local/share/rtx/installs/python/3.12.3/bin/python3` (search path)
DEBUG Using Python 3.12.3 environment at /Users/crmarsh/.local/share/rtx/installs/python/3.12.3/bin/python3
```
This is achieved by updating the `LockedFile::acquire` API to be async —
as in some cases we were attempting to acquire the lock synchronously,
i.e., without yielding, which blocked the runtime.
Closes https://github.com/astral-sh/uv/issues/6691 — I tested with the
reproduction there and a local release build and no longer reproduce the
deadlock with these changes.
Some additional context in the [internal Discord
thread](1278478941)
