Resolves
https://github.com/astral-sh/uv/pull/13474#discussion_r2112586405
This kind of dynamic ordering freaks me out a little, but I think it's
probably the best solution and is static at compile-time.
Currently, we're just sorting by the stringified representation! which
is just convenient for reproducibility, but we rely on these orderings
for prioritization in discovery.
## Summary
Part of #12838. Allow users to configure `python-downloads-json-url` in
`uv.toml` and not just from env.
I followed similar PR #8695, so same as there it's also available in the
CLI (I think maybe it's better not to be configurable from the CLI, but
since the mirror parameters are, I think it's better to do the same)
## Test Plan
<!-- How was it tested? -->
## Summary
Add an option to overwrite the list of available Python downloads from a
local JSON file by using the environment variable
`UV_PYTHON_DOWNLOADS_JSON_URL`
as an experimental support for providing custom sources for Python
distribution binaries #8015
related #10203
I probably should make the JSON to be fetched from a remote URL instead
of a local file.
please let me know what you think and I will modify the code
accordingly.
## Test Plan
### normal run
```
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.14.0a4+freethreaded-linux-x86_64-gnu <download available>
cpython-3.14.0a4-linux-x86_64-gnu <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu <download available>
cpython-3.13.1-linux-x86_64-gnu <download available>
cpython-3.12.8-linux-x86_64-gnu <download available>
cpython-3.11.11-linux-x86_64-gnu <download available>
cpython-3.10.16-linux-x86_64-gnu <download available>
cpython-3.9.21-linux-x86_64-gnu <download available>
cpython-3.8.20-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
pypy-3.10.14-linux-x86_64-gnu <download available>
pypy-3.9.19-linux-x86_64-gnu <download available>
pypy-3.8.16-linux-x86_64-gnu <download available>
pypy-3.7.13-linux-x86_64-gnu <download available>
```
### empty JSON file
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL
{}
root@75c66494ba8b:/# /code/target/release/uv python list
root@75c66494ba8b:/#
```
### JSON file with valid version
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL
{
"cpython-3.11.9-linux-x86_64-gnu": {
"name": "cpython",
"arch": {
"family": "x86_64",
"variant": null
},
"os": "linux",
"libc": "gnu",
"major": 3,
"minor": 11,
"patch": 9,
"prerelease": "",
"url": "20240814/cpython-3.11.9%2B20240814-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz",
"sha256": "daa487c7e73005c4426ac393273117cf0e2dc4ab9b2eeda366e04cd00eea00c9",
"variant": null
}
}
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.11.9-linux-x86_64-gnu <download available>
root@75c66494ba8b:/#
```
### Remote Path
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=http://a.com/file.json
root@75c66494ba8b:/# /code/target/release/uv python list
error: Remote python downloads JSON is not yet supported, please use a local path (without `file://` prefix)
```
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
## Summary
In preview mode on windows, register und un-register the managed python build standalone installations in the Windows registry following PEP 514.
We write the values defined in the PEP plus the download URL and hash. We add an entry when installing a version, remove an entry when uninstalling and removing all values when uninstalling with `--all`. We update entries only by overwriting existing values, there is no "syncing" involved.
Since they are not official builds, pbs gets a prefix. `py -V:Astral/CPython3.13.1` works, `py -3.13` doesn't.
```
$ py --list-paths
-V:3.12 * C:\Users\Konsti\AppData\Local\Programs\Python\Python312\python.exe
-V:3.11.9 C:\Users\Konsti\.pyenv\pyenv-win\versions\3.11.9\python.exe
-V:3.11 C:\Users\micro\AppData\Local\Programs\Python\Python311\python.exe
-V:3.8 C:\Users\micro\AppData\Local\Programs\Python\Python38\python.exe
-V:Astral/CPython3.13.1 C:\Users\Konsti\AppData\Roaming\uv\data\python\cpython-3.13.1-windows-x86_64-none\python.exe
```
Registry errors are reported but not fatal, except for operations on the company key since it's not bound to any specific python interpreter.
On uninstallation, we prune registry entries that have no matching Python installation (i.e. broken entries).
The code uses the official `windows_registry` crate of the `winreg` crate.
Best reviewed commit-by-commit.
## Test Plan
We're reusing an existing system check to test different (un)installation scenarios.
## Summary
Fixes#10598
## Test Plan
Looking for input here @zanieb. How/where would you include tests for
this?
More broadly: do we want a failure to perform the rename to be a hard
error? Or should it start out as a warning?
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
This PR reimplements
[`sysconfigpatcher`](https://github.com/bluss/sysconfigpatcher) in Rust
and applies it to our Python installations at install-time, ensuring
that the `sysconfig` data is more likely to be correct.
For now, we only rewrite prefixes (i.e., any path that starts with
`/install` gets rewritten to the correct absolute path for the current
machine).
Unlike `sysconfigpatcher`, this PR does not yet do any of the following:
- Patch `pkginfo` files.
- Change `clang` references to `cc`.
A few things that we should do as follow-ups, in my opinion:
1. Rewrite
[`AR`](c1ebf8ab92/src/sysconfigpatcher.py (L61)).
2. Remove `-isysroot`, which we already do for newer builds.
## Summary
This PR adds `--install-dir` argument for the following commands:
- `uv python install`
- `uv python uninstall`
The `UV_PYTHON_INSTALL_DIR` env variable can be used to set it
(previously it was also used internally).
Any more commands we would want to add this to?
## Test Plan
For now just manual test (works on my machine hehe)
```
❯ ./target/debug/uv python install --install-dir /tmp/pythons 3.8.12
Searching for Python versions matching: Python 3.8.12
Installed Python 3.8.12 in 4.31s
+ cpython-3.8.12-linux-x86_64-gnu
❯ /tmp/pythons/cpython-3.8.12-linux-x86_64-gnu/bin/python --help
usage: /tmp/pythons/cpython-3.8.12-linux-x86_64-gnu/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
```
Open to add some tests after the initial feedback.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This pull request is best viewed with [whitespace
hidden](https://github.com/astral-sh/uv/pull/8650/files?diff=unified&w=1)
Adds a `--default` flag to `uv python install` in preview. This includes
a `python` and `python{major}` executable in addition to the
`python{major}.{minor}` executable. We will replace uv-managed
executables, but externally managed executables require the `--force`
flag to overwrite.
If you run `uv python install` (without arguments), we include the
`--default` flag implicitly to populate `python` and `python3` for the
"default" install version.
In the future, we should add a warning if the installed executable isn't
at the front of the PATH.
## Summary
A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.
IIUC, this _does_ require bumping our MSRV.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
-->
## Summary
Adds python-install-mirror and pypy-install-mirror as keys for uv.toml,
and cli args for `uv python install`.
Could leave the cli args out if we think the env vars and configs are
sufficient.
Fixes#8186
<!-- What's the purpose of the change? What does it do, and why? -->
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Updates `uv python install` to link `python3.x` in the executable
directory (i.e., `~/.local/bin`) to the the managed interpreter path.
Includes
- #8569
- #8571
Remaining work
- #8663
- #8650
- Add an opt-out setting and flag
- Update documentation
## Summary
This PR delays the removal of an existing version after downloading the
new version when running `uv python install --reinstall`.
If the download fails, we can keep the existing version working.
## Test Plan
```console
$ cargo run -- python install 3.13
$ cargo run -- python install --reinstall 3.13 # when downloading, `ctrl-c` to interrupt
$ cargo run -- python list
```
A temporary fix for https://github.com/astral-sh/uv/issues/8298 while we
wait for my slower upstream fix at
https://github.com/indygreg/python-build-standalone/pull/373
I think we'll want this machinery anyway to ensure that the various
executable names are available? Otherwise we need to special-case all
the `python` names in `uv run`?
We don't have unit test coverage of managed downloads, so I added an
[integration
test](3170395680)
similar to what we have for Linux.
## Summary
Closes#7977. Makes `PythonDownloadRequest` account for the prerelease
part if allowed. Also stores the prerelease in `PythonInstallationKey`
directly as a `Prerelease` rather than a string.
## Test Plan
Correctly picks the relevant prerelease (rather than picking the most
recent one):
```
λ cargo run python install 3.13.0rc2
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.17s
Running `target/debug/uv python install 3.13.0rc2`
Searching for Python versions matching: Python 3.13rc2
cpython-3.13.0rc2-macos-aarch64-none ------------------------------ 457.81 KiB/14.73 MiB ^C
λ cargo run python install 3.13.0rc3
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.17s
Running `target/debug/uv python install 3.13.0rc3`
Searching for Python versions matching: Python 3.13rc3
Found existing installation for Python 3.13rc3: cpython-3.13.0rc3-macos-aarch64-none
```
Closes#7118
This only really affects managed interpreters, as we exclude alternative
Python implementations from the search path during the
`VersionRequest::executable_names` part of discovery.
As we support more complex Python discovery behaviors such as:
- #7431
- #7335
- #7300
`Any` is no longer accurate, we actually are looking for a reasonable
default Python version to use which may exclude the first one we find.
Separately, we need the idea of `Any` to improve behavior when listing
versions (e.g., #7286) where we do actually want to match _any_ Python
version. As a first step, we'll rename `Any` to `Default`. Then, we'll
introduce a new `Any` that actually behaves as we'd expect.
This is achieved by updating the `LockedFile::acquire` API to be async —
as in some cases we were attempting to acquire the lock synchronously,
i.e., without yielding, which blocked the runtime.
Closes https://github.com/astral-sh/uv/issues/6691 — I tested with the
reproduction there and a local release build and no longer reproduce the
deadlock with these changes.
Some additional context in the [internal Discord
thread](1278478941)
As described in #4242, we're currently incorrectly downloading glibc
python-build-standalone on musl target, but we also can't fix this by
using musl python-build-standalone on musl targets since the musl builds
are effectively broken.
We reintroduce the libc detection previously removed in #2381, using it
to detect which libc is the current one before we have a python
interpreter. I changed the strategy a big to support an empty `PATH`
which we use in the tests.
For simplicity, i've decided to just filter out the musl
python-build-standalone archives from the list of available archive,
given this is temporary. This means we show the same error message as if
we don't have a build for the platform. We could also add a dedicated
error message for musl.
Fixes#4242
## Test Plan
Tested manually.
On my ubuntu host, python downloads continue to pass:
```
target/x86_64-unknown-linux-musl/debug/uv python install
```
On alpine, we fail:
```
$ docker run -it --rm -v .:/io alpine /io/target/x86_64-unknown-linux-musl/debug/uv python install
Searching for Python installations
error: No download found for request: cpython-any-linux-x86_64-musl
```
The loose consensus is that "fetch" doesn't have much meaning and that a
boolean flag makes more sense from the command line.
1. Adds `--allow-python-downloads` (hidden, default) and
`--no-python-downloads` to the CLI to quickly enable or disable
downloads
2. Deprecates `--python-fetch` in favor of the options from (1)
3. Removes `python-fetch` in favor of a `python-downloads` setting
5. Adds a `never` variant to the enum, allowing even explicit installs
to be disabled via the configuration file
## Test plan
I tested this with various `pyproject.toml`-level settings and `uv venv
--preview --python 3.12.2` and `uv python install 3.12.2` with and
without the new CLI flags.
## Summary
It's fine for this to be in the cache, I think, since we don't
necessarily need to colocate it with the Python directory.
Closes https://github.com/astral-sh/uv/issues/5747.
Collapses the previous default into "managed" and makes the "managed"
behavior match "installed". People should use "only-managed" if they
want that behavior, it seems overly complicated otherwise.
> DEBUG Found `cpython-3.12.1-macos-aarch64-none` at
`/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
Instead of `<implementation> <version>`
> DEBUG Found cpython 3.12.1 at `/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
## Summary
Resolves#5139
`PythonInstallationKey` was sorted as a string, which caused `3.8` to
appear before `3.11`. This update changes the sorting of
`PythonInstallationKey` to be a descending order by version.
## Test Plan
```sh
$ cargo run -- python install 3.8 3.12
$ cargo run -- tool run -v python -V
DEBUG uv 0.2.25
warning: `uv tool run` is experimental and may change without warning.
DEBUG Searching for Python interpreter in managed installations, system path, or `py` launcher
DEBUG Searching for managed installations at `C:\Users\xx\AppData\Roaming\uv\data\python`
DEBUG Found managed Python `cpython-3.12.3-windows-x86_64-none`
DEBUG Found cpython 3.12.3 at `C:\Users\xx\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none\install\python.exe` (managed installations)
DEBUG Using request timeout of 30s
DEBUG Using request timeout of 30s
DEBUG Acquired lock for `C:\Users\nigel\AppData\Roaming\uv\data\tools`
DEBUG Using existing environment for tool `httpx`: C:\Users\xx\AppData\Roaming\uv\data\tools\httpx
DEBUG Using existing tool `httpx`
DEBUG Running `httpx -v`
```
## Summary
Closes https://github.com/astral-sh/uv/issues/4848.
## Test Plan
```
> cargo run -- run -vv --preview --isolated --python 3.12.4 python -V
error: No interpreter found for Python 3.12.4 in virtual environments or managed installations or system path
```
Whew this is a lot.
The user-facing changes are:
- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"
The internal changes are:
- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
2024-07-03 07:44:29 -05:00
Renamed from crates/uv-toolchain/src/toolchain.rs (Browse further)
