I was looking into `uv tool` not supporting version files, and noticed
this implementation was confusing and skipped handling like a tracing
log if `--no-config` excludes selection a file. I've refactored it in
preparation for the next change.
> NOTE: The PRs that were merged into this feature branch have all been
independently reviewed. But it's also useful to see all of the changes
in their final form. I've added comments to significant changes
throughout the PR to aid discussion.
This PR introduces transparent Python version upgrades to uv, allowing
for a smoother experience when upgrading to new patch versions.
Previously, upgrading Python patch versions required manual updates to
each virtual environment. Now, virtual environments can transparently
upgrade to newer patch versions.
Due to significant changes in how uv installs and executes managed
Python executables, this functionality is initially available behind a
`--preview` flag. Once an installation has been made upgradeable through
`--preview`, subsequent operations (like `uv venv -p 3.10` or patch
upgrades) will work without requiring the flag again. This is
accomplished by checking for the existence of a minor version symlink
directory (or junction on Windows).
### Features
* New `uv python upgrade` command to upgrade installed Python versions
to the latest available patch release:
```
# Upgrade specific minor version
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview
```
* Transparent upgrades also occur when installing newer patch versions:
```
uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18
```
* Support for transparently upgradeable Python `bin` installations via
`--preview` flag
```
uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview
```
* Virtual environments can still be tied to a patch version if desired
(ignoring patch upgrades):
```
uv venv -p 3.10.8
```
### Implementation
Transparent upgrades are implemented using:
* Minor version symlink directories (Unix) or junctions (Windows)
* On Windows, trampolines simulate paths with junctions
* Symlink directory naming follows Python build standalone format: e.g.,
`cpython-3.10-macos-aarch64-none`
* Upgrades are scoped to the minor version key (as represented in the
naming format: implementation-minor version+variant-os-arch-libc)
* If the context does not provide a patch version request and the
interpreter is from a managed CPython installation, the `Interpreter`
used by `uv python run` will use the full symlink directory executable
path when available, enabling transparently upgradeable environments
created with the `venv` module (`uv run python -m venv`)
New types:
* `PythonMinorVersionLink`: in a sense, the core type for this PR, this
is a representation of a minor version symlink directory (or junction on
Windows) that points to the highest installed managed CPython patch
version for a minor version key.
* `PythonInstallationMinorVersionKey`: provides a view into a
`PythonInstallationKey` that excludes the patch and prerelease. This is
used for grouping installations by minor version key (e.g., to find the
highest available patch installation for that minor version key) and for
minor version directory naming.
### Compatibility
* Supports virtual environments created with:
* `uv venv`
* `uv run python -m venv` (using managed Python that was installed or
upgraded with `--preview`)
* Virtual environments created within these environments
* Existing virtual environments from before these changes continue to
work but aren't transparently upgradeable without being recreated
* Supports both standard Python (`python3.10`) and freethreaded Python
(`python3.10t`)
* Support for transparently upgrades is currently only available for
managed CPython installations
Closes#7287Closes#7325Closes#7892Closes#9031Closes#12977
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This PR is a combination of #12920 and #13754. Prior to these changes,
following a redirect when searching indexes would bypass our
authentication middleware. This PR updates uv to support propagating
credentials through our middleware on same-origin redirects and to
support netrc credentials for both same- and cross-origin redirects. It
does not handle the case described in #11097 where the redirect location
itself includes credentials (e.g.,
`https://user:pass@redirect-location.com`). That will be addressed in
follow-up work.
This includes unit tests for the new redirect logic and integration
tests for credential propagation. The automated external registries test
is also passing for AWS CodeArtifact, Azure Artifacts, GCP Artifact
Registry, JFrog Artifactory, GitLab, Cloudsmith, and Gemfury.
[Two benchmark
jobs](4433771099)
were failing with `error: cannot find attribute clap in this scope`
based on #14120. This updates the recently added `#[clap(name = rocm...`
lines to use `cfg_attr(feature = "clap",`.
Close#13922
## Summary
Add a warning if the directory given by the `--index` argument is empty.
## Test Plan
Added test case `add_index_empty_directory` in `edit.rs`
When working on support for reading global Python pins in tool
operations, I noticed that we weren't using the canonicalized Python
request in receipts — we were using the raw string provided by the user.
Since we'll need to compare these values, we should be using the
canonicalized string.
The `Tool` and `ToolReceipt` types have been updated to hold a
`PythonRequest` instead of a `String`, and `Serialize` was implemented
for `PythonRequest` so canonicalization can happen at the edge instead
of being the caller's responsibility.
Fix `uv run -p 3.7` by not using a walrus operator. Python 3.7 isn't
really supported anymore, but there's no reason to break interpreter
discovery for it.
When using `uv lock --upgrade-package=python` after changing
`requires-python`, it was possible to get into a state where the fork
markers produced corresponded to the empty set. This in turn resulted in
an empty lock file.
There was already some infrastructure in place that I think was perhaps
intended to handle this. In particular, `Lock::check_marker_coverage`
checks whether the fork markers have some overlap with the supported
environments (including the `requires-python`). But there were two
problems with this.
First is that in lock validation, this marker coverage check came
_after_ a path that returned `Preferable` (meaning that the fork markers
should be kept) when `--upgrade-package` was used. Second is that the
marker coverage check used the `requires-python` in the lock file and
_not_ the `requires-python` in the now updated `pyproject.toml`.
We attempt to solve this conundrum by slightly re-arranging lock file
validation and by explicitly checking whether the *new*
`requires-python` is disjoint from the fork markers in the lock file. If
it is, then we return `Versions` from lock file validation (indicating
that the fork markers should be dropped).
Fixes#13951
We always ignore the `clippy::struct_excessive_bools` rule and formerly
annotated this at the function level. This PR specifies the allow in
`workspace.lints.clippy` in `Cargo.toml`.
Using a companion change in the middleware
(https://github.com/TrueLayer/reqwest-middleware/pull/235, forked&tagged
pending review), we can check and show retries for HTTP status core
errors, to consistently report retries again.
We fix two cases:
* Show retries for status code errors for cache client requests
* Show retries for status code errors for Python download requests
Not handled:
* Show previous retries when a distribution download fails mid-streaming
* Perform retries when a distribution download fails mid-streaming
* Show previous retries when a Python download fails mid-streaming
* Perform retries when a Python download fails mid-streaming
(or legacy tool.uv.workspace).
This cleaves out a dedicated SourcedDependencyGroups type based on
RequiresDist but with only the DependencyGroup handling implemented.
This allows `uv pip` to read `dependency-groups` from pyproject.tomls
that only have that table defined, per PEP 735, and as implemented by
`pip`.
However we want our implementation to respect various uv features when
they're available:
* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)
As such we want to opportunistically detect "as much as possible" while
doing as little as possible when things are missing. The issue with the
old RequiresDist path was that it fundamentally wanted to build the
package, and if `[project]` was missing it would try to desperately run
setuptools on the pyproject.toml to try to find metadata and make a hash
of things.
At the same time, the old code also put in a lot of effort to try to
pretend that `uv pip` dependency-groups worked like `uv`
dependency-groups with defaults and non-only semantics, only to separate
them back out again. By explicitly separating them out, we confidently
get the expected behaviour.
Note that dependency-group support is still included in RequiresDist, as
some `uv` paths still use it. It's unclear to me if those paths want
this same treatment -- for now I conclude no.
Fixes#13138
This allows you to specify requires-python on individual dependency-groups,
with the intended usecase being "oh my dev-dependencies have a higher
requires-python than my actual project".
This includes a large driveby move of the RequiresPython type to
uv-distribution-types to allow us to generate the appropriate markers at
this point in the code. It also migrates RequiresPython from
pubgrub::Range to version_ranges::Ranges, and makes several pub(crate)
items pub, as it's no longer defined in uv_resolver.
Fixes#11606
In the case where we have platform information in a Python request, we
should filter managed Python distributions by it prior to querying them.
Closes https://github.com/astral-sh/uv/issues/13935
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
Often, HTTP requests don't fail due to server errors, but from spurious
network errors such as connection resets. reqwest surfaces these as
`io::Error`, and we have to handle their retrying separately.
Companion PR: https://github.com/LukeMathWalker/wiremock-rs/pull/159
Unlike regular packages, specifying all `__init__.py` directories for a
namespace package would be very verbose There is e.g.
https://github.com/python-poetry/poetry/tree/main/src/poetry, which has
18 modules, or https://github.com/googleapis/api-common-protos which is
inconsistently nested. For both the Google Cloud SDK, there are both
packages with a single module and those with complex structures, with
many having multiple modules due to versioning through `<module>_v1`
versioning. The Azure SDK seems to use one module per package (it's not
explicitly documented but seems to follow from the process in
https://azure.github.io/azure-sdk/python_design.html#azure-sdk-distribution-packages
and
ccb0e03a3d/doc/dev/packaging.md).
For simplicity with complex projects, we add a `namespace = true` switch
which disabled checking for an `__init__.py`. We only check that there's
no `<module_root>/<module_name>/__init__.py` and otherwise add the whole
`<module_root>/<module_name>` folder. This comes at the cost of
`namespace = true` effectively creating an opt-out from our usual checks
that allows creating an almost entirely arbitrary package.
For simple projects with only a single module, the module name can be
dotted to point to the target module, so the build still gets checked:
```toml
[tool.uv.build-backend]
module-name = "poetry.core"
```
## Alternatives
### Declare all packages
We could make `module-name` a list and allow or require declaring all
packages:
```toml
[tool.uv.build-backend]
module-name = ["cloud_sdk.service.storage", "cloud_sdk.service.storage_v1", "cloud_sdk.billing.storage"]
```
Or for Poetry:
```toml
[tool.uv.build-backend]
module-name = [
"poetry.config",
"poetry.console",
"poetry.inspection",
"poetry.installation",
"poetry.json",
"poetry.layouts",
"poetry.masonry",
"poetry.mixology",
"poetry.packages",
"poetry.plugins",
"poetry.publishing",
"poetry.puzzle",
"poetry.pyproject",
"poetry.repositories",
"poetry.toml",
"poetry.utils",
"poetry.vcs",
"poetry.version"
]
```
### Support multiple namespaces
We could also allow namespace packages with multiple root level module:
```toml
[tool.uv.build-backend]
module-name = ["cloud_sdk.my_ext", "local_sdk.my_ext"]
```
For lack of use cases, we delegate this to creating a workspace with one
package per module.
## Implementation
Due to the more complex options for the module name, I'm moving
verification on deserialization later, dropping the source span we'd get
from serde. We also don't show similarly named directories anymore.
---------
Co-authored-by: Andrew Gallant <andrew@astral.sh>
Investigating #13744
I tried reproducing here by running the test in a loop, but could not. I
presume it's an interaction with other tests.
This drops the snapshot, but I think it's worth it to try to examine the
flake?
Use TTY detection to determine when we should forward SIGINT instead of
counting signals, which can lead to various problems where multiple
SIGINTs are sent to a child after the first signal. Counting does not
make sense in interactive situations that do not exit on interrupt,
e.g., the Python REPL.
Closes https://github.com/astral-sh/uv/issues/13919
Closes https://github.com/astral-sh/uv/issues/12108
As another follow-up in the vein of
https://github.com/astral-sh/uv/pull/13944, I noticed `uv python pin`
doesn't download Python versions, which is a bit weird because we'll
warn it's not found.
See https://github.com/astral-sh/uv/issues/13935#issuecomment-2957300516
where we fail to write a pin file because we encounter an unusable
interpreter. This is actually a special case where `MissingPython` is
not raised because we want to show why we failed to find a usable
interpreter, which is useful in commands where you _need_ an interpreter
to use, but here we don't actually need it. Here, we just log the
failure and move on.
Related https://github.com/astral-sh/uv/pull/13936
Add basic tests for error messages on retryable network errors.
This test mod is intended to grow to ensure that we handle retryable
errors correctly and that we show the appropriate error message if we
failed after retrying.
The starter tests show some common cases we've seen download errors in:
simple and find links indexes, file downloads and Python installs.
For `io::Error` fault injection to test the reqwest `Err` path besides
the HTTP status code `Ok` path, see
https://github.com/LukeMathWalker/wiremock-rs/issues/149.
As per #13874, passing a relative URL like `test` to `--index` for `uv
add` causes unexpected behavior if the directory does not exist. The
non-existent index is effectively ignored and uv falls back to PyPI. If
a package is found there, the spurious index is then written to
`pyproject.toml`. This doesn't happen for `--default-index` since
resolution will fail without fallback to PyPI.
This PR adds a validation step for indexes provided on the command line.
If a directory does not exist, uv will fail with an error.
Closes#13874
For the case where there was no matching wheel on sync, we previously
added a note about which wheels are available vs. on which platform you
are on. We extend this error message to link directly towards
`tool.uv.required-environments`, which otherwise has a discovery
problem.
On Linux (Setting `tool.uv.required-environments` doesn't help here
either, but it's a clear example):
```
[project]
name = "debug"
version = "0.1.0"
requires-python = "==3.10.*"
dependencies = ["tensorflow-macos>=2.13.1"]
```
```
Resolved 41 packages in 24ms
error: Distribution `tensorflow-macos==2.16.2 @ registry+https://pypi.org/simple` can't be installed because it doesn't have a source distribution or wheel for the current platform
hint: You're on Linux (`manylinux_2_39_x86_64`), but there are no wheels for the current platform, consider configuring `tool.uv.required-environments`.
hint: `tensorflow-macos` (v2.16.2) only has wheels for the following platform: `macosx_12_0_arm64`.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Users are not (yet) properly familiar with the concept of universal
resolution and its implication that we need to resolve for all possible
platforms and Python versions. Some projects only target a specific
platform or Python version, and users experience resolution errors due
to failures for other platforms. Indicated by the number of questions we
get about it, `tool.uv.environments` for restricting environments is not
well discoverable.
We add a special hint when resolution failed on a fork disjoint with the
current environment, hinting the user to constrain `requires-python` and
`tool.uv.environments` respectively.
The hint has false positives for cases where the resolution failed on a
different platform, but equally fails on the current platform, in cases
where the non-current fork was tried earlier. Given that conflicts can
be based on `requires-python`, afaik we can't parse whether the current
platform would also be affected from the derivation tree.
Two cases not covered by this are build errors as well as install errors
that need `tool.uv.required-environments`.
- Define all list elements using `-`: it used to be a mix of `*` and
`-`. `-` is what Prettier linter formats it to by default.
- Removed unnecessary blank line between 2 list elements. Other elements
were stitched together without blank lines in between.
- Only the first list element started in sentence case (capital letter
first) - I made all start like so.
