Initially, we were limiting Git schemes to HTTPS and SSH as only
supported schemes. We lost this validation in #3429. This incidentally
allowed file schemes, which apparently work with Git out of the box.
A caveat for this is that in tool.uv.sources, we parse the git field
always as URL. This caused a problem with #11425: repo = { git =
'c:\path\to\repo', rev = "xxxxx" } was parsed as a URL where c: is the
scheme, causing a bad error message down the line.
This PR:
* Puts Git URL validation back in place. It bans everything but HTTPS,
SSH, and file URLs. This could be a breaking change, if users were using
a git transport protocol were not aware of, even though never
intentionally supported.
* Allows file: URL in Git: This seems to be supported by Git and we were
supporting it albeit unintentionally, so it's reasonable to continue to
support it.
* It does not allow relative paths in the git field in tool.uv.sources.
Absolute file URLs are supported, whether we want relative file URLs for
Git too should be discussed separately.
Closes#3429: We reject the input with a proper error message, while
hinting the user towards file:. If there's still desire for relative
path support, we can keep it open.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
We want to build `uv-build` without depending on the network crates. In
preparation for that, we split uv-git into uv-git and uv-git-types,
where only uv-git depends on reqwest, so that uv-build can use
uv-git-types.
## Summary
This PR revives https://github.com/astral-sh/uv/pull/10017, which might
be viable now that we _don't_ enforce any platforms by default.
The basic idea here is that users can mark certain platforms as required
(empty, by default). When resolving, we ensure that the specified
platforms have wheel coverage, backtracking if not.
For example, to require that we include a version of PyTorch that
supports Intel macOS:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = ["torch>1.13"]
[tool.uv]
required-platforms = [
"sys_platform == 'darwin' and platform_machine == 'x86_64'"
]
```
Other than that, the forking is identical to past iterations of this PR.
This would give users a way to resolve the tail of issues in #9711, but
with manual opt-in to supporting specific platforms.
## Summary
We need to add indexes in the order in which they're respected by the
resolver. Otherwise, we risk writing an index to the `pyproject.toml`
that is canonically equal (but not verbatim equivalent) to the index we
use during resolutin.
Closes https://github.com/astral-sh/uv/issues/11312.
## Summary
This lets us drop a dependency entirely. `percent-encoding` is used by
`url` and so is already in the graph, whereas `urlencoding` isn't used
by anything else.
## Summary
This PR adds an additional normalization step to `CanonicalUrl` whereby
we now percent-decode the path, to ensure that (e.g.)
`torch-2.5.1%2Bcpu.cxx11.abi-cp39-cp39-linux_x86_64.whl` and
`torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl` are considered
equal. Further, when generating the "reinstall" report, we use the
canonical URL rather than the verbatim URL.
In making this change, I also learned that we don't apply any of the
normalization passes to `file://` URLs. I inadvertently removed it in
93d606aba2,
since setting the password or URL on ` file://` URL errors -- but now
suppress those errors anyway.
Closes https://github.com/astral-sh/uv/issues/11082.
## Test Plan
- Downloaded a [PyTorch
wheel](https://download.pytorch.org/whl/cpu-cxx11-abi/torch-2.5.1%2Bcpu.cxx11.abi-cp39-cp39-linux_x86_64.whl)
- `python3.9 -m pip install
torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl --platform
linux_x86_64 --target foo --no-deps`
- `cargo run pip install
torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl --python-platform
linux --python-version 3.9 --target foo --no-deps`
- Verified that the package had the `~` symbol for the reinstall.
## One-liner
Relative find-links configuration to local path from a pyproject.toml or
uv.toml is now relative to the config file
## Summary
### Background
One can configure find-links in a `pyproject.toml` or `uv.toml` file,
which are located from the cli arg, system directory, user directory, or
by traversing parent directories until one is encountered.
This PR addresses the following scenario:
- A project directory which includes a `pyproject.toml` or `uv.toml`
file
- The config file includes a `find-links` option. (eg under `[tool.uv]`
for `pyproject.toml`)
- The `find-links` option is configured to point to a local subdirectory
in the project: `packages/`
- There is a subdirectory called `subdir`, which is the current working
directory
- I run `uv run my_script.py`. This will locate the `pyproject.toml` in
the parent directory
### Current Behavior
- uv tries to use the path `subdir/packages/` to find packages, and
fails.
### New Behavior
- uv tries to use the path `packages/` to find the packages, and
succeeds
- Specifically, any relative local find-links path will resolve to be
relative to the configuration file.
### Why is this behavior change OK?
- I believe no one depends on the behavior that a relative find-links
when running in a subdir will refer to different directories each time
- Thus this change only allows a more common use case which didn't work
previously.
## Test Plan
- I re-created the setup mentioned above:
```
UvTest/
├── packages/
│ ├── colorama-0.4.6-py2.py3-none-any.whl
│ └── tqdm-4.67.1-py3-none-any.whl
├── subdir/
│ └── my_script.py
└── pyproject.toml
```
```toml
# pyproject.toml
[project]
name = "uvtest"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"tqdm>=4.67.1",
]
[tool.uv]
offline = true
no-index = true
find-links = ["packages/"]
```
- With working directory under `subdir`, previously, running `uv sync
--offline` would fail resolving the tdqm package, and after the change
it succeeds.
- Additionally, one can use `uv sync --show-settings` to show the
actually-resolved settings - now having the desired path in
`flat_index.url.path`
## Alternative designs considered
- I considered modifying the `impl Deserialize for IndexUrl` to parse
ahead of time directly with a base directory by having a custom
`Deserializer` with a base dir field, but it seems to contradict the
design of the serde `Deserialize` trait - which should work with all
`Deserializer`s
## Future work
- Support for adjusting all other local-relative paths in `Options`
would be desired, but is out of scope for the current PR.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
We can retain the small-size advantage of our new tags by moving the
"unknown tag" case into `WheelTagLarge`. This ensures that we can still
represent unknown tags, but avoid paying the cost for them.
## Summary
This PR modifies the lockfile to omit versions for source trees that use
`dynamic` versioning, thereby enabling projects to use dynamic
versioning with `uv.lock`.
Prior to this change, dynamic versioning was largely incompatible with
locking, especially for popular tools like `setuptools_scm` -- in that
case, every commit bumps the version, so every commit invalidates the
committed lockfile.
Closes https://github.com/astral-sh/uv/issues/7533.
## Summary
I previously made this required, but we now need to be able to create
these from a lockfile that _omits_ versions for dynamic source trees.
They should still be present in most cases, but it's best-effort.
## Summary
This PR extends the thinking in #10525 to platform tags, and then uses
the structured tag enums everywhere, rather than passing around strings.
I think this is a big improvement! It means we're no longer doing ad hoc
tag parsing all over the place.
## Summary
The idea here is to show both (1) an example of a compatible tag and (2)
the tags that were available, whenever we fail to resolve due to an
abscence of matching wheels.
Closes https://github.com/astral-sh/uv/issues/2777.
N.B. After fixing #10430, `ArcStr` became the fastest implementation
(and the gains were significantly reduced, down to 1-2%). See:
https://github.com/astral-sh/uv/pull/10453#issuecomment-2583344414.
## Summary
I tried out a variety of small string crates, but `Arc<str>`
outperformed them, giving a ~10% speed-up:
```console
❯ hyperfine "../arcstr lock" "../flexstr lock" "uv lock" "../arc lock" "../compact_str lock" --prepare "rm -f uv.lock" --min-runs 50 --warmup 20
Benchmark 1: ../arcstr lock
Time (mean ± σ): 304.6 ms ± 2.3 ms [User: 302.9 ms, System: 117.8 ms]
Range (min … max): 299.0 ms … 311.3 ms 50 runs
Benchmark 2: ../flexstr lock
Time (mean ± σ): 319.2 ms ± 1.7 ms [User: 317.7 ms, System: 118.2 ms]
Range (min … max): 316.8 ms … 323.3 ms 50 runs
Benchmark 3: uv lock
Time (mean ± σ): 330.6 ms ± 1.5 ms [User: 328.1 ms, System: 139.3 ms]
Range (min … max): 326.6 ms … 334.2 ms 50 runs
Benchmark 4: ../arc lock
Time (mean ± σ): 303.0 ms ± 1.2 ms [User: 301.6 ms, System: 118.4 ms]
Range (min … max): 300.3 ms … 305.3 ms 50 runs
Benchmark 5: ../compact_str lock
Time (mean ± σ): 320.4 ms ± 2.0 ms [User: 318.7 ms, System: 120.8 ms]
Range (min … max): 317.3 ms … 326.7 ms 50 runs
Summary
../arc lock ran
1.01 ± 0.01 times faster than ../arcstr lock
1.05 ± 0.01 times faster than ../flexstr lock
1.06 ± 0.01 times faster than ../compact_str lock
1.09 ± 0.01 times faster than uv lock
```
## Summary
We shouldn't consider incompatible distributions (e.g., those that don't
match the required Python version) when determining the implied markers.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
https://docs.rs/serde_json/latest/serde_json/fn.from_reader.html
suggests that
> When reading from a source against which short reads are not
efficient, such as a
[File](https://doc.rust-lang.org/std/fs/struct.File.html), you will want
to apply your own buffering because serde_json will not buffer the
input. See
[std::io::BufReader](https://doc.rust-lang.org/std/io/struct.BufReader.html).
Without this buffering, we observe a sequence of single byte reads which
can be quite inefficient depending on the underlying filesystem.
This adds buffering with `std::io::BufReader` to resolve this.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Unit tests cover this code.
<!-- How was it tested? -->
## Summary
This PR extends #10046 to also handle architectures, which allows us to
correctly include `2.5.1` on the `cu124` index for ARM Linux.
Closes https://github.com/astral-sh/uv/issues/9655.
## Summary
This should address the comment here:
https://github.com/astral-sh/uv/pull/10179#issuecomment-2569189265. We
don't compute implied markers if the marker is already `TRUE`, and we
set it to `TRUE` as soon as we see a source distribution. So if we visit
the source distribution before the wheels, we'll avoid computing these
for any irrelevant distributions.
## Summary
This is yet another variation on
https://github.com/astral-sh/uv/pull/9928, with a few minor changes:
1. It only applies to local versions (e.g., `2.5.1+cpu`).
2. It only _considers_ the non-local version as an alternative (e.g.,
`2.5.1`).
3. It only _considers_ the non-local alternative if it _does_ support
the unsupported platform.
4. Instead of failing, it falls back to using the local version.
So, this is far less strict, and is effectively designed to solve
PyTorch but nothing else. It's also not user-configurable, except by way
of using `environments` to exclude platforms.
Build failures are one of the most common user facing failures that
aren't "obivous" errors (such as typos) or resolver errors. Currently,
they show more technical details than being focussed on this being an
error in a subprocess that is either on the side of the package or -
more likely - in the build environment, e.g. the user needs to install a
dev package or their python version is incompatible.
The new error message clearly delineates the part that's important (this
is a build backend problem) from the internals (we called this hook) and
is consistent about which part of the dist building stage failed. We
have to calibrate the exact wording of the error message some more. Most
of the implementation is working around the orphan rule, (this)error
rules and trait rules, so it came out more of a refactoring than
intended.
Example:
For publishing, we want to allow all simple `[[tool.uv.index]]` entries,
whether they are explicit or not. We don't allow flat indexes here,
assuming that an index you can upload to has a simple index URL (and
generally doesn't have a flat index URL, at least I don't know any case
that has).
The `no_index` branch isn't used atm, but I left it in case the method
gathers more users.
Fixes#9919
## Summary
This PR addresses a significant limitation in the resolver whereby we
avoid choosing the latest versions of packages when the user supports a
wider range.
For example, with NumPy, the latest versions only support Python 3.10
and later. If you lock a project with `requires-python = ">=3.8"`, we
pick the last NumPy version that supported Python 3.8, and use that for
_all_ Python versions. So you get `1.24.4` for all versions, rather than
`2.2.0`. And we'll never upgrade you unless you bump your
`requires-python`. (Even worse, those versions don't have wheels for
Python 3.12, etc., so you end up building from source.)
(As-is, this is intentional. We optimize for minimizing the number of
selected versions, and the current logic does that well!)
Instead, we know recognize when a version has an elevated
`requires-python` specifier and fork. This is a new fork point, since we
need to fork once we have the package metadata, as opposed to when we
see the dependencies.
In this iteration, I've made this behavior the default. I'm sort of
undecided on whether I want to push on that... Previously, I'd suggested
making it opt-in via a setting
(https://github.com/astral-sh/uv/pull/8686).
Closes https://github.com/astral-sh/uv/issues/8492.
When publishing, we currently ask the user to set `--publish-url` to the
upload URL and `--check-url` to the simple index URL, or the equivalent
configuration keys. But that's redundant with the `[[tool.uv.index]]`
declaration. Instead, we extend `[[tool.uv.index]]` with a `publish-url`
entry and allow passing `uv publish --index <name>`.
`uv publish --index <name>` requires the `pyproject.toml` to be present
when publishing, unlike using `--publish-url ... --check-url ...` which
can be used e.g. in CI without a checkout step. `--index` also always
uses the check URL feature to aid upload consistency.
The documentation tries to explain both approaches together, which
overlap for the check URL feature.
Fixes#8864
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Instead of modifying the error to replace a dummy derivation chain from
construction with the real one, build the error with the real derivation
chain directly.
This came up when trying to improve the build error reporting.
Introduces `DistErrorKind` to avoid error variants for each case that
are only different in one line of the message.
When encountering `dynamic = ["version"]` in the pyproject.toml of a
source dist, we can ignore that and treat it as a statically known
metadata distribution, since the filename tells us the version and that
version must not change on build.
This fixed locking PyGObject 3.50.0 from `pygobject-3.50.0.tar.gz`
(minimized):
```toml
[project]
name = "PyGObject"
description = "Python bindings for GObject Introspection"
requires-python = ">=3.9, <4.0"
dependencies = [
"pycairo>=1.16"
]
dynamic = ["version"]
```
Afterwards, `uv add --no-sync toga` passes on Ubuntu 24.04 without the
pygobject build deps, when previously it needed `{ name = "pygobject",
version = "3.50.0", requires-dist = [], requires-python = ">=3.9" }`.
I've added a check that source distribution versions are respected after
build.
Fixes#9548
