It turns out that we use `UniversalMarker::pep508` quite a bit. To the
point that it makes sense to pre-compute it when constructing a
`UniversalMarker`.
This still isn't necessarily the fastest thing we can do, but this
results in a major speed-up and `without_extras` no longer shows up for
me in a profile.
Motivating benchmarks. First, from #10430:
```
$ hyperfine 'rm -f uv.lock && uv lock' 'rm -f uv.lock && uv-ag-optimize-without-extras lock'
Benchmark 1: rm -f uv.lock && uv lock
Time (mean ± σ): 408.3 ms ± 276.6 ms [User: 333.6 ms, System: 111.1 ms]
Range (min … max): 316.9 ms … 1195.3 ms 10 runs
Warning: The first benchmarking run for this command was significantly slower than the rest (1.195 s). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.
Benchmark 2: rm -f uv.lock && uv-ag-optimize-without-extras lock
Time (mean ± σ): 209.4 ms ± 2.2 ms [User: 209.8 ms, System: 103.8 ms]
Range (min … max): 206.1 ms … 213.4 ms 14 runs
Summary
rm -f uv.lock && uv-ag-optimize-without-extras lock ran
1.95 ± 1.32 times faster than rm -f uv.lock && uv lock
```
And now from #10438:
```
$ hyperfine 'uv pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null' 'uv-ag-optimize-without-extras pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null'
Benchmark 1: uv pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null
Time (mean ± σ): 12.718 s ± 0.052 s [User: 12.818 s, System: 0.140 s]
Range (min … max): 12.650 s … 12.815 s 10 runs
Benchmark 2: uv-ag-optimize-without-extras pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null
Time (mean ± σ): 419.5 ms ± 6.7 ms [User: 434.7 ms, System: 100.6 ms]
Range (min … max): 412.7 ms … 434.3 ms 10 runs
Summary
uv-ag-optimize-without-extras pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null ran
30.32 ± 0.50 times faster than uv pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null
```
Fixes#10430, Fixes#10438
## Summary
If a user provides a constraint like `flask==3.0.0`, that gets expanded
to `[3.0.0, 3.0.0+[max])`. So it's not a _singleton_, but it should be
treated as such for the purposes of prioritization, since in practice it
will almost always map to a single version.
## Summary
The issue here is that we add `urllib3{python_full_version >= '3.8'}` as
a dependency, then `requests{python_full_version >= '3.8'}`, which adds
`urllib3`, but at that point, we haven't expanded
`urllib3{python_full_version >= '3.8'}`, so we "lose" the singleton
constraint. The solution is to ensure that we visit proxies eagerly, so
that we accumulate constraints as early as possible.
Closes
https://github.com/astral-sh/uv/issues/10425#issuecomment-2580324578.
## Summary
This PR revives https://github.com/astral-sh/uv/pull/7827 to improve
tool resolutions such that, if the resolution fails, and the selected
interpreter doesn't match the required Python version from the solve, we
attempt to re-solve with a newly-discovered interpreter that _does_
match the required Python version.
For now, we attempt to choose a Python interpreter that's greater than
the inferred `requires-python`, but compatible with the same Python
minor. This helps avoid successive failures for cases like Posting,
where choosing Python 3.13 fails because it has a dependency that lacks
source distributions and doesn't publish any Python 3.13 wheels. We
should further improve the strategy to solve _that_ case too, but this
is at least the more conservative option...
In short, if you do `uv tool instal posting`, and we find Python 3.8 on
your machine, we'll detect that `requires-python: >=3.11`, then search
for the latest Python 3.11 interpreter and re-resolve.
Closes https://github.com/astral-sh/uv/issues/6381.
Closes https://github.com/astral-sh/uv/issues/10282.
## Test Plan
The following should succeed:
```
cargo run python uninstall --all
cargo run python install 3.8
cargo run tool install posting
```
In the logs, we see:
```
...
DEBUG No compatible version found for: posting
DEBUG Refining interpreter with: Python >=3.11, <3.12
DEBUG Searching for Python >=3.11, <3.12 in managed installations or search path
DEBUG Searching for managed installations at `/Users/crmarsh/.local/share/uv/python`
DEBUG Skipping incompatible managed installation `cpython-3.8.20-macos-aarch64-none`
DEBUG Found `cpython-3.13.1-macos-aarch64-none` at `/opt/homebrew/bin/python3` (search path)
DEBUG Skipping interpreter at `/opt/homebrew/opt/python@3.13/bin/python3.13` from search path: does not satisfy request `>=3.11, <3.12`
DEBUG Found `cpython-3.11.7-macos-aarch64-none` at `/opt/homebrew/bin/python3.11` (search path)
DEBUG Re-resolving with Python 3.11.7
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.7
DEBUG Solving with target Python version: >=3.11.7
DEBUG Adding direct dependency: posting*
DEBUG Searching for a compatible version of posting (*)
...
```
Ref https://github.com/astral-sh/uv/issues/10344
Not a performance optimization, but the function had become too large.
No logic changes, just code moving around. Looks slightly better when
ignoring whitespace changes.
It's still too complex but i haven't found an apt simplification.
## Summary
When `--upgrade` is provided, we should retain already-installed
packages _if_ they're newer than whatever is available from the
registry.
Closes https://github.com/astral-sh/uv/issues/10089.
## Summary
Sort of undecided on this. These are already stored as `dyn Reporter` in
each struct, so we're already using dynamic dispatch in that sense. But
all the methods take `impl Reporter`. This is sometimes nice (the
callsites are simpler?), but it also means that in practice, you often
_can't_ pass `None` to these methods that accept `Option<impl
Reporter>`, because Rust can't infer the generic type.
Anyway, this adds more consistency and simplifies the setup by using
`Arc<dyn Reporter>` everywhere.
## Summary
This PR extends #10046 to also handle architectures, which allows us to
correctly include `2.5.1` on the `cu124` index for ARM Linux.
Closes https://github.com/astral-sh/uv/issues/9655.
## Summary
This PR introduces a `LockTarget`, which is peer to `InstallTarget` and
enables us to capture the common functionality necessary to support
locking.
For now, to minimize changes, only the `Workspace` target is
implemented. In a future PR, I'll add a `Script` target for both locking
and installing.
## Summary
The proximate motivation is that I want to add new variant for scripts,
but `uv-resolver` can't depend on `uv-scripts` without creating a
circular dependency. However, I think this _does_ just make more sense
-- the resolver crate shouldn't be coupled to the various kinds of
workspaces, and these details are mostly encoded in `projects/lock.rs`
and similar files.
## Summary
This is necessary for some future improvements to non-`[project]`
workspaces and PEP 723 scripts. It's not "breaking", but it will
invalidate lockfiles for non-`[project]` workspaces. I think that's
okay, since we consider those legacy right now, and they're really rare.
## Summary
A few places where there are extra conversions to and from string that
seem unnecessary; a few places where we're using `PathBuf` instead of
`PortablePathBuf`.
## Summary
This is yet another variation on
https://github.com/astral-sh/uv/pull/9928, with a few minor changes:
1. It only applies to local versions (e.g., `2.5.1+cpu`).
2. It only _considers_ the non-local version as an alternative (e.g.,
`2.5.1`).
3. It only _considers_ the non-local alternative if it _does_ support
the unsupported platform.
4. Instead of failing, it falls back to using the local version.
So, this is far less strict, and is effectively designed to solve
PyTorch but nothing else. It's also not user-configurable, except by way
of using `environments` to exclude platforms.
uv gives priorities to packages by package name, not by virtual package
(`PubGrubPackage`). pubgrub otoh when prioritizing order the virtual
packages. When the order of virtual packages changes, uv changes its
resolutions and error messages. This means uv was depending on
implementation details of pubgrub's prioritization caching.
This broke with https://github.com/pubgrub-rs/pubgrub/pull/299, which
added a tiebreaker term that made pubgrub's sorting deterministic given
a deterministic ordering of allocating the packages (which happens the
first time pubgrub sees a package).
The new custom tiebreaker decreases the difference to upstream pubgrub.
Previously, the batch prefetcher was part of the solver loop, used
across forks. This would lead to each preference in a fork being counted
as a tried version, so that after 5 forks with the identical version, we
would start batch prefetching. The reported numbers of tried versions
are also reported. By tracking the batch prefetcher on the fork the
numbers are corrected.
An alternative would be tracking the actually tried versions, but that
would mean more overhead in the top level solver loop when the current
heuristic works.
In `ecosystem/transformers`:
```
$ hyperfine --runs 10 --prepare "rm -f uv.lock" "../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z" "uv lock --exclude-newer 2024-08-08T00:00:00Z"
Benchmark 1: ../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z
Time (mean ± σ): 386.2 ms ± 6.1 ms [User: 396.0 ms, System: 144.5 ms]
Range (min … max): 378.5 ms … 397.9 ms 10 runs
Benchmark 2: uv lock --exclude-newer 2024-08-08T00:00:00Z
Time (mean ± σ): 422.0 ms ± 5.5 ms [User: 459.6 ms, System: 190.3 ms]
Range (min … max): 415.0 ms … 430.5 ms 10 runs
Summary
../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z ran
1.09 ± 0.02 times faster than uv lock --exclude-newer 2024-08-08T00:00:00Z
```
## Summary
With the advent of `--fork-strategy requires-python` (the default), we
actually _want_ to solve higher lower-bound forks before lower
lower-bound forks. The former ensures we get the most compatible
versions, while the latter ensures we get fewer overall versions. These
two strategies match up with `--fork-strategy`, but need to be respected
as such.
Closes https://github.com/astral-sh/uv/issues/9998.
## Summary
A revival of an old idea (#9344) that I have slightly more confidence in
now. I abandoned this idea because (1) it couldn't capture that, e.g.,
`platform_system == 'Windows' and sys_platform == 'foo'` (or some other
unknown value) are disjoint, and (2) I thought that Android returned
`"android"` for one of `sys_platform` or `platform_system`, which
would've made this logic incorrect.
However, it looks like Android... doesn't do that? And the values here
are almost always in a small, known set. So in the end, the tradeoffs
here actually seem pretty good.
Vis-a-vis our current solution, this can (e.g.) _simplify out_
expressions like `sys_platform == 'win32' or platform_system ==
'Windows'`.
Closes https://github.com/astral-sh/uv/issues/9891
There are two changes here
1. We now exclude pre-releases (if they are not allowed) from the
available versions set when simplifying ranges, this means the
simplified range reflects the _allowed_ available versions — which is
what we want. We no longer segment ranges into arbitrary looking
segments..
2. We improve on #9885, expanding the scope to avoid regressions where
we would now otherwise enumerate a bunch of versions
---------
Co-authored-by: konsti <konstin@mailbox.org>
Build failures are one of the most common user facing failures that
aren't "obivous" errors (such as typos) or resolver errors. Currently,
they show more technical details than being focussed on this being an
error in a subprocess that is either on the side of the package or -
more likely - in the build environment, e.g. the user needs to install a
dev package or their python version is incompatible.
The new error message clearly delineates the part that's important (this
is a build backend problem) from the internals (we called this hook) and
is consistent about which part of the dist building stage failed. We
have to calibrate the exact wording of the error message some more. Most
of the implementation is working around the orphan rule, (this)error
rules and trait rules, so it came out more of a refactoring than
intended.
Example:
In a message like
```
❯ echo "numpy>2" | uv pip compile -p 3.8 -
× No solution found when resolving dependencies:
╰─▶ Because the requested Python version (>=3.8.0) does not satisfy Python>=3.10 and the requested
Python version (>=3.8.0) does not satisfy Python>=3.9,<3.10, we can conclude that Python>=3.9 is incompatible.
And because numpy>=2.0.1,<=2.0.2 depends on Python>=3.9 and only the following versions of numpy are available:
numpy<=2.0.2
```
I'm surprised that `-p 3.8` leads to expressions like `>=3.8.0` (I
understand it, of course, but it's not intuitive) and then all the
_other_ Python versions in the message omit the trailing zero. This
updates the `PythonRequirement` parsing to drop the trailing zeros. It's
easier to do there because the version is not yet abstracted.
