This is a minimal redux of #10861 to be compatible with `uv pip`.
This implements the interface described in:
https://github.com/pypa/pip/pull/13065#issuecomment-2544000876 for `uv
pip install` and `uv pip compile`. Namely `--group <[path:]name>`, where
`path` when not defined defaults to `pyproject.toml`.
In that interface they add `--group` to `pip install`, `pip download`,
and `pip wheel`. Notably we do not define `uv pip download` and `uv pip
wheel`, so for parity we only need to implement `uv pip install`.
However, we also support `uv pip compile` which is not part of pip
itself, and `--group` makes sense there too.
----
The behaviour of `--group` for `uv pip` commands makes sense for the
cases upstream pip supports, but has confusing meanings in cases that
only we support (because reading pyproject.tomls is New Tech to them but
heavily supported by us). **Specifically case (h) below is a concerning
footgun, and case (e) below may get complaints from people who aren't
well-versed in dependency-groups-as-they-pertain-to-wheels.**
## Only Group Flags
Group flags on their own work reasonably and uncontroversially, except
perhaps that they don't do very clever automatic project discovery.
a) `uv pip install --group path/to/pyproject.toml:mygroup` pulls up
`path/to/project.toml` and installs all the packages listed by its
`mygroup` dependency-group (essentially treating it like another kind of
requirements.txt). In this regard it functions similarly to
`--only-group` in the rest of uv's interface.
b) `uv pip install --group mygroup` is just sugar for `uv pip install
--group pyproject.toml:mygroup` (**note that no project discovery
occurs**, upstream pip simply hardcodes the path "pyproject.toml" here
and we reproduce that.)
c) `uv pip install --group a/pyproject.toml:groupx --group
b/pyproject.toml:groupy`, and any other instance of multiple `--group`
flags, can be understood as completely independent requests for the
given groups at the given files.
## Groups With Named Packages
Groups being mixed with named packages also work in a fairly
unsurprising way, especially if you understand that things like
dependency-groups are not really supposed to exist on pypi, they're just
for local development.
d) `uv pip install mypackage --group path/to/pyproject.toml:mygroup`
much like multiple instances of `--group` the two requests here are
essentially completely independent: pleases install `mypackage`, and
please also install `path/to/pyproject.toml:mygroup`.
e) `uv pip install mypackage --group mygroup` is exactly the same, but
this is where it becomes possible for someone to be a little confused,
as you might think `mygroup` is supposed to refer to `mypackage` in some
way (it can't). But no, it's sourcing `pyproject.toml:mygroup` from the
current working directory.
## Groups With Requirements/Sourcetrees/Editables
Requirements and sourcetrees are where I expect users to get confused.
It behaves *exactly* the same as it does in the previous sections but
you would absolutely be forgiven for expecting a different behaviour.
*Especially* because `--group` with the rest of uv *does* do something
different.
f) `uv pip install -r a/pyproject.toml --group b/pyproject.toml:mygroup`
is again just two independent requests (install `a/pyproject.toml`'s
dependencies, and `b/pyproject.toml`'s `mygroup`).
g) `uv pip install -r pyproject.toml --group mygroup` is exactly like
the previous case but *incidentally* the two requests refer to the same
file. What the user wanted to happen is almost certainly happening, but
they are likely getting "lucky" here that they're requesting something
simple.
h) `uv pip install -r a/pyproject.toml --group mygroup` is again exactly
the same but the user is likely to get surprised and upset as this
invocation actually sources two different files (install
`a/pyproject.toml`'s dependencies, and `pyproject.toml`'s `mygroup`)! I
would expect most people to assume the `--group` flag here is covering
all applicable requirements/sourcetrees/editables, but no, it continues
to be a totally independent reference to a file with a hardcoded
relative path.
------
Fixes https://github.com/astral-sh/uv/issues/8590
Fixes https://github.com/astral-sh/uv/issues/8969
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.
In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
## Summary
This is the pattern I see in a variety of crates, and I believe this is
preferred if you don't _need_ an owned `String`, since you can avoid the
allocation. This could be pretty impactful for us?
## Summary
Since we use `SmallString` internally, there's no benefit to passing an
owned string to the `PackageName` constructor (same goes for
`ExtraName`, etc.). I've kept them for now (maybe that will change in
the future, so it's useful to have clients passed own values if they
_can_), but removed a bunch of usages where we were casting from `&str`
to `String` needlessly to use the constructor.
## Summary
* Upgrade the rust toolchain to 1.85.0. This does not increase the MSRV.
* Update windows trampoline to 1.86 nightly beta (previously in 1.85
nightly beta).
## Test Plan
Existing tests
## Motivation
No-op `uv lock` in apache airflow
(891c67f210ab7c877d1f00ea6ea3d3cdbb0e96ef) is slow, which makes `uv run`
slow, too.
Reference project:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 16.3 ms ± 1.5 ms [User: 9.8 ms, System: 6.4 ms]
Range (min … max): 13.0 ms … 20.0 ms 186 runs
```
Apache airflow before:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 161.0 ms ± 5.2 ms [User: 135.3 ms, System: 24.1 ms]
Range (min … max): 155.0 ms … 176.3 ms 18 runs
```
## Optimization
`FlatRequiresDist::from_requirements` is taking 50% of main thread
runtime.
Before:
After both commits:
Apache airflow after the first commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 122.3 ms ± 5.4 ms [User: 96.1 ms, System: 24.7 ms]
Range (min … max): 114.0 ms … 133.2 ms 23 runs
```
Apache airflow after the second commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 108.5 ms ± 3.4 ms [User: 83.2 ms, System: 24.2 ms]
Range (min … max): 103.6 ms … 119.9 ms 28 runs
```
N.B. After fixing #10430, `ArcStr` became the fastest implementation
(and the gains were significantly reduced, down to 1-2%). See:
https://github.com/astral-sh/uv/pull/10453#issuecomment-2583344414.
## Summary
I tried out a variety of small string crates, but `Arc<str>`
outperformed them, giving a ~10% speed-up:
```console
❯ hyperfine "../arcstr lock" "../flexstr lock" "uv lock" "../arc lock" "../compact_str lock" --prepare "rm -f uv.lock" --min-runs 50 --warmup 20
Benchmark 1: ../arcstr lock
Time (mean ± σ): 304.6 ms ± 2.3 ms [User: 302.9 ms, System: 117.8 ms]
Range (min … max): 299.0 ms … 311.3 ms 50 runs
Benchmark 2: ../flexstr lock
Time (mean ± σ): 319.2 ms ± 1.7 ms [User: 317.7 ms, System: 118.2 ms]
Range (min … max): 316.8 ms … 323.3 ms 50 runs
Benchmark 3: uv lock
Time (mean ± σ): 330.6 ms ± 1.5 ms [User: 328.1 ms, System: 139.3 ms]
Range (min … max): 326.6 ms … 334.2 ms 50 runs
Benchmark 4: ../arc lock
Time (mean ± σ): 303.0 ms ± 1.2 ms [User: 301.6 ms, System: 118.4 ms]
Range (min … max): 300.3 ms … 305.3 ms 50 runs
Benchmark 5: ../compact_str lock
Time (mean ± σ): 320.4 ms ± 2.0 ms [User: 318.7 ms, System: 120.8 ms]
Range (min … max): 317.3 ms … 326.7 ms 50 runs
Summary
../arc lock ran
1.01 ± 0.01 times faster than ../arcstr lock
1.05 ± 0.01 times faster than ../flexstr lock
1.06 ± 0.01 times faster than ../compact_str lock
1.09 ± 0.01 times faster than uv lock
```
## Summary
We can read from the slice directly. I don't think this will affect
performance today, because `from_str` will then allocate, but it
_should_ be a speedup once #10475 merges, since we can then avoid
allocating a `String` and go straight from `str` to `ArcStr`.
The `cdylib` was used for the pyo3 bindings to uv-pep508, which don't
exist anymore. It was now creating warnings on musl due to musl
(statically linked) no supporting shared libraries.
## Summary
A revival of an old idea (#9344) that I have slightly more confidence in
now. I abandoned this idea because (1) it couldn't capture that, e.g.,
`platform_system == 'Windows' and sys_platform == 'foo'` (or some other
unknown value) are disjoint, and (2) I thought that Android returned
`"android"` for one of `sys_platform` or `platform_system`, which
would've made this logic incorrect.
However, it looks like Android... doesn't do that? And the values here
are almost always in a small, known set. So in the end, the tradeoffs
here actually seem pretty good.
Vis-a-vis our current solution, this can (e.g.) _simplify out_
expressions like `sys_platform == 'win32' or platform_system ==
'Windows'`.
## Summary
This has been bothering me a bit: `uv pip install "foo @
https://github.com/user/foo"` fails, telling you that it doesn't end in
a supported extension. But we should be able to tell you that it looks
like a Git repo.
## Summary
This is an alternative to #9344. If accepted, I need to audit the
codebase and call sites to apply it everywhere, but the basic idea is:
rather than encoding mutually-incompatible pairs of markers in the
representation itself, we have an additional method on `MarkerTree` that
expands the false-y definition to take into account assumptions about
which markers can be true alongside others. We then check if the the
current marker implies that at least one of them is true.
So, for example, we know that `sys_platform == 'win32'` and
`platform_system == 'Darwin'` are mutually exclusive. When given a
marker expression like `python_version >= '3.7'`, we test if
`python_version >= '3.7'` and `sys_platform != 'win32' or
platform_system != 'Darwin'` are disjoint, i.e., if the following can't
be satisfied:
```
python_version >= '3.7' and (sys_platform != 'win32' or platform_system != 'Darwin')
```
Since, if this can't be satisfied, it implies that the left-hand
expression requires `sys_platform == 'win32'` and `platform_system ==
'Darwin'` to be true at the same time.
I think the main downsides here are:
1. We can't _simplify_ markers based on these implications. So we'd
still write markers like `sys_platform == 'win32' and platform_system !=
'Darwin'`, even though we know the latter expression is redundant.
2. It might be expensive? I'm not sure. I don't think we test for
falseness _that_ often though.
Closes#7760.
Closes#9275.
In the course of working on #9289, I've had to devise
some additions to our markers. While we are still staying
strictly compatible with the PEP 508 format, we will be
abusing the `extra` expression to carry a lot more
information.
Specifically, we want the following additional
operations:
* Simplify `extra != 'foo'`
* Remove all extra expressions
* Remove everything except extra expressions
My work on #9289 requires all of these (which will be
in a future in PR).
## Summary
A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.
IIUC, this _does_ require bumping our MSRV.
## Summary
We never construct these -- they should be impossible, since we always
translate to `python_full_version`. This PR encodes that impossibility
in the types.
## Summary
This PR modifies our lowered representation such that any deprecated
aliases are treated as "the same" marker in the algebra.
So, for example, we now recognize that this is impossible, despite the
marker names being different:
```
typing-extensions ; platform.python_implementation == 'CPython' and python_implementation != 'CPython'
```
Similarly, we now recognize that this is just `sys_platform == 'win32'`,
despite the presence of both markers:
```
anyio ; sys_platform == 'win32' and sys.platform == 'win32'
```
## Summary
I want to move towards a more normalized marker representation within
the marker tree, which means that the things we warn against will
disappear by the time we get to evaluation. I think it makes more sense
to show these warnings when we create the tree, rather than when we
evaluate it.
I think Ibraheem had this routine at some point in the past, but
we ended up dropping it because we didn't have a use for it. Well,
now we do!
It turns out that when we generate "conflict markers," they don't
actually take "world knowledge" into account. In particular, there
is "world knowledge" that a particular set of extras cannot be
enabled simultaneously. This in turn allows us to simplify most
conflict markers. If we didn't do this, it's likely that lock files
would become littered with conflict markers whenever any conflicts
are declared.
This is somewhat (although not completely) analogous to how we
"simplify" markers with respect to `requires-python`. That is,
`requires-python` reflects world knowledge that enables markers
to be written more simply than they otherwise would be without
world knowledge.
This test demonstrates the difference between `extra != "foo"` and
`sys_platform != "foo"`.
I wrote this test down to test the extra simplification logic was
correct. And I also wanted to test whether we could somehow hackily
encode `group` (as opposed to just `extra`) logic into marker
expressions by reusing another field. But I don't think we can.
## Summary
This PR enables something like the "final boss" of PyTorch setups --
explicit support for CPU vs. GPU-enabled variants via extras:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = []
[project.optional-dependencies]
cpu = [
"torch==2.5.1+cpu",
]
gpu = [
"torch==2.5.1",
]
[tool.uv.sources]
torch = [
{ index = "torch-cpu", extra = "cpu" },
{ index = "torch-gpu", extra = "gpu" },
]
[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
[tool.uv]
conflicts = [
[
{ extra = "cpu" },
{ extra = "gpu" },
],
]
```
It builds atop the conflicting extras work to allow sources to be marked
as specific to a dedicated extra being enabled or disabled.
As part of this work, sources now have an `extra` field. If a source has
an `extra`, it means that the source is only applied to the requirement
when defined within that optional group. For example, `{ index =
"torch-cpu", extra = "cpu" }` above only applies to
`"torch==2.5.1+cpu"`.
The `extra` field does _not_ mean that the source is "enabled" when the
extra is activated. For example, this wouldn't work:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = ["torch"]
[tool.uv.sources]
torch = [
{ index = "torch-cpu", extra = "cpu" },
{ index = "torch-gpu", extra = "gpu" },
]
[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
```
In this case, the sources would effectively be ignored. Extras are
really confusing... but I think this is correct? We don't want enabling
or disabling extras to affect resolution information that's _outside_ of
the relevant optional group.
## Summary
These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
## Summary
Like pip, we now allow the semicolon to directly proceed the URL (but
require that it's either preceded or followed by a space):
```
# OK
./test.whl; sys_platform == 'darwin'
# OK
./test.whl ;sys_platform == 'darwin'
# Error
./test.whl;sys_platform == 'darwin'
```
Closes https://github.com/astral-sh/uv/issues/8831.
