uv/scripts
Charlie Marsh 5a95f50619
Add support for PyTorch-style local version semantics (#2430)
## Summary

This PR adds limited support for PEP 440-compatible local version
testing. Our behavior is _not_ comprehensively in-line with the spec.
However, it does fix by _far_ the biggest practical limitation, and
resolves all the issues that've been raised on uv related to local
versions without introducing much complexity into the resolver, so it
feels like a good tradeoff for me.

I'll summarize the change here, but for more context, see [Andrew's
write-up](https://github.com/astral-sh/uv/issues/1855#issuecomment-1967024866)
in the linked issue.

Local version identifiers are really tricky because of asymmetry.
`==1.2.3` should allow `1.2.3+foo`, but `==1.2.3+foo` should not allow
`1.2.3`. It's very hard to map them to PubGrub, because PubGrub doesn't
think of things in terms of individual specifiers (unlike the PEP 440
spec) -- it only thinks in terms of ranges.

Right now, resolving PyTorch and friends fails, because...

- The user provides requirements like `torch==2.0.0+cu118` and
`torchvision==0.15.1+cu118`.
- We then match those exact versions.
- We then look at the requirements of `torchvision==0.15.1+cu118`, which
includes `torch==2.0.0`.
- Under PEP 440, this is fine, because `torch @ 2.0.0+cu118` should be
compatible with `torch==2.0.0`.
- In our model, though, it's not, because these are different versions.
If we change our comparison logic in various places to allow this, we
risk breaking some fundamental assumptions of PubGrub around version
continuity.
- Thus, we fail to resolve, because we can't accept both `torch @ 2.0.0`
and `torch @ 2.0.0+cu118`.

As compared to the solutions we explored in
https://github.com/astral-sh/uv/issues/1855#issuecomment-1967024866, at
a high level, this approach differs in that we lie about the
_dependencies_ of packages that rely on our local-version-using package,
rather than lying about the versions that exist, or the version we're
returning, etc.

In short:

- When users specify local versions upfront, we keep track of them. So,
above, we'd take note of `torch` and `torchvision`.
- When we convert the dependencies of a package to PubGrub ranges, we
check if the requirement matches `torch` or `torchvision`. If it's
an`==`, we check if it matches (in the above example) for
`torch==2.0.0`. If so, we _change_ the requirement to
`torch==2.0.0+cu118`. (If it's `==` some other version, we return an
incompatibility.)

In other words, we selectively override the declared dependencies by
making them _more specific_ if a compatible local version was specified
upfront.

The net effect here is that the motivating PyTorch resolutions all work.
And, in general, transitive local versions work as expected.

The thing that still _doesn't_ work is: imagine if there were _only_
local versions of `torch` available. Like, `torch @ 2.0.0` didn't exist,
but `torch @ 2.0.0+cpu` did, and `torch @ 2.0.0+gpu` did, and so on.
`pip install torch==2.0.0` would arbitrarily choose one one `2.0.0+cpu`
or `2.0.0+gpu`, and that's correct as per PEP 440 (local version
segments should be completely ignored on `torch==2.0.0`). However, uv
would fail to identify a compatible version. I'd _probably_ prefer to
fix this, although candidly I think our behavior is _ok_ in practice,
and it's never been reported as an issue.

Closes https://github.com/astral-sh/uv/issues/1855.

Closes https://github.com/astral-sh/uv/issues/2080.

Closes https://github.com/astral-sh/uv/issues/2328.
2024-03-16 10:24:50 -04:00
..
bench fix: remove uv version from uv pip compile header (#1716) 2024-02-19 20:26:53 +00:00
benchmarks Rename to uv (#1302) 2024-02-15 11:19:46 -06:00
bootstrap Add musl to python bootstrapping script (#1758) 2024-02-22 10:46:48 +01:00
editable-installs Remove NumPy from editable test (#2473) 2024-03-15 14:41:52 +00:00
popular_packages Finish install-many with pypi 10k most dependents (#732) 2023-12-27 09:42:51 -05:00
release Update the lockfile in release/bump.sh (#1924) 2024-02-23 13:45:46 -06:00
requirements Replace boto3.in with slow scenario (#2451) 2024-03-14 09:58:01 +00:00
scenarios Add support for PyTorch-style local version semantics (#2430) 2024-03-16 10:24:50 -04:00
wheels Win Trampoline: Use Python executable path encoded in binary (#1803) 2024-02-22 16:10:02 +01:00
check_system_python.py Add system install test for alpine (#2371) 2024-03-12 14:26:05 +00:00