898 commits

Author	SHA1	Message	Date
Charlie Marsh	56d39d21c2	Visit proxy packages eagerly (#10441) ... ## Summary The issue here is that we add `urllib3{python_full_version >= '3.8'}` as a dependency, then `requests{python_full_version >= '3.8'}`, which adds `urllib3`, but at that point, we haven't expanded `urllib3{python_full_version >= '3.8'}`, so we "lose" the singleton constraint. The solution is to ensure that we visit proxies eagerly, so that we accumulate constraints as early as possible. Closes https://github.com/astral-sh/uv/issues/10425#issuecomment-2580324578.	2025-01-09 20:37:28 +00:00
Charlie Marsh	f65fcf23b3	Remove duplicated comment (#10416)	2025-01-08 23:06:09 +00:00
Charlie Marsh	4c161d284b	Respect requires-python when installing tools (#10401) ... ## Summary This PR revives https://github.com/astral-sh/uv/pull/7827 to improve tool resolutions such that, if the resolution fails, and the selected interpreter doesn't match the required Python version from the solve, we attempt to re-solve with a newly-discovered interpreter that _does_ match the required Python version. For now, we attempt to choose a Python interpreter that's greater than the inferred `requires-python`, but compatible with the same Python minor. This helps avoid successive failures for cases like Posting, where choosing Python 3.13 fails because it has a dependency that lacks source distributions and doesn't publish any Python 3.13 wheels. We should further improve the strategy to solve _that_ case too, but this is at least the more conservative option... In short, if you do `uv tool instal posting`, and we find Python 3.8 on your machine, we'll detect that `requires-python: >=3.11`, then search for the latest Python 3.11 interpreter and re-resolve. Closes https://github.com/astral-sh/uv/issues/6381. Closes https://github.com/astral-sh/uv/issues/10282. ## Test Plan The following should succeed: ``` cargo run python uninstall --all cargo run python install 3.8 cargo run tool install posting ``` In the logs, we see: ``` ... DEBUG No compatible version found for: posting DEBUG Refining interpreter with: Python >=3.11, <3.12 DEBUG Searching for Python >=3.11, <3.12 in managed installations or search path DEBUG Searching for managed installations at `/Users/crmarsh/.local/share/uv/python` DEBUG Skipping incompatible managed installation `cpython-3.8.20-macos-aarch64-none` DEBUG Found `cpython-3.13.1-macos-aarch64-none` at `/opt/homebrew/bin/python3` (search path) DEBUG Skipping interpreter at `/opt/homebrew/opt/python@3.13/bin/python3.13` from search path: does not satisfy request `>=3.11, <3.12` DEBUG Found `cpython-3.11.7-macos-aarch64-none` at `/opt/homebrew/bin/python3.11` (search path) DEBUG Re-resolving with Python 3.11.7 DEBUG Using request timeout of 30s DEBUG Solving with installed Python version: 3.11.7 DEBUG Solving with target Python version: >=3.11.7 DEBUG Adding direct dependency: posting* DEBUG Searching for a compatible version of posting (*) ... ```	2025-01-08 12:38:17 -05:00
konsti	c5583b326f	Shrink Dist from 352 to 288 bytes (#10389) ... Found this when looking at #10385. Since we're constructing a lot of `Dist`s, we should keep it small.	2025-01-08 09:33:19 -05:00
konsti	68adadf806	Improve file pinning comments (#10387)	2025-01-08 11:42:25 +00:00
konsti	14a9008e54	Avoid overcounting versions in batch prefetcher (#10350)	2025-01-07 14:12:18 +00:00
konsti	3cbf8ab7b1	Optimize requirements_for_extra (#10348)	2025-01-07 14:12:12 +00:00
konsti	c6ac121ed0	Refactor batch prefetch (#10349)	2025-01-07 14:58:36 +01:00
konsti	3dc481b063	Speed up file pins (#10346) ... Ref https://github.com/astral-sh/uv/issues/10344 Avoid the nested hashmap, file pinning is called in the version selection hot loop. ``` $ hyperfine --warmup 1 --prepare "uv venv -p 3.12" "./uv-2 pip compile scripts/requirements/airflow.in" "./uv-1 pip compile scripts/requirements/airflow.in" Finished `profiling` profile [optimized + debuginfo] target(s) in 0.12s Benchmark 1: ./uv-2 pip compile scripts/requirements/airflow.in Time (mean ± σ): 420.1 ms ± 4.7 ms [User: 585.4 ms, System: 195.1 ms] Range (min … max): 413.1 ms … 429.2 ms 10 runs Benchmark 2: ./uv-1 pip compile scripts/requirements/airflow.in Time (mean ± σ): 473.0 ms ± 4.9 ms [User: 654.4 ms, System: 209.4 ms] Range (min … max): 468.0 ms … 481.1 ms 10 runs Summary ./uv-2 pip compile scripts/requirements/airflow.in ran 1.13 ± 0.02 times faster than ./uv-1 pip compile scripts/requirements/airflow.in ```	2025-01-07 08:51:08 -05:00
konsti	d8fcf2a58f	Simplify requirements_for_extra (#10347) ... Ref https://github.com/astral-sh/uv/issues/10344 Not a performance optimization, but the function had become too large. No logic changes, just code moving around. Looks slightly better when ignoring whitespace changes. It's still too complex but i haven't found an apt simplification.	2025-01-07 08:50:13 -05:00
konsti	180a138161	Deactivate tracing for choose version (#10351)	2025-01-07 09:55:06 +00:00
Charlie Marsh	0d57d298e6	Avoid downgrading packages when --upgrade is provided (#10097) ... ## Summary When `--upgrade` is provided, we should retain already-installed packages _if_ they're newer than whatever is available from the registry. Closes https://github.com/astral-sh/uv/issues/10089.	2025-01-06 17:41:43 -05:00
Charlie Marsh	66a603b6c4	Use dynamic dispatch to simplify reporters (#10086) ... ## Summary Sort of undecided on this. These are already stored as `dyn Reporter` in each struct, so we're already using dynamic dispatch in that sense. But all the methods take `impl Reporter`. This is sometimes nice (the callsites are simpler?), but it also means that in practice, you often _can't_ pass `None` to these methods that accept `Option<impl Reporter>`, because Rust can't infer the generic type. Anyway, this adds more consistency and simplifies the setup by using `Arc<dyn Reporter>` everywhere.	2025-01-06 12:04:00 -05:00
Charlie Marsh	833519d5d8	Extract supported architectures from wheel tags (#10179) ... ## Summary This PR extends #10046 to also handle architectures, which allows us to correctly include `2.5.1` on the `cu124` index for ARM Linux. Closes https://github.com/astral-sh/uv/issues/9655.	2025-01-04 02:42:17 +00:00
Charlie Marsh	2eba52f674	Avoid forking on version in non-universal resolutions (#10274) ... ## Summary Closes https://github.com/astral-sh/uv/issues/10275.	2025-01-02 11:40:48 -05:00
Charlie Marsh	ed4e4ff4b5	Use BTreeMap::range to avoid iterating over unnecessary versions (#10266) ... ## Summary Closes https://github.com/astral-sh/uv/issues/6174.	2025-01-02 01:21:36 +00:00
Charlie Marsh	7bbec6b2e4	Parse URLs lazily in resolver (#10259) ... ## Summary The idea here is to avoid parsing all registry URLs upfront, and instead parse them when we need them. Closes #6133.	2025-01-01 12:35:30 -05:00
Charlie Marsh	cf88828e55	Avoid stripping query parameters from URLs (#10253) ... ## Summary Closes https://github.com/astral-sh/uv/issues/10251.	2024-12-31 12:59:26 -05:00
Charlie Marsh	dcd96a83aa	Respect static metadata for already-installed distributions (#10242) ... ## Summary Closes https://github.com/astral-sh/uv/issues/10239#issuecomment-2565663046	2024-12-30 12:47:06 -05:00
Charlie Marsh	b316d780bf	Consider workspace dependencies to be 'direct' (#10197) ... ## Summary Closes https://github.com/astral-sh/uv/issues/7863.	2024-12-28 20:32:45 -05:00
Charlie Marsh	d3f06de4f5	Fix incorrect mismatched constraints reference (#10184)	2024-12-26 23:25:38 +00:00
Charlie Marsh	79dce7391e	Avoid need for universal markers in requirements.txt export (#10171)	2024-12-26 15:32:32 +00:00
Charlie Marsh	bec8468183	Remove unnecessary prefixes (#10158)	2024-12-25 14:18:01 -05:00
Charlie Marsh	3cb723220e	Remove anyhow::Result for lock serialization (#10151)	2024-12-25 01:24:26 +00:00
Charlie Marsh	facd21a679	Replace Option<BTreeSet> with BTreeSet for members (#10150)	2024-12-25 00:37:46 +00:00
Charlie Marsh	f872b56124	Add a lock target abstraction (#10129) ... ## Summary This PR introduces a `LockTarget`, which is peer to `InstallTarget` and enables us to capture the common functionality necessary to support locking. For now, to minimize changes, only the `Workspace` target is implemented. In a future PR, I'll add a `Script` target for both locking and installing.	2024-12-24 19:26:31 -05:00
Charlie Marsh	7c47a457d9	Move installable targets out of uv-resolver crate (#10126) ... ## Summary The proximate motivation is that I want to add new variant for scripts, but `uv-resolver` can't depend on `uv-scripts` without creating a circular dependency. However, I think this _does_ just make more sense -- the resolver crate shouldn't be coupled to the various kinds of workspaces, and these details are mostly encoded in `projects/lock.rs` and similar files.	2024-12-24 19:01:50 -05:00
Charlie Marsh	6745a8b00a	Show non-project dependencies in uv tree (#10149) ... ## Summary Closes #10147.	2024-12-24 23:34:58 +00:00
Charlie Marsh	e09b1080f4	Store dependency groups separate from dependencies in lockfile (#10148) ... ## Summary This is necessary for some future improvements to non-`[project]` workspaces and PEP 723 scripts. It's not "breaking", but it will invalidate lockfiles for non-`[project]` workspaces. I think that's okay, since we consider those legacy right now, and they're really rare.	2024-12-24 17:32:29 -05:00
Charlie Marsh	3435777e87	Support uv export for non-project workspaces (#10144)	2024-12-24 15:23:28 +00:00
Charlie Marsh	33cb3497aa	Strip fragment when storing direct URL (#10093) ... ## Summary Closes https://github.com/astral-sh/uv/issues/10088#issuecomment-2558280467.	2024-12-22 09:07:04 -05:00
Charlie Marsh	19a6b5fe4b	Include hashes for local source archives (#10080) ... ## Summary Closes https://github.com/astral-sh/uv/issues/10077	2024-12-21 09:31:28 -05:00
Charlie Marsh	9128fe1866	Use portable path instead of string for subdirectory (#10069) ... ## Summary A few places where there are extra conversions to and from string that seem unnecessary; a few places where we're using `PathBuf` instead of `PortablePathBuf`.	2024-12-21 00:28:36 +00:00
Charlie Marsh	ddc290feb4	Add support for subdirectories in direct URLs in uv.lock (#10068) ... ## Summary There were some subtle bugs here and no test coverage.	2024-12-20 19:00:50 -05:00
Charlie Marsh	2c68dfd4a9	Backtrack to non-local versions when wheels are missing platform support (#10046) ... ## Summary This is yet another variation on https://github.com/astral-sh/uv/pull/9928, with a few minor changes: 1. It only applies to local versions (e.g., `2.5.1+cpu`). 2. It only _considers_ the non-local version as an alternative (e.g., `2.5.1`). 3. It only _considers_ the non-local alternative if it _does_ support the unsupported platform. 4. Instead of failing, it falls back to using the local version. So, this is far less strict, and is effectively designed to solve PyTorch but nothing else. It's also not user-configurable, except by way of using `environments` to exclude platforms.	2024-12-20 19:11:27 +00:00
konsti	ae659c8bfe	Stable order for virtual packages (#10024) ... uv gives priorities to packages by package name, not by virtual package (`PubGrubPackage`). pubgrub otoh when prioritizing order the virtual packages. When the order of virtual packages changes, uv changes its resolutions and error messages. This means uv was depending on implementation details of pubgrub's prioritization caching. This broke with https://github.com/pubgrub-rs/pubgrub/pull/299, which added a tiebreaker term that made pubgrub's sorting deterministic given a deterministic ordering of allocating the packages (which happens the first time pubgrub sees a package). The new custom tiebreaker decreases the difference to upstream pubgrub.	2024-12-20 09:28:46 +00:00
konsti	ac348eecdf	Batch prefetch per fork (#10029) ... Previously, the batch prefetcher was part of the solver loop, used across forks. This would lead to each preference in a fork being counted as a tried version, so that after 5 forks with the identical version, we would start batch prefetching. The reported numbers of tried versions are also reported. By tracking the batch prefetcher on the fork the numbers are corrected. An alternative would be tracking the actually tried versions, but that would mean more overhead in the top level solver loop when the current heuristic works. In `ecosystem/transformers`: ``` $ hyperfine --runs 10 --prepare "rm -f uv.lock" "../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z" "uv lock --exclude-newer 2024-08-08T00:00:00Z" Benchmark 1: ../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z Time (mean ± σ): 386.2 ms ± 6.1 ms [User: 396.0 ms, System: 144.5 ms] Range (min … max): 378.5 ms … 397.9 ms 10 runs Benchmark 2: uv lock --exclude-newer 2024-08-08T00:00:00Z Time (mean ± σ): 422.0 ms ± 5.5 ms [User: 459.6 ms, System: 190.3 ms] Range (min … max): 415.0 ms … 430.5 ms 10 runs Summary ../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z ran 1.09 ± 0.02 times faster than uv lock --exclude-newer 2024-08-08T00:00:00Z ```	2024-12-19 15:47:01 +01:00
Charlie Marsh	4d3c1b325e	Prefer higher Python lower-bounds when forking (#10007) ... ## Summary With the advent of `--fork-strategy requires-python` (the default), we actually _want_ to solve higher lower-bound forks before lower lower-bound forks. The former ensures we get the most compatible versions, while the latter ensures we get fewer overall versions. These two strategies match up with `--fork-strategy`, but need to be respected as such. Closes https://github.com/astral-sh/uv/issues/9998.	2024-12-18 16:54:56 -05:00
Charlie Marsh	dd760ee507	Normalize platform_system to sys_platform (#9949) ... ## Summary A revival of an old idea (#9344) that I have slightly more confidence in now. I abandoned this idea because (1) it couldn't capture that, e.g., `platform_system == 'Windows' and sys_platform == 'foo'` (or some other unknown value) are disjoint, and (2) I thought that Android returned `"android"` for one of `sys_platform` or `platform_system`, which would've made this logic incorrect. However, it looks like Android... doesn't do that? And the values here are almost always in a small, known set. So in the end, the tradeoffs here actually seem pretty good. Vis-a-vis our current solution, this can (e.g.) _simplify out_ expressions like `sys_platform == 'win32' or platform_system == 'Windows'`.	2024-12-18 10:29:34 -05:00
Charlie Marsh	294da52610	Add nuance to prefetch logging (#9984)	2024-12-17 22:52:45 +00:00
Zanie Blue	6dfe1774e8	Use the build options value to improve hints for no wheel / source distribution errors (#9950) ... Extends the hints from https://github.com/astral-sh/uv/pull/9948 with `BuildOptions` context so we can explain a bit more.	2024-12-17 14:08:22 -06:00
Zanie Blue	515d72c6f9	Improve display of ranges when pre-releases are not allowed (#9944) ... Closes https://github.com/astral-sh/uv/issues/9891 There are two changes here 1. We now exclude pre-releases (if they are not allowed) from the available versions set when simplifying ranges, this means the simplified range reflects the _allowed_ available versions — which is what we want. We no longer segment ranges into arbitrary looking segments.. 2. We improve on #9885, expanding the scope to avoid regressions where we would now otherwise enumerate a bunch of versions --------- Co-authored-by: konsti <konstin@mailbox.org>	2024-12-17 17:05:15 +00:00
konsti	ebc6d20d9d	Better build error messages (#9660) ... Build failures are one of the most common user facing failures that aren't "obivous" errors (such as typos) or resolver errors. Currently, they show more technical details than being focussed on this being an error in a subprocess that is either on the side of the package or - more likely - in the build environment, e.g. the user needs to install a dev package or their python version is incompatible. The new error message clearly delineates the part that's important (this is a build backend problem) from the internals (we called this hook) and is consistent about which part of the dist building stage failed. We have to calibrate the exact wording of the error message some more. Most of the implementation is working around the orphan rule, (this)error rules and trait rules, so it came out more of a refactoring than intended. Example: 	2024-12-17 09:44:32 -06:00
Zanie Blue	052c1a6fd1	Collapse redundant Python version incompatibilities in resolver error message (#9957) ... Part of https://github.com/astral-sh/uv/issues/9886 Technically could affect other redundant clauses, but that does not appear to be the case in practice.	2024-12-17 14:27:07 +00:00
Zanie Blue	a78e7468a7	Omit trailing zeros on Python requirements inferred from versions (#9952) ... In a message like ``` ❯ echo "numpy>2" | uv pip compile -p 3.8 - × No solution found when resolving dependencies: ╰─▶ Because the requested Python version (>=3.8.0) does not satisfy Python>=3.10 and the requested Python version (>=3.8.0) does not satisfy Python>=3.9,<3.10, we can conclude that Python>=3.9 is incompatible. And because numpy>=2.0.1,<=2.0.2 depends on Python>=3.9 and only the following versions of numpy are available: numpy<=2.0.2 ``` I'm surprised that `-p 3.8` leads to expressions like `>=3.8.0` (I understand it, of course, but it's not intuitive) and then all the _other_ Python versions in the message omit the trailing zero. This updates the `PythonRequirement` parsing to drop the trailing zeros. It's easier to do there because the version is not yet abstracted.	2024-12-17 08:18:27 -06:00
Zanie Blue	686f383fa4	Improve phrasing for single term incompatibilities (#9953) ... It makes more sense to say "cannot be used" rather than "is incompatible" when the term is a single package	2024-12-17 08:14:33 -06:00
Charlie Marsh	9e4b842382	Add some misc. touch-ups in resolver (#9954)	2024-12-17 03:59:57 +00:00
Charlie Marsh	85e17ddfa7	Remove TODO around dev dependency edges (#9956) ... ## Summary I think I fixed this?	2024-12-17 03:59:07 +00:00
Zanie Blue	5c3dafc1a5	Simplify ranges in the derivation tree before reporting (#9897) ... An internal refactor to apply simplifications at the tree-level instead of in the report formatter.	2024-12-16 19:42:05 -06:00
Zanie Blue	d257bea720	Add resolver error hint for no-binary and no-build failures (#9948) ... Moves some of the context out of the error chain to improve readability.	2024-12-16 18:47:40 -06:00