3924 commits

Author	SHA1	Message	Date
Charlie Marsh	8126a5ed32	Make MarkerTree Copy (#9542) ... ## Summary It's just a `usize`. It seems simpler and perhaps even more performant (?) to make it `Copy`.	2024-11-30 14:07:07 -05:00
Charlie Marsh	6bebf79ac3	Remove uses of Option<MarkerTreeContents> in PubGrubPackage (#9541) ... ## Summary Just use `MarkerTree::TRUE` instead of `None`.	2024-11-30 10:36:18 -05:00
Charlie Marsh	22fd9f7ff1	Pass extra when evaluating lockfile markers (#9539) ... ## Summary When we serialize and deserialize the lockfile, we remove the conflict markers. So in the linked case, the edges for the `tqdm` entries are like: ``` complexified_marker: UniversalMarker { pep508_marker: python_full_version >= '3.9.0', conflict_marker: true, }, ``` However... when we evaluate in-memory, the conflict markers are still there... ``` complexified_marker: UniversalMarker { pep508_marker: true, conflict_marker: extra == 't1' and extra != 't2', }, ``` So if `uv run` creates the lockfile, we evaluate this as `false`. We should make this consistent, and I expect @BurntSushi is aware. But for now, it's reasonable / correct to pass the extra when evaluating at this specific point, since we know the dependency was enabled by the marker. Closes https://github.com/astral-sh/uv/issues/9533#issuecomment-2508908591.	2024-11-30 14:22:18 +00:00
Charlie Marsh	53fe301b1d	Avoid using IDs across PubGrub states (#9538) ... ## Summary This isn't safe, because the prefetcher is global but the IDs could come from different PubGrub states (i.e., different forks).	2024-11-30 13:59:21 +00:00
Charlie Marsh	2aca623691	Remove unused specifier field from PubGrub structs (#9536)	2024-11-30 03:59:34 +00:00
Charlie Marsh	69811837e5	Support recursive extras with marker in pip compile -r pyproject.toml (#9535) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9530.	2024-11-30 03:40:22 +00:00
Charlie Marsh	950855877a	Use constraints in trace rather than irrelevant requires-python (#9529)	2024-11-29 19:25:43 +00:00
Charlie Marsh	58cf93a219	Un-gate --emit-marker-expression tests via --python-platform (#9528) ... These are tedious to update on non-Linux, but I think we can just use `--python-platform` to make them work on all platforms.	2024-11-29 19:23:12 +00:00
konsti	772251027e	Make pip install snapshot independent of settings fields (#9522) ... When changing something about the settings, `invalid_pyproject_toml_option_unknown_field` would fail unexpectedly because the exact list of possible options had changed. Since we're already testing this list in the settings-related test `resolve_config_file`, i'm stubbing the exact output here.	2024-11-29 12:32:47 -05:00
Charlie Marsh	57900f33bd	Override manylinux_compatible with --python-platform (#9526) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9521.	2024-11-29 17:26:24 +00:00
Jp	b9740d4e16	Align tempfile workspace dependencies with root project (#9524) ... <!-- Thank you for contributing to uv! To help us out with reviewing, please consider the following: - Does this pull request include a summary of the change? (See below.) - Does this pull request include a descriptive title? - Does this pull request include references to any relevant issues? --> ## Summary While working on potential bug fixes with temporary files on Windows (I think I am currently ecountering the same issue as #2810) I noticed that sub-workspaces were not all having the same `tempfile` version. And they were not relying on the cargo root project dependency. I don't know at all if it was done on purpose or not. (I also wanted to override the root dependency with a local source but it was not possible due to sub-workspaces not relying on the same). The root lockfile already pinned to the `3.14.0`. Some sub-workspaces were depending on the `3.12.0`, some others on the `3.14.0`. So I updated the root `Cargo.toml` to the `3.14.0`. Feel free to decline if it was done on purpose! No worries at all 🙂 Thanks! <!-- What's the purpose of the change? What does it do, and why? --> ## Test Plan All units tests are still passing on my side. Let's see with the pull-request CI 😄	2024-11-29 12:05:10 -05:00
Charlie Marsh	cf20673197	Upgrade to Rust 1.83 (#9511) ... ## Summary A lot of good new lints, and most importantly, error stabilizations. I tried to find a few usages of the new stabilizations, but I'm sure there are more. IIUC, this _does_ require bumping our MSRV.	2024-11-29 12:04:22 -05:00
konsti	b9b37a9bab	Build backend: Warn when visiting over 10k files (#9523) ... Also log how many files we visited in total as a debugging aid. No test for this one since we don't want to create 10k on disk in our test suite.	2024-11-29 16:46:11 +01:00
Charlie Marsh	590294a0ac	Propagate markers for recursive extras in resolver (#9509) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9499#issuecomment-2506491209.	2024-11-28 12:40:00 -05:00
konsti	b80bd87b99	Don't emit unpinned warning for proxy packages (#9497) ... Whenever we see an unpinned proxy package, we know that its base package is also unpinned and emitted a warning. Part 1 of 2 for #8155	2024-11-28 12:39:38 +00:00
Charlie Marsh	8a27d4d340	Allow file: URLs to include package names (#9493) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9489.	2024-11-27 22:27:07 -05:00
Charlie Marsh	201dfef780	Insert backslash when appending to system drive (#9488) ... ## Summary When you pass a system drive to `Path::join`, Rust doesn't insert a backslash between the drive and the path itself, so our lookups for system configuration were failing. Closes https://github.com/astral-sh/uv/issues/9416.	2024-11-27 19:38:29 -05:00
Charlie Marsh	f1ccbcb065	Consistently enforce requested-vs.-built metadata when retrieving wheels (#9484) ... ## Summary We were missing a bunch of edge cases, e.g., the wheel exists in the cache already. Closes https://github.com/astral-sh/uv/issues/9480.	2024-11-27 16:51:01 -05:00
Charlie Marsh	4f2b30ca02	Improve error messages for mismatches in tool.uv.sources (#9482) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9479.	2024-11-27 20:10:39 +00:00
konsti	0b0d0f44f8	Publish: Warn when keyring has no password (#8827) ... When trying to upload without a password but with the keyring, check that the keyring has a password for the upload URL and username and warn if it doesn't. Fixes #8781	2024-11-27 20:54:49 +01:00
Charlie Marsh	68e3228f2b	Remove lxml from lock test (#9481) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9467.	2024-11-27 18:39:38 +00:00
Zanie Blue	95cd8b8b3f	Bump version to 0.5.5 (#9478)	2024-11-27 11:37:39 -06:00
Andrew Gallant	cc6bfa14d1	uv/tests: move conflicts tests to its own file ... There are already a fair number and I'm planning to add more. And `lock.rs` is already quite big. There aren't any new tests or other changes here. This is just moving tests and trimming down the function names to avoid redundancy in the names.	2024-11-27 10:51:11 -05:00
Charlie Marsh	ee84620e90	Use consistent formatting for build system errors (#9340) ... ## Summary These look pretty different from the help / hint messages we show on resolver failure. I've added color, backticks, and a "hint:" prefix. Before:  After: 	2024-11-27 14:22:53 +00:00
Charlie Marsh	0cd9c542ad	Show an interpreter-focused message for --target and --prefix (#9373) ... ## Summary With `uv pip install --target` and `--prefix`, we (1) should allow managed Pythons, and (2) should show a different message that's focused on the interpreter we selected, rather than the environment.	2024-11-27 14:21:46 +00:00
Charlie Marsh	7169b2c427	Respect sources in overrides and constraints (#9455) ... ## Summary We still only respect overrides and constraints in the workspace root -- which we may want to change -- but overrides and constraints are now correctly lowered. Closes https://github.com/astral-sh/uv/issues/8148.	2024-11-27 13:56:14 +00:00
konsti	8c8a1f071c	Fix tool.uv.dependency-metadata.[].version schema (#9468) ... Fixes #9443	2024-11-27 13:26:03 +00:00
Zanie Blue	b503a25556	Avoid filtering Python executables names during install tests (#9458) ... These were erroneously being filtered, interfering with the snapshots in #8650	2024-11-27 01:34:39 +00:00
konsti	8074917449	Upload: All metadata incl. PEP 639 (#9442) ... We were previously not uploading all metadata in the formdata of an upload request in the legacy api. Notably, we were missing the PEP 639 license-files field. I had to switch to pdm due to https://github.com/pypa/hatch/issues/1828	2024-11-27 00:25:08 +01:00
Charlie Marsh	2534156eff	Use rich diagnostics for build failures (#9335) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9323.	2024-11-26 15:05:59 -05:00
Charlie Marsh	916d5d7778	Migrate to PubGrub's arena for package names (#9448) ... ## Summary There's more we can do here, i.e., to leverage the IDs more widely, but this is a start.	2024-11-26 15:05:39 -05:00
Charlie Marsh	8aeaf98f59	Rename from Lowered to Canonical (#9447) ... By request: https://github.com/astral-sh/uv/pull/9341#pullrequestreview-2460979421.	2024-11-26 13:34:43 -05:00
Zanie Blue	de84a897a1	Add dedicated error message for musl install attempts (#9430) ... Until https://github.com/astral-sh/uv/issues/6890 is fixed, it seems nice to explain that we do not support it rather than the generic message in https://github.com/astral-sh/uv/issues/9428	2024-11-26 12:30:07 -06:00
konsti	c94777fc54	Initialize rayon lazily (#9435) ... When performing a noop sync, we don't need the rayon threadpool, yet we pay for its initialization:  Be making the initialization lazy, we avoid that cost:  This code runs every time before user code in `uv run`. This means that before calling rayon, one now needs to call `LazyLock::force(&RAYON_INITIALIZE);`. Performance mode (CPU 0 is a perf core): ``` $ taskset -c 0 hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync" Benchmark 1: /home/konsti/projects/uv/uv-main sync Time (mean ± σ): 4.5 ms ± 0.1 ms [User: 2.7 ms, System: 1.8 ms] Range (min … max): 4.4 ms … 6.4 ms 640 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync Time (mean ± σ): 4.4 ms ± 0.1 ms [User: 2.7 ms, System: 1.6 ms] Range (min … max): 4.3 ms … 5.0 ms 679 runs Summary /home/konsti/projects/uv/target/profiling/uv sync ran 1.03 ± 0.04 times faster than /home/konsti/projects/uv/uv-main sync ``` Power saver mode: ``` $ hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync" Benchmark 1: /home/konsti/projects/uv/uv-main sync Time (mean ± σ): 28.1 ms ± 1.2 ms [User: 15.5 ms, System: 20.3 ms] Range (min … max): 25.7 ms … 31.9 ms 102 runs Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync Time (mean ± σ): 24.0 ms ± 1.2 ms [User: 13.8 ms, System: 9.9 ms] Range (min … max): 22.2 ms … 28.2 ms 122 runs Summary /home/konsti/projects/uv/target/profiling/uv sync ran 1.17 ± 0.08 times faster than /home/konsti/projects/uv/uv-main sync ```	2024-11-26 14:58:38 +00:00
Charlie Marsh	edcff575f0	Treat less compatible tags as lower priority in resolver (#9339) ... ## Summary This is a second pass at https://github.com/astral-sh/uv/pull/7556, which was reverted in https://github.com/astral-sh/uv/pull/7608 due to a regression in https://github.com/astral-sh/uv/issues/7606. The behavior is actually correct, but a package (`nmslib`) publishes inconsistent metadata, and the change here happened to cause us to select a wheel with "wrong" metadata. It's arbitrary, but it did cause a regression for folks. Since we're now seeing other issues caused by the wrongness here (and since the reporter in https://github.com/astral-sh/uv/issues/7606 has since removed the dependency), I'm inclined to ship this fix. Closes https://github.com/astral-sh/uv/issues/7553. Closes https://github.com/astral-sh/uv/issues/9283.	2024-11-26 14:51:32 +00:00
Charlie Marsh	d18753527f	Remove python_version from lowered marker representation (#9343) ... ## Summary We never construct these -- they should be impossible, since we always translate to `python_full_version`. This PR encodes that impossibility in the types.	2024-11-26 14:39:36 +00:00
Charlie Marsh	df844e1ec9	Treat deprecated aliases as equivalent in marker algebra (#9342) ... ## Summary This PR modifies our lowered representation such that any deprecated aliases are treated as "the same" marker in the algebra. So, for example, we now recognize that this is impossible, despite the marker names being different: ``` typing-extensions ; platform.python_implementation == 'CPython' and python_implementation != 'CPython' ``` Similarly, we now recognize that this is just `sys_platform == 'win32'`, despite the presence of both markers: ``` anyio ; sys_platform == 'win32' and sys.platform == 'win32' ```	2024-11-26 14:27:24 +00:00
Charlie Marsh	106863069d	Report marker diagnostics during parsing, rather than evaluation (#9338) ... ## Summary I want to move towards a more normalized marker representation within the marker tree, which means that the things we warn against will disappear by the time we get to evaluation. I think it makes more sense to show these warnings when we create the tree, rather than when we evaluate it.	2024-11-26 14:15:33 +00:00
konsti	f886d08094	Make pytest output column width independent (#9436) ... Fixes #9336	2024-11-26 10:59:04 +01:00
Charlie Marsh	c30a314e0e	Allow dependency groups to include the containing package (#9385) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9383.	2024-11-25 21:56:14 -05:00
Zanie Blue	21aa9bc53a	Allow syncing to empty virtual environment directories (#9427) ... As discussed in https://github.com/astral-sh/uv/issues/9423, it's confusing that we do not allow `uv sync` just because the `.venv` directory _exists_. This change matches `uv venv`.	2024-11-25 22:23:49 +00:00
Charlie Marsh	5759cb9891	Enable constraints in uv tool upgrade CLI (#9375) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9321.	2024-11-25 22:22:30 +00:00
Charlie Marsh	0158717ae6	Don't warn when --output-file is empty (#9417) ... ## Summary Closes https://github.com/astral-sh/uv/issues/9410.	2024-11-25 22:09:18 +00:00
Zanie Blue	f0c4865d3f	Add test case for empty virtual environment directory (#9426) ... Test case for #9427	2024-11-25 16:05:25 -06:00
Charlie Marsh	d47cf10042	Remove conflict between --no-sync and --frozen in uv run (#9400) ... ## Summary For reasons outlined in the linked issue, this is needlessly strict. Closes https://github.com/astral-sh/uv/issues/9397.	2024-11-25 01:18:27 +00:00
Li-Lun Lin	e485dfd7f1	feat: add support for --no-extra flag and setting (#9387) ... <!-- Thank you for contributing to uv! To help us review effectively, please ensure that: - The pull request includes a summary of the change. - The title is descriptive and concise. - Relevant issues are referenced where applicable. --> ## Summary Resolves #9333 This pull request introduces support for the `--no-extra` command-line flag and the corresponding `no-extra` UV setting. ### Behavior - When `--all-extras` is supplied, the specified extras in `--no-extra` will be excluded from the installation. - If `--all-extras` is not supplied, `--no-extra` has no effect and is safely ignored. ## Test Plan Since `ExtrasSpecification::from_args` and `ExtrasSpecification::extra_names` are the most important parts in the implementation, I added the following tests in the `uv-configuration/src/extras.rs` module: - **`test_no_extra_full`**: Verifies behavior when `no_extra` includes the entire list of extras. - **`test_no_extra_partial`**: Tests partial exclusion, ensuring only specified extras are excluded. - **`test_no_extra_empty`**: Confirms that no extras are excluded if `no_extra` is empty. - **`test_no_extra_excessive`**: Ensures the implementation ignores `no_extra` values that don't match any available extras. - **`test_no_extra_without_all_extras`**: Validates that `no_extra` has no effect when `--all-extras` is not supplied. - **`test_no_extra_without_package_extras`**: Confirms correct behavior when no extras are available in the package. - **`test_no_extra_duplicates`**: Verifies that duplicate entries in `pkg_extras` or `no_extra` do not cause errors. --------- Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>	2024-11-24 02:25:09 +00:00
Charlie Marsh	c63616c190	Remove dependency on buf index (#9391) ... ## Summary These are now failing.	2024-11-23 21:15:56 -05:00
Skyler Hawthorne	e5f5bd63cf	feat: export --prune (#9389) ... ## Summary This adds a `--prune` flag to the `export` command to correspond with the `--prune` flag of the `tree` command. The purpose is for generating a `requirements.txt` that omits a package and all of that package's unique dependencies. This is useful for cases where the project has a dependency on a common core package, but where that package does not need to be installed in the target environment. For example, a pyspark job needs spark for development, but when installing into a cluster that already has pyspark installed, it is desirable to omit pyspark's whole dependency tree so that only the unique dependencies that your job needs get installed, and do not risk breaking the pyspark dependencies with something incompatible. Dev groups cannot always cover this case because there are other projects where this common dependency occurs as a transitive. One example is Airflow providers, which include Airflow itself as a dependency, but it is unnecessary and undesirable to include Airflow's dependency tree in the `requirements.txt` for your DAGs. Partly related to #7214, though I'm not sure it covers the ask in that one of having this functionality extend to the project's actual published metadata. ## Test Plan An integration test was added, and some manual testing. Let me know if more would be better. --------- Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>	2024-11-24 02:11:54 +00:00
Andrew Gallant	ac5cee0128	uv/tests: update resolution-markers in conflict test ... This change is correct because disjointness checks now incorporate conflicts. In this case, there are actually four forks. Two of them correspond to `sys_platform == 'darwin'` and `sys_platform != 'darwin'`, but neither of those contain `jinja2==3.1.3`. Instead, they contain other versions of `jinja2` linked to other extras. If we ever add conflicts to our `resolution-markers` in the lock file, then those forks should show up here again. (Because, of course, some forks do contain `jinja2==3.1.3` here.)	2024-11-23 13:14:27 -05:00
Andrew Gallant	f7bed37a4e	uv-resolver: add "include" rules to ResolverEnvironment ... When we generate conflict markers for each resolution after the resolver runs, it turns out that generating them just from exclusion rules is not sufficient. For example, if `foo` and `bar` are declared as conflicting extras, then we end up with the following forks: A: extra != 'foo' B: extra != 'bar' C: extra != 'foo' and extra != 'bar' Now let's take an example where these forks don't share the same version for all packages. Consider a case where `idna==3.9` is in forks A and C, but where `idna==3.10` is in fork B. If we combine the markers in forks A and C through disjunction, we get the following: idna==3.9: extra != 'foo' or (extra != 'foo' and extra != 'bar') idna==3.10: extra != 'bar' Which simplifies to: idna==3.9: extra != 'foo' idna==3.10: extra != 'bar' But these are clearly not disjoint. Both dependencies could be selected, for example, when neither `foo` nor `bar` are active. We can remedy this by keeping around the inclusion rules for each fork: A: extra != 'foo' and extra == 'bar' B: extra != 'bar' and extra == 'foo' C: extra != 'foo' and extra != 'bar' And so for `idna`, we have: idna==3.9: (extra != 'foo' and extra == 'bar') or (extra != 'foo' and extra != 'bar') idna==3.10: extra != 'bar' and extra == 'foo' Which simplifies to: idna==3.9: extra != 'foo' idna==3.10: extra != 'bar' and extra == 'foo' And these *are* properly disjoint. There is no way for them both to be active. This also correctly accounts for fork C where neither `foo` nor `bar` are active, and yet, `idna==3.9` is still enabled but `idna==3.10` is not. (In the [motivating example], this comes from `baz` being enabled.) That is, this captures the idea that for `idna==3.10` to be installed, there must actually be a specific extra that is enabled. That's what makes it disjoint from `idna==3.9`. We aren't quite done yet, because this does add *too many* conflict markers to dependency edges that don't need it. In the next commit, we'll add in our world knowledge to simplify these conflict markers. [motivating example]: https://github.com/astral-sh/uv/issues/9289	2024-11-23 13:14:27 -05:00