Consider the following packse scenario:
```toml
[root]
requires = [
"a>=1.0.0 ; python_version < '3.10'",
"a>=1.1.0 ; python_version >= '3.10'",
"a>=1.2.0 ; python_version >= '3.11'",
]
[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```
On current `main`, this produces a dependency on `a` that looks like
this:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```
But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a" },
]
```
That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.
The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.
While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.
For the packse scenario test: https://github.com/astral-sh/packse/pull/206
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.
This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.
In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.
So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.
This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.
This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.
Fixes#5086
Partially addresses #4732
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
## Summary
If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.
This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.
Closes: https://github.com/astral-sh/uv/issues/5440.
## Summary
After consultation with @carljm, we learned that modifying `PYTHONPATH`
is insufficient, because Python won't resolve `.pth` files (editables)
in the base environment. We also saw in
https://github.com/astral-sh/uv/issues/5459 that continuously appending
to `PYTHONPATH` can have some unintended effects.
This PR instead uses a `sitecustomize.py` in the ephemeral environment
to add the base environment's `site-packages`.
Closes https://github.com/astral-sh/uv/issues/5459.
It looks like we had a bad merge where the result caused some test
failures. This commit just updates the snapshots to the new reality. I
haven't found the root cause of the bad merge yet.
## Summary
It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.
This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.
Closes https://github.com/astral-sh/uv/issues/5424.
## Summary
This PR re-introduces caching for source trees. In short, we treat the
metadata as cached unless the `pyproject.toml`, `setup.py`, or
`setup.cfg` file changes. This is a heuristic and not a good one,
especially for extension modules, but without it, we have to rebuild
every project every time (unless you have static metadata, like a
`pyproject.toml` that we can read directly).
Now that we support persistent configuration, users should add:
```toml
[tool.uv]
reinstall = ["foo"]
```
If they want a package to always be refreshed (ignore cache) and
reinstalled (ignore environment).
Closes https://github.com/astral-sh/uv/issues/5420.
Two changes split out from the instability work:
* Break `ResolutionGraph::from_state` into methods before adding new
logic to it.
* `ResolutionGraph`: Convert `NodeKey` type to `PackageRef` struct:
Another small refactoring to make subsequent changes easier.
No functional changes.
@BurntSushi I hope this doesn't interfere with your work too much, the
`PackageRef` should at least make debugging panics here easier.
In preparation for the preferences changes with forking, change the
method structure in `CandidateSelector`. Split out into its own PR to
avoid merge conflicts with main. No functional changes.
Consider these requirements from pylint 3.2.5:
```
Requires-Dist: dill >=0.3.6 ; python_version >= "3.11"
Requires-Dist: dill >=0.3.7 ; python_version >= "3.12"
```
We will split on the python version, but then we may pick a version of
`dill` that's `>=0.3.7` in both branches and also have an otherwise
identical resolution in both forks. In this case, we merge both forks
and store only their conjoined markers.
## Summary
Normalize the order of marker expressions on construction. This removes
the distinction between expressions like `os_name == 'Linux'` vs.
`'Linux' == os_name` throughout the codebase. One caveat here is that
the `in` operator does not have a direct inverse, so we introduce
`MarkerOperator::Contains` to handle that case.
I wanted to land this smaller change before some more intrusive changes
as it simplifies the existing code quite a bit.
## Summary
Prior to this change, the resolver would panic if we ran with
`--offline` and `--no-deps` and we had cached metadata for a _package_
(i.e., the versions) but no cached metadata for the _distribution_
(i.e., the specific wheel), since we weren't validating that the
returned metadata in the `--no-deps` case was actually successful. (We
need metadata, even for `--no-deps`, so that we can validate extras.)
## Test Plan
The added test panics on the previous branch.
## Summary
Users can now run `uv cache prune --ci` (open to feedback on the name of
that flag) to remove all pre-built wheels from the cache, leaving behind
zipped, built wheels (which tend to be the most expensive assets to
re-create). This should greatly increase cache performance in CI
environments, since uploading unzipped wheels can actually hurt
performance if you're persisting the uv cache.
Closes https://github.com/astral-sh/uv/issues/5282.
The test asserts that 28 files were removed. But on my system, 27 files
are removed.
This PR is first about debugging what the difference is (since CI
presumably passes with the status quo snapshot). And then I'm thinking
the right way to fix the test failure is with a filter that replaces the
specific number of files removed (limited to what we know to be
correct) with a placeholder.
## Summary
This is surprisingly complex because we need to decide what happens if
you run `uv run` from within a hidden folder, etc. For now, I did the
simplest thing: we just ignore workspace members that are hidden
directories if they lack a `pyproject.toml`, so you can still include
hidden members, they're just ignored if they don't seem to be projects.
Closes https://github.com/astral-sh/uv/issues/5403.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Implements #5340
## Test Plan
<!-- How was it tested? -->
## Summary
This is a bit simpler than #5333, but seems to work in my testing on
macOS and Windows. It's based on implementations that I found in
[Pixi](36f1bb297d/src/cli/exec.rs (L99))
and
[Wasmer](49e60af8df/lib/wasix/src/state/builder.rs (L1058)).
Closes https://github.com/astral-sh/uv/issues/5257.
## Test Plan
On both macOS and Windows:
- `cargo run -- tool run --from jupyterlab jupyter-lab` -- hit Ctrl-C;
verify that the process exits and the terminal is left in a good state.
- `cargo run -- run python` -- hit Ctrl-C; verify that the process does
_not_ exit, but does on Ctrl-D.
