Starting a discussion.
I find the navigation bar too long and verbose, it's overwhelming. I
think we either need to either:
- Limit the depth of the navigation bar (i.e. via `toc_depth: 2`)
- Move the table of contents out of the navigation bar over to the right
side
I prefer the second, though I think we ought to do some styling to
improve the aesthetics.
## Summary
First part of https://github.com/astral-sh/uv/issues/5093.
Remaining:
- Global settings
- `pip`-specific settings (some will be copied-over from here)
- Auto-generating the "Possible values" for enums
Add support for path dependencies from a package in one workspace to a
package in another workspace, which it self has workspace dependencies.
Say we have a main workspace with packages `a` and `b`, and a second
workspace with `c` and `d`. We have `a -> b`, `b -> c`, `c -> d`. This
would previously lead to a mangled path for `d`, which is now fixed.
Like distribution paths, we split workspace paths into an absolute
install path and a relative (or absolute, if the user provided an
absolute path) lock path.
Part of https://github.com/astral-sh/uv/issues/3943
Fixes a concurrency issue when multiple processes are installing the
same package in different virtual environments from Git ref (not a
specific Git commit).
## Symptoms
That's how some of symptoms looked like in our case:
```
DEBUG uv 0.2.21
DEBUG Checking for Python interpreter at path `/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7/bin/python`
DEBUG Using Python 3.12.4 environment at /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7/bin/python
DEBUG Acquired lock for `/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7`
DEBUG At least one requirement is not satisfied: torch
DEBUG Using request timeout of 300s
DEBUG Found 37 packages in `--find-links` entry: /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/pip/wheels
DEBUG Updating git source `Url { scheme: "https", cannot_be_a_base: false, username: "***", password: None, host: Some(Domain("github.com")), port: None, path: "/iterative/datachain", query: None, fragment: None }`
DEBUG Attempting GitHub fast path for: https://api.github.com/repos/iterative/datachain/commits/fix-distributed-test
DEBUG failed to check github HTTP status client error (404 Not Found) for url (https://api.github.com/repos/iterative/datachain/commits/fix-distributed-test)
DEBUG Performing a Git fetch for: https://***@github.com/iterative/datachain
error: Failed to download and build: `datachain @ git+https://***@github.com/iterative/datachain@fix-distributed-test`
Caused by: Git operation failed
Caused by: process didn't exit successfully: `git clone --local /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/db/9d45a3e6f56b0a69 /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/checkouts/9d45a3e6f56b0a69/56b15b8` (exit status: 128)
--- stderr
fatal: destination path '/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/checkouts/9d45a3e6f56b0a69/56b15b8' already exists and is not an empty directory.
```
## Cause of the issue
It is the same command that is failing - `git clone`, and I think it's
happening because it was trying to first get the repo to dereference the
`fix-distributed-test` branch:
`Given a remote source distribution, return a precise variant, if
possible.`
And it's happening w/i acquiring a lock around cache.
## Fix
I thinks we can reuse the existing `fetch` method that has already lock
around cache:
https://github.com/astral-sh/uv/pull/5051/files#diff-f58bb99dee2c4922d156ace3e7de651f0d9a81fc8e9447a2ad865de5c53543fcR61-R68
```python
// Avoid races between different processes, too.
let lock_dir = cache.join("locks");
....
```
## Questions
- Are there any tests that cover concurrency? I'm quite new to Rust and
if someone can point me to some examples and I can create a similar test
or a new one.
- Is error handling done correctly in this PR (again, I'm new to Rust -
I'll review and read about it, but it's better also for someone else to
review this)
## Summary
Currently, the `Locals` type relies on there being a single local
version for a given package. With marker expressions this may not be
true, a similar problem to https://github.com/astral-sh/uv/pull/4435.
This changes the `Locals` type to `ForkLocals`, which tracks locals for
a given fork. Local versions are now tracked on `PubGrubRequirement`
before forking.
Resolves https://github.com/astral-sh/uv/issues/4580.
Specifically, this shows the resolution produced by the
resolver *before* constructing a resolution graph.
Unlike most trace messages, this is a multi-line message
that needs to do some small amount of work to build
itself. So we do an explicit gating on the log level here
instead of just relying on the `trace!` macro itself.
I messed up the order of checks in #5033, due to which it failed to
exclude the case of `-P package`, as `arg.startswith("-P")` check came
first and skipped only the first argument.
That means that, in the following command:
```console
uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in -P attrs==18.1.0
```
The generated header would exclude `-P`, but keep `attrs==18.1.0`.
```plaintext
# This file was autogenerated by uv via the following command:
# uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in attrs==18.1.0
```
But we want to check for an exact match first and then only check for
the case when option and value are together.
This also affected `--find-links` short option of style `-f <uri>`.
Hopefully, third times going to be a charm. 😳
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
I tested locally, and also changed one snapshot test to use `-P` for
variation. I don't think it's worth an extra test, but can do that for
sure.
## Summary
When range requests aren't supported, we fall back to streaming the
wheel, stopping as soon as we hit a `METADATA` file. This is a small
optimization, but the downside is that we don't get to cache the
resulting wheel...
We don't know whether `METADATA` will be at the beginning or end of the
wheel, but it _seems_ like a better tradeoff to download and cache the
entire wheel?
Closes: https://github.com/astral-sh/uv/issues/5088.
Sort of a revert of: https://github.com/astral-sh/uv/pull/1792.
## Summary
This approach is based on https://github.com/PrefectHQ/docs. Rather than
publishing docs in the uv repo, we push to an independent repo that's
used solely to house the docs. In Prefect's case, this allows them to
publish versioned documentation (we should do that too). For us, though,
the benefit is that we can publish the Ruff and uv docs as a single site
(docs.astral.sh).
Prefect clones the source repo and builds the documentation from the
`docs` repo (i.e., the action runs in the `docs` repo). In our case,
I've instead set it up such that the action runs in `uv` (and later in
`ruff` too), clones the `docs` repo, and puts up a PR in that separate
repo. Because of these requirements, we have to use a PAT rather than a
deploy key (as PATs cannot do GitHub-specific things like create PRs --
they can only operate over the Git CLI).
See: https://github.com/astral-sh/docs/pull/2.
## Summary
Workaround the `stream_wheel` not retry issue
[found](https://github.com/astral-sh/uv/issues/3514#issuecomment-2229820667)
in #3514, it's not a perfect solution but I think it's acceptable
because the error should not occur frequently.
## Test Plan
Manually using `iptables -A OUTPUT -p tcp -dport 3128 -j REJECT
--reject-with tcp-reset` to inject connection reset error to the HTTP
proxy that proxies PyPI requests.
```
error: Failed to prepare distributions
Caused by: Failed to fetch wheel: piqp==0.4.1
Caused by: Request failed after 3 retries
Caused by: error sending request for url (09ade94dfd/piqp-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl)
Caused by: client error (Connect)
Caused by: tcp connect error: Connection refused (os error 111)
Caused by: Connection refused (os error 111)
```
## Summary
`echo "torch==1.10.0" | cargo run pip compile - -p 3.12 --no-deps` now
correctly fails. Previously, we were accepting the wheel
`torch-1.10.0-cp36-none-macosx_10_9_x86_64.whl` as compatible with
Python 3.10 due to the `none` ABI.
Closes https://github.com/astral-sh/uv/issues/5085.
## Summary
The example in the linked issue doesn't quite work, but I think it has
to do with the existing filtering logic. Will follow-up separately.
Closes https://github.com/astral-sh/uv/issues/5012.
## Summary
So this PR introduces change to how `Array` of dependencies
representation is reformatted while `PyProjectTomlMut` is manipulated.
These changes are here for it to respect the original indentation.
Closes https://github.com/astral-sh/uv/issues/5009
## Test Plan
Using `pyproject.toml` like
```
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests"
]
```
Executed
```
$ uv add httpx
```
And expected in `pyproject.toml`
```
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests",
"httpx",
]
```
Preserving original indentation
## Summary
We want to have consistency between the Ruff and uv documentation for
the upcoming release. We don't love the Ruff docs, but we'd rather have
consistency and then work towards improving them both, rather than have
two very-different documentation sites that both have weaknesses.
The setup here is simpler than in Ruff as: (1) we don't yet generate any
docs from Rust and (2) we don't try to reuse the README in the uv
documentation (which adds a lot of complexity in Ruff). So the change
here is mostly a 1-to-1 port to MkDocs.
## Test Plan
## Summary
This PR adds a new job to test that PyPy venvs are correctly created on
Windows and they contain all the expected binaries.
## Test Plan
Just let CI run.
This name should lead to less confusion. Unfortunately this is a
"breaking cache change" so everyone's cache will be invalidated. I'm not
sure if we should support a rename-on-upgrade.
edit: We can make the breaking change next time we bump the version
## Summary
Resolves#5017.
Note: This re-uses the same function defined in #5019 to find matching
packages.
## Test Plan
`cargo test`
```console
❯ ./target/debug/uvx --from fastapi fastapi
warning: `uvx` is experimental and may change without warning.
Resolved 33 packages in 427ms
warning: The fastapi executable is not part of the fastapi package. It is provided by the fastapi-cli package. Use `uvx --from fastapi-cli fastapi` instead.
Usage: fastapi [OPTIONS] COMMAND [ARGS]...
Try 'fastapi --help' for help.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Should fix#2092.
This PR changes `uv venv` so it also creates symlinks to `pypy` on Unix
and copies executables on Windows when creating a new environment using
PyPy.
I found a bit of discrepancy between creation of a venv using `python`
and `uv`, as using `python` brings all the executables with it. While
`uv` brings only those without any version number, at least on Windows.
The behaviour is different on Unix as we take the versioned symlinks
too.
Some examples below.
`python -m venv` generates the following `Scripts` folder.
```
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 7/14/2024 15:41 2031 activate
-a---- 7/14/2024 15:41 1029 activate.bat
-a---- 7/14/2024 15:41 9033 Activate.ps1
-a---- 7/14/2024 15:41 393 deactivate.bat
-a---- 7/14/2024 15:40 27648 libffi-8.dll
-a---- 7/14/2024 15:41 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 15:41 108424 pip.exe
-a---- 7/14/2024 15:41 108424 pip3.10.exe
-a---- 7/14/2024 15:41 108424 pip3.exe
-a---- 7/14/2024 15:41 79360 pypy.exe
-a---- 7/14/2024 15:41 79360 pypy3.10.exe
-a---- 7/14/2024 15:41 79360 pypy3.10w.exe
-a---- 7/14/2024 15:41 79360 pypy3.exe
-a---- 7/14/2024 15:41 79360 pypyw.exe
-a---- 7/14/2024 15:41 79360 python.exe
-a---- 7/14/2024 15:41 79360 python3.10.exe
-a---- 7/14/2024 15:41 79360 python3.exe
-a---- 7/14/2024 15:41 79360 pythonw.exe
```
`uv venv` instead generates this.
```
-a---- 7/14/2024 16:27 3360 activate
-a---- 7/14/2024 16:27 2251 activate.bat
-a---- 7/14/2024 16:27 2627 activate.csh
-a---- 7/14/2024 16:27 4191 activate.fish
-a---- 7/14/2024 16:27 3875 activate.nu
-a---- 7/14/2024 16:27 2766 activate.ps1
-a---- 7/14/2024 16:27 2378 activate_this.py
-a---- 7/14/2024 16:27 1728 deactivate.bat
-a---- 7/13/2024 19:19 27648 libffi-8.dll
-a---- 7/13/2024 19:19 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 16:27 1215 pydoc.bat
-a---- 7/13/2024 19:19 79360 pypy.exe
-a---- 7/13/2024 19:19 79360 pypyw.exe
-a---- 7/13/2024 19:19 79360 python.exe
-a---- 7/13/2024 19:19 79360 pythonw.exe
```
## Test Plan
To verify the correct behaviour:
1. Download and install PyPy from [official
website](https://www.pypy.org/download.html)
2. Call `uv venv -p <path_to_pypy_>`
3. Run `.\.venv\Scripts\activate` on Windows or
`./.venv/Scripts/activate` on Unix
4. Run `pypy`
I thought of writing some automated tests but I couldn't rely on `uv
python install` command to install PyPy as it's not in the list of
installable Python builds.
In some cases, it's possible for the marker expressions on conflicting
dependency specification to be disjoint but *incomplete*. That is, if
one unions the disjoint markers, the result is not the complete set of
marker environments possible. There may be some "gap" of marker
environments not covered by the markers.
This is a problem in practice because, before this commit, we only
created forks in the resolver for specific marker expressions. So if a
dependency happened to fall in a "gap," our resolver would never see it.
This commit fixes this by adding a new split covering the negation of
the union of all marker expressions in a set of forks for a specific
package.
Originally, I had planned on only creating this split when it was known
that the gap actually existed. That is, when the negation of the marker
expressions did *not* correspond to the empty set. After a lot of
thought, unfortunately, this (I believe) effectively boils down to 3SAT,
which is NP-complete.
Instead, what we do here is *always* create an extra split unless we can
definitively tell that it is empty. We look for a few cases, but
otherwise throw our hands up and potentially do wasted work.
This also updates the lock scenario tests to reflect the actual bug fix
here.
An important update here is the new `fork-incomplete-markers` test.
The snapshot saved here is actually wrong; it is the bug we want to
fix. In particular, it is missing a dependency on `c` in its lock file
because it falls into a gap created by incomplete-but-disjoint marker
expressions.
## Summary
As discussed in #3542 - there has been some confusion about how to get
`uv` to work with ADO Artifacts so I'm adding a quick guide.
## Test Plan
Smoke-tested the examples on my machine.
The only pubgrub error that can occur is a `NoSolutionError`, and the
only place it can occur is `unit_propagation`, all other variants if
`PubGrubError` are unreachable. By changing the return type on pubgrub's
side (https://github.com/astral-sh/pubgrub/pull/28), we can remove the
pattern matching and the `unreachable!()` asserts on `PubGrubError`.
Our pubgrub error wrapper used to have a two phased initialization,
first mostly stubs in `solve[_tracked]()` and then adding the actual
context in `resolve()`. When constructing the error in `solve` we
already have all this context, so we can unify this to a regular
constructor and remove the special casing in `resolve()` and `hints()`.
