As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html
Before that, there were 91 separate integration tests binary.
(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
Fixes: #8058
## Test Plan
Integration test (but only for Unix, because symlinks on Windows require
admin privs. Plus, they are not really all that idiomatic on Windows)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
closes#4828
First iteration for an implementation. I need to add more tests but
wanted your opinion on the implementation first.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Currently tested using the following command but will add tests shortly:
```console
D:\repo\uv> cargo run venv -p 3.13t && .venv\Scripts\python.exe
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
Running `target\debug\uv.exe venv -p 3.13t`
Using Python 3.13.0rc1 interpreter at: C:\Users\bschoen\AppData\Local\Programs\Python\Python313\python3.13t.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate
Python 3.13.0rc1 experimental free-threading build (tags/v3.13.0rc1:e4a3e78, Jul 31 2024, 21:06:58) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
`_virtualenv.py` doesn't need to import `__future__.annotations`, as it
has none.
Removing the import:
* Restores the action of the VIRTUALENV_PATCH on Python 3.6
* Eliminates 24 lines of error messages displayed by Python 3.6 when it
starts in an environment created by uv:
```plaintext
Error processing line 1 of /tmp/tmp.ENwqZ0oeyb/lib/python3.6/site-packages/_virtualenv.pth:
Traceback (most recent call last):
File "~/.pyenv/versions/3.6.15/lib/python3.6/site.py", line 168, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "/tmp/tmp.ENwqZ0oeyb/lib/python3.6/site-packages/_virtualenv.py", line 3
from __future__ import annotations
^
SyntaxError: future feature annotations is not defined
Remainder of file ignored
```
(Python displays the errors above twice.)
I appreciate the Python team no longer support Python 3.6, but
RedHat-style Linux distributions will support Python 3.6 in their
`/usr/libexec/platform-python` until [releasever 8 expires in
2029](https://access.redhat.com/support/policy/updates/errata#RHEL8_Planning_Guide).
I'm happy for the community to move on, in general, but don't see the
harm in helping those who can't.
I'm not yet sure what in the “remainder of file ignored” is necessary
for my project's build, as I haven't yet finished digging that from
under Hatch. I'll follow up on #6426 when I do, so we can concentrate on
getting to the happy cow.
## Test Plan
```sh
( set -eu
export VIRTUAL_ENV="$(mktemp -d)"
./target/release/uv venv "$VIRTUAL_ENV" --python=python3.6
./target/release/uv pip install cowsay
$VIRTUAL_ENV/bin/python -m cowsay --text 'Look, a talking cow!' )
```
Happy output:
```plaintext
Using Python 3.6.15 interpreter at: ~/.local/bin/python3.6
Creating virtualenv at: /tmp/tmp.VHl4XNi3oI
Activate with: source /tmp//tmp.VHl4XNi3oI/bin/activate
Resolved 1 package in 929ms
Installed 1 package in 17ms
+ cowsay==6.0
____________________
| Look, a talking cow! |
====================
\
\
^__^
(oo)\_______
(__)\ )\/\
||----w |
|| ||
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
A few of these should use `absolute` instead of `canonicalize`; and
apparently we no longer need to strip the `CANONICAL_CWD` to get tests
passing.
It transpires that detecting the directory a script was sourced from is
non-trivial across `bash`, `ksh` and `zsh`.
The previous version was a one-liner and supported `bash` and `zsh` but
not `ksh`.
It is possible to keep the one-liner and add `ksh` support, but that is
mutually-exclusive with `zsh`.
Therefore, the only way to square this circle is to add an `if` block. A
silver lining here is that although longer, the script is probably
easier to follow as there is less code-golfing going on.
## Summary
Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:
* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
`.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.
The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).
Fixes: #3863
## Test Plan
* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
## Summary
If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.
This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.
Closes: https://github.com/astral-sh/uv/issues/5440.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Currently, `uv` refuses to install anything on GraalPy. This is
currently blocking GraalPy testing with cibuildwheel, since manylinux
includes both `uv` and `graalpy` (but doesn't test with `uv`), whereas
cibuildwheel defaults to `uv`. See e.g.
2750618295
where it gives
```
+ python -m build /project/sample_proj --wheel --outdir=/tmp/cibuildwheel/built_wheel --installer=uv
* Creating isolated environment: venv+uv...
* Using external uv from /usr/local/bin/uv
* Installing packages in isolated environment:
- setuptools >= 40.8.0
> /usr/local/bin/uv pip install "setuptools >= 40.8.0"
< error: Unknown implementation: `graalpy`
```
## Test Plan
I simply based the GraalPy support on PyPy and added some small tests.
I'm open to discussing how to test this. GraalPy is available for
manylinux images and with setup-python, so we should be able to add
tests against it to the CI. I locally confirmed by installing `uv` into
a GraalPy venv and then trying things like `uv pip install Pillow` and
testing those extensions.
## Summary
"Bare" made sense when we had a variant that seeded the environment, but
now that the crate _only_ creates a bare environment, lets drop that
terminology.
## Summary
Should fix#2092.
This PR changes `uv venv` so it also creates symlinks to `pypy` on Unix
and copies executables on Windows when creating a new environment using
PyPy.
I found a bit of discrepancy between creation of a venv using `python`
and `uv`, as using `python` brings all the executables with it. While
`uv` brings only those without any version number, at least on Windows.
The behaviour is different on Unix as we take the versioned symlinks
too.
Some examples below.
`python -m venv` generates the following `Scripts` folder.
```
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 7/14/2024 15:41 2031 activate
-a---- 7/14/2024 15:41 1029 activate.bat
-a---- 7/14/2024 15:41 9033 Activate.ps1
-a---- 7/14/2024 15:41 393 deactivate.bat
-a---- 7/14/2024 15:40 27648 libffi-8.dll
-a---- 7/14/2024 15:41 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 15:41 108424 pip.exe
-a---- 7/14/2024 15:41 108424 pip3.10.exe
-a---- 7/14/2024 15:41 108424 pip3.exe
-a---- 7/14/2024 15:41 79360 pypy.exe
-a---- 7/14/2024 15:41 79360 pypy3.10.exe
-a---- 7/14/2024 15:41 79360 pypy3.10w.exe
-a---- 7/14/2024 15:41 79360 pypy3.exe
-a---- 7/14/2024 15:41 79360 pypyw.exe
-a---- 7/14/2024 15:41 79360 python.exe
-a---- 7/14/2024 15:41 79360 python3.10.exe
-a---- 7/14/2024 15:41 79360 python3.exe
-a---- 7/14/2024 15:41 79360 pythonw.exe
```
`uv venv` instead generates this.
```
-a---- 7/14/2024 16:27 3360 activate
-a---- 7/14/2024 16:27 2251 activate.bat
-a---- 7/14/2024 16:27 2627 activate.csh
-a---- 7/14/2024 16:27 4191 activate.fish
-a---- 7/14/2024 16:27 3875 activate.nu
-a---- 7/14/2024 16:27 2766 activate.ps1
-a---- 7/14/2024 16:27 2378 activate_this.py
-a---- 7/14/2024 16:27 1728 deactivate.bat
-a---- 7/13/2024 19:19 27648 libffi-8.dll
-a---- 7/13/2024 19:19 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 16:27 1215 pydoc.bat
-a---- 7/13/2024 19:19 79360 pypy.exe
-a---- 7/13/2024 19:19 79360 pypyw.exe
-a---- 7/13/2024 19:19 79360 python.exe
-a---- 7/13/2024 19:19 79360 pythonw.exe
```
## Test Plan
To verify the correct behaviour:
1. Download and install PyPy from [official
website](https://www.pypy.org/download.html)
2. Call `uv venv -p <path_to_pypy_>`
3. Run `.\.venv\Scripts\activate` on Windows or
`./.venv/Scripts/activate` on Unix
4. Run `pypy`
I thought of writing some automated tests but I couldn't rely on `uv
python install` command to install PyPy as it's not in the list of
installable Python builds.
Whew this is a lot.
The user-facing changes are:
- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"
The internal changes are:
- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
Extends #4120
Part of #2607
There should be no behavior changes here. Restructures the discovery API
to be focused on a toolchain first perspective in preparation for
exposing a `find_or_fetch` method for toolchains in
https://github.com/astral-sh/uv/pull/4138.
Updates our Python interpreter discovery to conform to the rules
described in #2386, please see that issue for a full description of the
behavior. Briefly, we now will search for interpreters that satisfy a
requested version without stopping at the first Python executable.
Additionally, if retrieving information about an interpreter fails we
will continue to search for a working interpreter. We also add the
plumbing necessary to request Python implementations other than CPython,
though we do not add support for other implementations at this time.
A major internal goal of this work is to prepare for user-facing managed
toolchains i.e. fetching a requested version during `uv run`. These APIs
are not introduced, but there is some managed toolchain handling as
required for our test suite.
Some noteworthy implementation changes:
- The `uv_interpreter::find_python` module has been removed in favor of
a `uv_interpreter::discovery` module.
- There are new types to help structure interpreter requests and track
sources
- Executable discovery is implemented as a big lazy iterator and is a
central authority for source precedence
- `uv_interpreter::Error` variants were split into scoped types in each
module
- There's much more unit test coverage, but not for Windows yet
Remaining work:
- [x] Write new test cases
- [x] Determine correct behavior around executables in the current
directory
- _Future_: Combine `PythonVersion` and `VersionRequest`
- _Future_: Consider splitting `ManagedToolchain` into local and remote
variants
- _Future_: Add Windows unit test coverage
- _Future_: Explore behavior around implementation precedence (i.e.
CPython over PyPy)
Refactors split into:
- #3329
- #3330
- #3331
- #3332Closes#2386
## Summary
Closes
https://github.com/astral-sh/uv/issues/3578#issuecomment-2110675382.
## Test Plan
Verified that in the OpenSUSE test, we create both, and they're
symlinks:
```text
INFO: Creating virtual environment with `venv`...
INFO: Installing into `venv` virtual environment...
DEBUG Found a virtualenv named .venv at: /tmp/tmp4nape29h/.venv
DEBUG Cached interpreter info for Python 3.10.14, skipping probing: .venv/bin/python
DEBUG Using Python 3.10.14 environment at .venv/bin/python
DEBUG Trying to lock if free: .venv/.lock
purelib: "/tmp/tmp4nape29h/.venv/lib/python3.10/site-packages"
platlib: "/tmp/tmp4nape29h/.venv/lib64/python3.10/site-packages"
is_same_file(purelib, platlib): Ok(true)
```
## Summary
runpy.run_path was added in python 2.7 and 3.2 - and every python that
is not EOL supports it.
It is arguably nicer to read and the path is only given once in the
command.
At least right now, runpy - unlike exec with S102 - is not flagged by
any bandit-derived ruff check.
(I guess because it loads from a file instead of a simple string...)
Because of the import, it is also not a one-liner anymore. (But that
could be fixed with an __import__('runpy').run_path...)
## Test Plan
import runpy
runpy.run_path('/path/to/venv/bin/activate_this.py')
We now use the getters and setters everywhere.
There were some places where we wanted to build a `MarkerEnvironment`
out of whole cloth, usually in tests. To facilitate those use cases, we
add a `MarkerEnvironmentBuilder` that provides a convenient constructor.
It's basically like a `MarkerEnvironment::new`, but with named
parameters. That's useful here because there are so many fields (and
they many have the same type).
## Summary
Refreshes some of the activation scripts, and fixes some bugs in
`activate_this.py` that were likely the rest of some erroneous
copy-pasting.
Closes https://github.com/astral-sh/uv/issues/3346.
## Test Plan
```
❯ python
Python 3.12.0 (main, Feb 28 2024, 09:44:16) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import httpx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'httpx'
>>> activator = '.venv/bin/activate_this.py'
>>> with open(activator) as f:
... exec(f.read(), {'__file__': activator})
...
>>> import httpx
```
When running
```
set UV_CACHE_DIR=%LOCALAPPDATA%\uv\cache-foo && uv venv venv
```
in windows CMD, the error would be just
```
error: The system cannot find the path specified. (os error 3)
```
The problem is that the first action in the cache dir is adding the tag,
and the `cachedir` crate is using `std::fs` instead of `fs_err`. I've
copied the two functions we use from the crate and changed the import
from `std::fs` to `fs_err`.
The new error is
```
error: failed to open file `C:\Users\Konstantin\AppData\Local\uv\cache-foo \CACHEDIR.TAG`
Caused by: The system cannot find the path specified. (os error 3)
```
which correctly explains the problem.
Closes#3280
## Summary
This is mainly a cleanup PR to leverage uv-version in uv-virtualenv
instead of passing it via `uv`.
In #1852 I introduced the ability to pass extra cfg to `gourgeist` for
the primary purpose of passing the uv version, but since the dawn of the
uv-version crate dynamically passing more values to pyvenv.cfg is no
longer needed.
## Test Plan
Existing `uv` tests should still verify `uv = <version>` exists in the
venv and make sure no regressions were introduced.
I can't get this to reproduce on GitHub Actions -- maybe the builds
there differ, or maybe the builds changed since we added this fix? I'll
check locally, but regardless, this is a typo.
Closes#3158.
Scott schafer got me the idea: We can avoid repeating the path for
workspaces dependencies everywhere if we declare them in the virtual
package once and treat them as workspace dependencies from there on.
## Summary
This PR changes our user-facing representation for paths to use relative
paths, when the path is within the current working directory. This
mirrors what we do in Ruff. (If the path is _outside_ the current
working directory, we print an absolute path.)
Before:
```shell
❯ uv venv .venv2
Using Python 3.12.2 interpreter at: /Users/crmarsh/workspace/uv/.venv/bin/python3
Creating virtualenv at: .venv2
Activate with: source .venv2/bin/activate
```
After:
```shell
❯ cargo run venv .venv2
Finished dev [unoptimized + debuginfo] target(s) in 0.15s
Running `target/debug/uv venv .venv2`
Using Python 3.12.2 interpreter at: .venv/bin/python3
Creating virtualenv at: .venv2
Activate with: source .venv2/bin/activate
```
Note that we still want to use the existing `.simplified_display()`
anywhere that the path is being simplified, but _still_ intended for
machine consumption (e.g., when passing to `.current_dir()`).
## Summary
I tried out `cargo shear` to see if there are any unused dependencies
that `cargo udeps` isn't reporting. It turned out, there are a few. This
PR removes those dependencies.
## Test Plan
`cargo build`
The architecture of uv does not necessarily match that of the python
interpreter (#2326). In cross compiling/testing scenarios the operating
system can also mismatch. To solve this, we move arch and os detection
to python, vendoring the relevant pypa/packaging code, preventing
mismatches between what the python interpreter was compiled for and what
uv was compiled for.
To make the scripts more manageable, they are now a directory in a
tempdir and we run them with `python -m` . I've simplified the
pypa/packaging code since we're still building the tags in rust. A
`Platform` is now instantiated by querying the python interpreter for
its platform. The pypa/packaging files are copied verbatim for easier
updates except a `lru_cache()` python 3.7 backport.
Error handling is done by a `"result": "success|error"` field that allow
passing error details to rust:
```console
$ uv venv --no-cache
× Can't use Python at `/home/konsti/projects/uv/.venv/bin/python3`
╰─▶ Unknown operation system `linux`
```
I've used the [maturin sysconfig
collection](855f6d2cb1/sysconfig)
as reference. I'm unsure how to test these changes across the wide
variety of platforms.
Fixes#2326
## Summary
This PR adds support for pip's `--no-build-isolation`. When enabled,
build requirements won't be installed during PEP 517-style builds, but
the source environment _will_ be used when executing the build steps
themselves.
Closes https://github.com/astral-sh/uv/issues/1715.
## Summary
This PR migrates our virtualenv creation from a setup that assumes prior
knowledge of the correct paths, to a technique borrowed from
`virtualenv` whereby we use `sysconfig` and `distutils` to determine the
paths. The general trick is to grab the expected paths with `sysconfig`,
then make them all relative, then make them absolute for a given
directory.
Closes#2095.
Closes#2153.
## Summary
This will make it easier to use the paths returned by `distutils.py`
(for some cases). No code or behavior changes; just removing some fields
we don't need.
