Commit graph

2240 commits

Author SHA1 Message Date
Zanie Blue
b07c132ede
Skip invalid interpreters when searching for requested interpreter executable name (#4308)
Previously, we took the first executable on the `PATH` but if it was not
a usable interpreter we'd fail. Now, we'll continue searching in the
path until we find an interpreter as we do with the standard executable
names.
2024-06-13 12:36:12 -05:00
konsti
30126950fe
Fix relative and absolute path handling in lockfiles (#4266)
Previously, `b` in the test case would have been incorrectly locked to
the path of `a`. I've moved `relative_to` into uv-fs since it's now used
in two different places.

Previously failing lockfile when `a/pyproject.toml` and
`a/b/pyproject.toml` exist (not in a workspace) and `a` was depending on
`b`:

```toml
version = 1
requires-python = ">=3.11, <3.13"

[[distribution]]
name = "b"
version = "0.1.0"
source = "directory+/home/konsti/projects/uv/a"
sdist = { path = "/home/konsti/projects/uv/a" }

[[distribution]]
name = "black"
version = "0.1.0"
source = "editable+."
sdist = { path = "." }

[[distribution.dependencies]]
name = "b"
version = "0.1.0"
source = "directory+/home/konsti/projects/uv/a"
```
2024-06-13 11:51:08 -04:00
Zanie Blue
89daa51dbe
Add support for listing system toolchains (#4172)
Includes system interpreters in `uv toolchain list`.

This includes a refactor of `find_toolchain` to support iterating over
all toolchains
that match a request rather than ending earlier.
2024-06-13 10:25:30 -05:00
Charlie Marsh
b1bf7f0524
Ignore compile_invalid_pyc_invalidation_mode on macOS (#4305)
## Summary

This keeps failing in CI.
2024-06-13 10:08:28 -04:00
Brian Mego
cd461f1243
Allow --no-binary with uv pip compile (#4301)
## Summary

This i still a draft, but it gets some of the work started on issue
#4064 , which requests --no-binary functionality similar to `uv pip
install` to exist on `uv pip compile`.

So far this has moved the command line shape from install and cloned it
over to compile. The actual functionality of respecting --no-binary
<package> and generating the resulting line in requirements.txt I could
use a hand with.

My understanding is we want to create a requirements.in file such as:

```
yt
```

Then when running `cargo run -- pip compile --no-binary yt
requirements.in`
we want the file to have this line in it:

```
yt==4.3.1 --no-binary yt
```

## Test Plan

Existing unit tests continue to pass.
No new unit tests have been created yet.
The new command line options do show up when testing with `cargo run --
pip compile -h`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-06-13 09:59:03 -04:00
Charlie Marsh
5d1305aa6b
Respect workspace-wide requires-python in interpreter selection (#4298)
## Summary

Closes https://github.com/astral-sh/uv/issues/4296.

## Test Plan

Ran `cargo run lock --verbose` from
`scripts/workspaces/albatross-virtual-workspace`:

```
DEBUG uv 0.2.11 (ef3bc1612 2024-06-12)
warning: `uv lock` is experimental and may change without warning.
DEBUG Found workspace root: `/Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace`
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/albatross
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/seeds
DEBUG Searching for Python >=3.12 in search path or managed toolchains
DEBUG Searching for managed toolchains at `/Users/crmarsh/Library/Application Support/uv/toolchains`
DEBUG Found managed toolchain `cpython-3.12.3-macos-aarch64-none`
DEBUG Found CPython 3.12.3 at `/Users/crmarsh/Library/Application Support/uv/toolchains/cpython-3.12.3-macos-aarch64-none/install/bin/python3` (managed toolchains)
Using Python 3.12.3 interpreter at: /Users/crmarsh/Library/Application Support/uv/toolchains/cpython-3.12.3-macos-aarch64-none/install/bin/python3
```
2024-06-13 12:55:56 +00:00
Zanie Blue
b43de79275
Fix incorrect parsing of requested Python version as empty version specifiers (#4289)
Before 0.2.10 we would parse `--python=python` as an executable name.
After https://github.com/astral-sh/uv/pull/4214, we started treating
this as a Python version range request (with an empty version range).
This is not entirely unreasonable, but it was an unexpected regression
and I don't think `VersionRequest` should support empty ranges in its
`from_str` implementation without more consideration.
2024-06-12 19:48:59 -05:00
Charlie Marsh
a547d7f9dc
Hide --no-system from the CLI (#4292) 2024-06-12 20:22:34 -04:00
Charlie Marsh
544b900432
Tweak copy on some command-line arguments (#4293) 2024-06-12 20:18:22 -04:00
Zanie Blue
5a007b6b9f
Add BuildOptions for centralized combination of NoBuild and NoBinary (#4284)
As requested in review of https://github.com/astral-sh/uv/pull/4067
2024-06-12 21:33:33 +00:00
Zanie Blue
1ab4041baa
Allow specific --only-binary and --no-binary packages to override :all: (#4067)
Updates `--no-binary <package>` to take precedence over `--only-binary
:all:` and `--only-binary <package>` to take precedence over
`--no-binary :all:`.

I'm not entirely sure about this behavior, e.g. maybe I provided
`--only-binary :all:` later on the command line and really want it to
override those earlier arguments of `--no-binary <package>` for safety.
Right now we just fail to solve though since we can't satisfy the
overlapping requests.

Closes https://github.com/astral-sh/uv/issues/4063
2024-06-12 15:47:45 -05:00
Charlie Marsh
cf830288f3
Rename some internal settings structs (#4288)
## Summary

In preparation for future refactors.
2024-06-12 20:35:41 +00:00
Charlie Marsh
d8f1de6134
Use separate path types for directories and files (#4285)
## Summary

This is what I consider to be the "real" fix for #8072. We now treat
directory and path URLs as separate `ParsedUrl` types and
`RequirementSource` types. This removes a lot of `.is_dir()` forking
within the `ParsedUrl::Path` arms and makes some states impossible
(e.g., you can't have a `.whl` path that is editable). It _also_ fixes
the `direct_url.json` for direct URLs that refer to files. Previously,
we wrote out to these as if they were installed as directories, which is
just wrong.
2024-06-12 15:59:21 -04:00
Charlie Marsh
c4483017ac
Add UV_EXCLUDE_NEWER environment variable (#4287)
## Summary

Closes https://github.com/astral-sh/uv/issues/4286.
2024-06-12 15:54:01 -04:00
Zanie Blue
3910b7a90c
Respect requires-python in uv lock (#4282)
We weren't using the common interface in `uv lock` because it didn't
support finding an interpreter without touching the virtual environment.
Here I refactor the project interface to support what we need and update
`uv lock` to use the shared implementation.
2024-06-12 13:19:00 -05:00
Andrew Gallant
e6d0c4d9fe uv/tests: add Requires-Python packse tests
Ref https://github.com/astral-sh/packse/pull/187
Ref https://github.com/astral-sh/packse/pull/188
Ref https://github.com/astral-sh/packse/pull/189
2024-06-12 13:30:47 -04:00
Andrew Gallant
75b323232d uv-resolver: use Requires-Python to filter dependencies during universal resolution
In the time before universal resolving, we would always pass a
`MarkerEnvironment`, and this environment would capture any relevant
`Requires-Python` specifier (including if `-p/--python` was provided on
the CLI).

But in universal resolution, we very specifically do not use a
`MarkerEnvironment` because we want to produce a resolution that is
compatible across potentially multiple environments. This in turn meant
that we lost `Requires-Python` filtering.

This PR adds it back. We do this by converting our `PythonRequirement`
into a `MarkerTree` that encodes the version specifiers in a
`Requires-Python` specifier. We then ask whether that `MarkerTree` is
disjoint with a dependency specification's `MarkerTree`. If it is, then
we know it's impossible for that dependency specification to every be
true, and we can completely ignore it.
2024-06-12 13:30:47 -04:00
Charlie Marsh
c32667caec
Avoid treating direct path archives as always dynamic (#4283)
## Summary

Right now, we're _always_ reinstalling local wheel archives, even if the
timestamp didn't change.

I want to fix the TODO properly but I will do so in a separate PR.
2024-06-12 17:28:29 +00:00
Charlie Marsh
baee826517
Use FxHashMap for available versions (#4278)
## Summary

I don't think we ever iterate over these in-order so I'd rather use
`FxHash` to avoid creating the appearance that the order matters.
2024-06-12 13:16:37 -04:00
Charlie Marsh
44f1afd6b0
Use registry URL for fetching source distributions from lockfile (#4280)
## Summary

This is just a logic bug with no testing. We were using the registry URL
(like `https://pypi.org/simple`) as the URL from which a source
distribution should be downloaded.

Closes https://github.com/astral-sh/uv/issues/4281.

## Test Plan

`cargo test`
2024-06-12 17:01:29 +00:00
Charlie Marsh
16b4a886a8
Use consistent order for extra groups in lockfile (#4275)
## Summary

Closes #4274.
2024-06-12 11:46:16 -04:00
Charlie Marsh
aef74dac2c
Allow normalization to completely eliminate markers (#4271)
## Summary

`normalize` now takes an owned value and returns `Option<MarkerTree>`,
such that if any sub-expression evaluates to `true`, we can normalize
out the entire marker.

Closes https://github.com/astral-sh/uv/issues/4267.
2024-06-12 10:25:43 -04:00
Zanie Blue
f7f55ede2f
Refactor uv-toolchain::platform to use target-lexicon (#4236)
Closes https://github.com/astral-sh/uv/issues/3857

Instead of using custom `Arch`, `Os`, and `Libc` types I just use
`target-lexicon`'s which enumerate way more variants and implement
display and parsing. We use a wrapper type to represent a couple special
cases to support the "x86" alias for "i686" and "macos" for "darwin".
Alternatively we could try to use our `platform-tags` types but those
capture more information (like operating system versions) that we don't
have for downloads.

As discussed in https://github.com/astral-sh/uv/pull/4160, this is not
sufficient for proper libc detection but that work is larger and will be
handled separately.
2024-06-12 09:11:56 -05:00
Nicolas Delaby
5a09c26e77
Fix doc for uv add cli command s/remove/add/ (#4269)
## Summary

Fix the docsting where `remove` was used instead of `add` in the context
of `uv add` command.

## Test Plan

```
cargo run -- add --help
```
```
Add one or more packages to the project requirements

Usage: uv add [OPTIONS] <REQUIREMENTS>...

Arguments:
  <REQUIREMENTS>...
          The packages to add, as PEP 508 requirements (e.g., `flask==2.2.3`)

```
2024-06-12 13:44:08 +00:00
Charlie Marsh
5e7b98d3e7
Use relative path for lib64 symlink (#4268)
## Summary

Closes https://github.com/astral-sh/uv/issues/4265.

## Test Plan

```
❯ ls -l .venv
total 16
-rw-r--r--   1 crmarsh  staff   43 Jun 12 09:23 CACHEDIR.TAG
drwxr-xr-x  14 crmarsh  staff  448 Jun 12 09:23 bin
drwxr-xr-x   3 crmarsh  staff   96 Jun 12 09:23 lib
lrwxr-xr-x   1 crmarsh  staff    3 Jun 12 09:23 lib64 -> lib
-rw-r--r--   1 crmarsh  staff  174 Jun 12 09:23 pyvenv.cfg
```

```
❯ ls .venv/lib64/
python3.12
```
2024-06-12 09:36:27 -04:00
Charlie Marsh
034b4790ea
Add uv version to debug output (#4259)
## Summary

I think this is a useful piece of connective tissue that will let us
avoid back-and-forths when folks include traces.

## Test Plan

```
❯ cargo run pip list --verbose
DEBUG uv 0.2.11 (44041bccd 2024-06-11)
DEBUG Searching for Python interpreter in virtual environments
DEBUG Found CPython 3.12.3 at `/Users/crmarsh/workspace/puffin/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
```
2024-06-11 22:59:31 -04:00
Charlie Marsh
8a8e1af513
Deduplicate markers during normalization (#4263)
## Summary

We need to sort _before_ deduplicating; otherwise, we can't detect
adjacent elements, so we aren't guaranteed to deduplicate anything.
2024-06-12 02:38:15 +00:00
Charlie Marsh
cc2e9ec111
DRY up index argument parsing (#4262)
## Summary

Stealing this from @ibraheemdev's
https://github.com/astral-sh/uv/pull/4237 because it's a nice, isolated
change and I'm about to build atop it.
2024-06-11 22:25:28 -04:00
Charlie Marsh
a1aa35b640
Move project commands into their own subcommand struct (#4261)
## Summary

No changes to the CLI itself (since this is flattened); just code
reorganization.
2024-06-12 02:18:16 +00:00
Charlie Marsh
22795f85bc
Flatten ORs and ANDs in marker construction (#4260)
## Summary

If we're ORing an OR, we should just append rather than nesting in
another OR.

In my branch, this let us simplify:

```
python_version < '3.10' or python_version > '3.12' or (python_version < '3.8' or python_version > '3.12')
```

To:

```
python_version < '3.10' or python_version > '3.12
```
2024-06-11 21:44:49 -04:00
Charlie Marsh
44041bccd2
Bump version to v0.2.11 (#4258) 2024-06-11 20:47:25 -04:00
Charlie Marsh
44592681a0
Represent build tag as u64 (#4253)
## Summary

The build tags in this case are like, e.g., `202206090410`. That's
larger than a `u32`, so we're rejecting the wheel. In theory build tags
could be even larger, but we already use `u64` for version segment so I
think it's fine to keep that constraint here.

I'm going to look into surfacing these errors separately.

Closes https://github.com/astral-sh/uv/issues/4252.

## Test Plan

`cargo run pip install monailabel`
2024-06-11 21:40:08 +00:00
Charlie Marsh
6dae1920af
Make missing METADATA file a recoverable error (#4247)
## Summary

I don't have a great way to test it, but this makes the error described
in https://github.com/astral-sh/uv/issues/4246 an incompatibility rather
than a fatal error.

Closes https://github.com/astral-sh/uv/issues/4246.
2024-06-11 19:49:38 +00:00
Zanie Blue
8cfe202e4e
Improve output when an older toolchain version is already installed (#4248)
e.g.

```
❯ uv toolchain install
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
A toolchain is already installed. Use `uv toolchain install <request>` to install a specific toolchain
```

instead of

```
❯ uv toolchain install
Using latest Python version
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
Already installed at /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.19-macos-aarch64-none
```
2024-06-11 19:49:15 +00:00
Charlie Marsh
dce913c542
Warn when 'requires-python' does not include a lower bound (#4234)
## Summary

Closes https://github.com/astral-sh/uv/issues/4089.
2024-06-11 18:50:05 +00:00
Charlie Marsh
b3a99d9ff9
Allow direct URLs for dev dependencies (#4233)
## Summary

Ensures that they're included in the lookahead resolver.

Closes https://github.com/astral-sh/uv/issues/4230.
2024-06-11 15:30:34 +00:00
Ibraheem Ahmed
3ea5a2a65d
De-duplicate resolution environment logic (#4232)
## Summary

Cleans up some duplicated logic in `pip install/sync/compile`. Will also
be needed for `uv add`.
2024-06-11 11:13:30 -04:00
Ibraheem Ahmed
a75ed22a2f
Fix outdated comment (#4229)
## Summary

Looks like this wasn't updated at some point. I think it's referring to
`UnresolvedRequirement`?
2024-06-11 16:06:47 +02:00
Ibraheem Ahmed
eefa9e62fc
Initial implementation of uv add and uv remove (#4193)
## Summary

Basic implementation of `uv add` and `uv remove` that supports writing
PEP508 requirements to `project.dependencies`.

First step for https://github.com/astral-sh/uv/issues/3959 and
https://github.com/astral-sh/uv/issues/3960.
2024-06-11 09:21:28 -04:00
Nyakku Shigure
60431ce78c
Replace toolchain fetch with toolchain install (#4228)
## Summary

Something that looks like it was forgotten to replace in #4164.

## Test Plan

Run `cargo run toolchain install` should display the warning: ``warning:
`uv toolchain install` is experimental and may change without warning.``
2024-06-11 08:00:33 -04:00
konsti
44833801b3
Support locking relative paths (#4205)
By splitting `path` into a lockable, relative (or absolute) and an
absolute installable path and by splitting between urls and paths by
dist type, we can store relative paths in the lockfile.
2024-06-11 11:58:03 +00:00
Charlie Marsh
33cf47182f
Migrate lock errors to thiserror (#4225)
## Summary

Do we prefer this?
2024-06-11 07:40:48 -04:00
Charlie Marsh
656fc427b9
Add support for local directories with --index-url (#4226)
## Summary

Closes #4078.
2024-06-10 22:27:04 -04:00
Zanie Blue
f296ef08d6
Refactor project interpreter request for requires-python specifiers (#4216)
Refactor following #4214 to avoid parsing the specifiers again
2024-06-10 20:32:40 -05:00
samypr100
68abf85f0d
feat: mTLS support (#4171)
## Summary

Closes https://github.com/astral-sh/uv/issues/3626

This adds mTLS support to uv via the standard env var `SSL_CLIENT_CERT`.

## Test Plan

Tested locally using a [nginx proxy to
pypi](https://github.com/hauntsaninja/nginx_pypi_cache) using my own
self-signed ca + certs + client certs generated via
[mkcert](https://github.com/FiloSottile/mkcert). Used this proxy with
both uv and pip to make sure we have feature partity in mTLS
functionality.
2024-06-10 20:11:35 -05:00
Zanie Blue
5f37395f45
Allow version specifiers to be used in Python version requests (#4214)
In service of https://github.com/astral-sh/uv/issues/4212 but this is
user-facing e.g. Python discovery will support version specifiers
everywhere now.

Closes https://github.com/astral-sh/uv/issues/4212
2024-06-10 18:20:09 -05:00
Charlie Marsh
10e0abc9b1
Create temporary environments in dedicated cache bucket (#4223)
## Summary

We may choose to persist these eventually, but for now, it's useful to
have them colocated with the cache, and in their own dedicated bucket
(so, at the very least, we can keep track of the use-cases).

Closes https://github.com/astral-sh/uv/issues/4219.
2024-06-10 23:17:35 +00:00
Zanie Blue
b5c9003883
Fix pip_compile::missing_venv test (#4224)
A merge kerfuffle from #4222  and #4218 

Now we fail because we genuinely can't find any interpreters since tests
contexts are isolated by default. I'll improve the error message and
maybe add another test case once `main` is fixed.
2024-06-10 22:39:25 +00:00
Zanie Blue
546e23bb1d
Update test context to avoid discovery of external Pythons (#4222)
By setting the test search path to an empty path, we avoid accidentally
pulling interpreters from the system during a test case.

Cherry-picked from https://github.com/astral-sh/uv/pull/4214
2024-06-10 17:26:53 -05:00
Zanie Blue
98d1ea6bb0
Improve handling of missing interpreters during discovery (#4218)
Cherry-picked from https://github.com/astral-sh/uv/pull/4214

The first commit gets us some context on an IO error during queries:

Previously:

```
failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

Now:

```
Failed to query Python interpreter
    Caused by: failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

but really we shouldn't attempt to query a missing interpreter during
discovery anyway, so we improve handling of that too.
2024-06-10 22:26:34 +00:00