Commit graph

199 commits

Author SHA1 Message Date
Di-Is
32ad3323a1
Add constraint dependencies to pyproject.toml (#5248)
Resolves #4467.

## Summary

This PR implements the following

1. Add `tool.uv.constraint-dependencies` to pyproject.toml
1. Support to refer `tool.uv.constraint-dependencies` in `uv lock`
1. Support to refer `tool.uv.constraint-dependencies` in `uv pip
compile/install`

These are analogues of the override features implemented in #3839 and
#4369.

## Test Plan

Add test.
2024-07-21 19:45:04 -04:00
Ibraheem Ahmed
bb73edb03b
Respect local versions for all user requirements (#5232)
## Summary

This fixes a few bugs introduced by
https://github.com/astral-sh/uv/pull/5104. I previously thought we could
track conflicting locals the same way we track conflicting URLs in
forks, but it turns out that ends up being very tricky. URL forks work
because we prioritize directly URL requirements. We can't prioritize
locals in the same way without conflicting with the URL prioritization
(this may be possible but it's not trivial), so we run into issues where
a correct resolution depends on the order in which dependencies are
traversed.

Instead, we track local versions across all forks in `Locals`. When
applying a local version, we apply all locals with markers that
intersect with the current fork. This way we end up applying some local
versions without creating a fork. For example, given:
```
// pyproject.toml
dependencies = [
    "torch==2.0.0+cu118 ; platform_machine == 'x86_64'",
]

// requirements.in
torch==2.0.0
.
```

We choose `2.0.0+cu118` in all cases. However, if a disjoint fork is
created based on local versions, the resolver will choose the most
compatible local when it narrows to a specific fork. Thus we correctly
respect local versions when forking:
```
// pyproject.toml
dependencies = [
    "torch==2.0.0+cu118 ; platform_machine == 'x86_64'",
    "torch==2.0.0+cpu ; platform_machine != 'x86_64'"
]

// requirements.in
torch==2.0.0
.
``` 

We should also be able to use a similar strategy for
https://github.com/astral-sh/uv/pull/5150.

## Test Plan

This fixes https://github.com/astral-sh/uv/issues/5220 locally for me,
as well as a few other bugs that were not reported yet.
2024-07-19 16:56:09 -05:00
Charlie Marsh
ed9b820815
Remove trailing period from user-facing messages (#5218)
## Summary

Per #5209, we only show periods in messages when the message itself
spans more than a single sentence.
2024-07-19 10:43:49 -04:00
konsti
fbb00f701c
Warn about unconstrained direct deps in lowest resolution (#5142)
Warn when there is a direct dependency without a lower bound and
`--resolution lowest` is set.

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-07-18 03:44:53 +00:00
konsti
a6dfd3953a
Handle universal vs. fork markers with ResolverMarkers (#5099)
* Use a dedicated `ResolverMarkers` check in the fork state. This is
better than the `MarkerTree::And(Vec::new())` check.
* Report the timing correct naming universal resolution instead of two
spaces around an empty string when there are no markers.
* On resolution error, show the split that we're in. I'm not sure how to
word this, since we're doing a universal resolution until we fork, so
the trace may contain information from requirements that are not part of
this fork.
2024-07-17 18:59:33 +02:00
Ibraheem Ahmed
d583847f8b
Allow conflicting locals when forking (#5104)
## Summary

Currently, the `Locals` type relies on there being a single local
version for a given package. With marker expressions this may not be
true, a similar problem to https://github.com/astral-sh/uv/pull/4435.
This changes the `Locals` type to `ForkLocals`, which tracks locals for
a given fork. Local versions are now tracked on `PubGrubRequirement`
before forking.

Resolves https://github.com/astral-sh/uv/issues/4580.
2024-07-16 16:57:30 +00:00
skshetry
e2dfab771a
pip compile: change order of check to handle exact argument first (#5111)
I messed up the order of checks in #5033, due to which it failed to
exclude the case of `-P package`, as `arg.startswith("-P")` check came
first and skipped only the first argument.

That means that, in the following command:
```console
uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in -P attrs==18.1.0
```

The generated header would exclude `-P`, but keep `attrs==18.1.0`.

```plaintext
# This file was autogenerated by uv via the following command:
#    uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in attrs==18.1.0
```

But we want to check for an exact match first and then only check for
the case when option and value are together.

This also affected `--find-links` short option of style `-f <uri>`.


Hopefully, third times going to be a charm. 😳 


<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->

I tested locally, and also changed one snapshot test to use `-P` for
variation. I don't think it's worth an extra test, but can do that for
sure.
2024-07-16 12:46:02 -04:00
skshetry
f770b25be2
pip compile: exclude --upgrade-package from the header (#5032)
## Summary

Fixes #5031.

## Test Plan

Existing snapshot tests should cover it.
2024-07-12 23:24:35 -05:00
Charlie Marsh
23eb42deed
Allow constraints to be provided in --upgrade-package (#4952)
## Summary

Allows, e.g., `--upgrade-package flask<3.0.0`.

Closes https://github.com/astral-sh/uv/issues/1964.
2024-07-09 20:09:13 -07:00
konsti
53db63f6dd
Apply extra to overrides and constraints (#4829)
This is an attempt to solve https://github.com/astral-sh/uv/issues/ by
applying the extra marker of the requirement to overrides and
constraints.

Say in `a` we have a requirements
```
b==1; python_version < "3.10"
c==1; extra == "feature"
```

and overrides
```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10"
c==3; python_version >= "3.10"
```

Our current strategy is to discard the markers in the original
requirements. This means that on 3.12 for `a` we install `b==3`, but it
also means that we add `c` to `a` without `a[feature]`, causing #4826.
With this PR, the new requirement become,

```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10" and extra == "feature"
c==3; python_version >= "3.10" and extra == "feature"
```

allowing to override markers while preserving optional dependencies as
such.

Fixes #4826
2024-07-09 20:37:24 +02:00
Charlie Marsh
72dd34b225
Filter out markers based on Python requirement (#4912)
## Summary

In marker normalization, we now remove any markers that are redundant
with the `requires-python` specifier (i.e., always true for the given
Python requirement).

For example, given `iniconfig ; python_version >= '3.7'`, we can remove
the `python_version >= '3.7'` marker when resolving with
`--python-version 3.8`.

Closes #4852.
2024-07-09 09:15:58 -07:00
Ibraheem Ahmed
ed4234d52e
Filter and flatten markers (#4639)
## Summary

More marker simplification:
- Filters out redundant subtrees based on outer expressions, e.g. `a and (a or
b)` simplifies to `a`.
- Flattens nested trees internally, e.g. `(a and b) and c`

Resolves https://github.com/astral-sh/uv/issues/4536.
2024-07-08 18:59:21 +00:00
Andrew Gallant
857d2e8f1e uv-resolver: partially revert Requires-Python version narrowing
The PR #4707 introduced the notion of "version narrowing," where a
Requires-Python constraint was _possibly_ narrowed whenever the
universal resolver created a fork. The version narrowing would occur
when the fork was a result of a marker expression on `python_version`
that is *stricter* than the configured `Requires-Python` (via, say,
`pyproject.toml`).

The crucial conceptual change made by #4707 is therefore that
`Requires-Python` is no longer an invariant configuration of resolution,
but rather a mutable constraint that can vary from fork to fork. This in
turn can result in some cases, such as in #4885, where different
versions of dependencies are selected. We aren't sure whether we can fix
those or not, with version narrowing, so for now, we do this revert to
restore the previous behavior and we'll try to address the version
narrowing some other time.

This also adds the case from #4885 as a regression test, ensuring that
we don't break that in the future. I confirmed that with version
narrowing, this test outputs duplicate distributions. Without narrowing,
there are no duplicates.

Ref #4707, Fixes #4885
2024-07-08 09:56:59 -07:00
Charlie Marsh
ac3a085084
Respect requires-python when prefetching (#4900)
## Summary

This is fallout from https://github.com/astral-sh/uv/pull/4705. We need
to respect `requires-python` in the prefetch code to avoid building
unsupported distributions.

Closes https://github.com/astral-sh/uv/issues/4898.
2024-07-08 16:32:09 +00:00
Charlie Marsh
32ea636585
Preserve verbatim URLs for --find-links (#4838)
Also gets rid of a lot of duplicated logic for `--find-links`.

Closes https://github.com/astral-sh/uv/issues/4797
2024-07-05 16:57:40 -05:00
Charlie Marsh
b588054dfb
Always use release-only comparisons for requires-python (#4794)
## Summary

There are a few ideas at play here:

1. pip always strips versions to the release when evaluating against a
`Requires-Python`, so we now do the same. That means, e.g., using
`3.13.0b0` will be accepted by a project with `Requires-Python: >=
3.13`, which does _not_ adhere to PEP 440 semantics but is somewhat
intuitive.
2. Because we know we'll only be evaluating against release-only
versions, we can use different semantics in PubGrub that let us collapse
ranges. For example, `python_version >= '3.10' or python_version <
'3.10'` can be collapsed to the truthy marker.

Closes https://github.com/astral-sh/uv/issues/4714.
Closes https://github.com/astral-sh/uv/issues/4272.
Closes https://github.com/astral-sh/uv/issues/4719.
2024-07-04 20:06:52 +00:00
Charlie Marsh
d9f389a58d
Narrow requires-python requirement in resolver forks (#4707)
## Summary

Given:

```text
numpy >=1.26 ; python_version >= '3.9'
numpy <1.26 ; python_version < '3.9'
```

When resolving for Python 3.8, we need to narrow the `requires-python`
requirement in the top branch of the fork, because `numpy >=1.26` all
require Python 3.9 or later -- but we know (in that branch) that we only
need to _solve_ for Python 3.9 or later.

Closes https://github.com/astral-sh/uv/issues/4669.
2024-07-02 12:23:38 +00:00
Charlie Marsh
ea6185e082
Merge markers when applying constraints (#4648)
## Summary

When a constraint is applied to a requirement with a marker, the marker
needs to be propagated to the constraint.

If both the constraint and the requirement have a marker, they need to
be merged together (via `and`).

Closes https://github.com/astral-sh/uv/issues/4575.
2024-06-29 16:51:04 +00:00
Charlie Marsh
9b38450998
Handle cycles when propagating markers (#4595)
## Summary

It turns out that `Topo` only works on graphs without cycles. If a graph
has a cycle, it seems to bail early. So we were losing markers for trees
that contain cycles (like Poetry, which depends on
`poetry-plugin-export`, which depends on Poetry).

Now, we remove cycles beforehand and re-add those edges afterwards.

It's a bit hard for me to reason about the implications of this. The way
that marker propagation works is that we do visit the nodes in-order and
propagate the markers from any incoming to any outgoing edges. We only
do this at a single depth (rather than recursively) because we visit the
nodes in-order anyway. But if you have a cycle... then in theory you
might need to propagate the markers recursively? Or maybe not?

As an example:

`A -> B -> C -> D -> B`

If `A -> B` has `sys_platform == 'darwin'`, and then `D -> B` has
`python_version >= '3.7`... then we don't need to propagate
`python_version >= '3.7'` back to `B` or any of its dependencies,
because the condition would be `(sys_platform == 'darwin' or
python_version >= '3.7) or sys_platform == 'darwin'`, which is
equivalent to `sys_platform == 'darwin'`.

Closes #4584.
2024-06-27 17:30:09 -04:00
Charlie Marsh
bbbe1f3968
Avoid enforcing extra-only constraints (#4570)
## Summary

In the dependency refactor (https://github.com/astral-sh/uv/pull/4430),
the logic for requirements and constraints was combined. Specifically,
we were applying constraints _before_ filtering on markers and extras,
and then applying that same filtering to the constraints. As a result,
constraints that should only be activated when an extra is enabled were
being enabled unconditionally.

Closes https://github.com/astral-sh/uv/issues/4569.
2024-06-26 22:52:46 +00:00
konsti
d9dbb8a4af
Support conflicting URL in separate forks (#4435)
Downstack PR: #4481

## Introduction

We support forking the dependency resolution to support conflicting
registry requirements for different platforms, say on package range is
required for an older python version while a newer is required for newer
python versions, or dependencies that are different per platform. We
need to extend this support to direct URL requirements.

```toml
dependencies = [
  "iniconfig @ 62565a6e1c/iniconfig-2.0.0-py3-none-any.whl ; python_version >= '3.12'",
  "iniconfig @ b3c12c6d70/iniconfig-1.1.1-py2.py3-none-any.whl ; python_version < '3.12'"
]
```

This did not work because `Urls` was built on the assumption that there
is a single allowed URL per package. We collect all allowed URL ahead of
resolution by following direct URL dependencies (including path
dependencies) transitively, i.e. a registry distribution can't require a
URL.

## The same package can have Registry and URL requirements

Consider the following two cases:

requirements.in:
```text
werkzeug==2.0.0
werkzeug @ 960bb4017c/Werkzeug-2.0.0-py3-none-any.whl
```
pyproject.toml:
```toml
dependencies = [
  "iniconfig == 1.1.1 ; python_version < '3.12'",
  "iniconfig @ git+https://github.com/pytest-dev/iniconfig@93f5930e668c0d1ddf4597e38dd0dea4e2665e7a ; python_version >= '3.12'",
]
```

In the first case, we want the URL to override the registry dependency,
in the second case we want to fork and have one branch use the registry
and the other the URL. We have to know about this in
`PubGrubRequirement::from_registry_requirement`, but we only fork after
the current method.

Consider the following case too:

a:
```
c==1.0.0
b @ https://b.zip
```
b:
```
c @ https://c_new.zip ; python_version >= '3.12'",
c @ https://c_old.zip ; python_version < '3.12'",
```

When we convert the requirements of `a`, we can't know the url of `c`
yet. The solution is to remove the `Url` from `PubGrubPackage`: The
`Url` is redundant with `PackageName`, there can be only one url per
package name per fork. We now do the following: We track the urls from
requirements in `PubGrubDependency`. After forking, we call
`add_package_version_dependencies` where we apply override URLs, check
if the URL is allowed and check if the url is unique in this fork. When
we request a distribution, we ask the fork urls for the real URL. Since
we prioritize url dependencies over registry dependencies and skip
packages with `Urls` entries in pre-visiting, we know that when fetching
a package, we know if it has a url or not.

## URL conflicts

pyproject.toml (invalid):
```toml
dependencies = [
  "iniconfig @ e96292c7f7/iniconfig-1.1.0.tar.gz",
  "iniconfig @ b3c12c6d70/iniconfig-1.1.1-py2.py3-none-any.whl ; python_version < '3.12'",
  "iniconfig @ 62565a6e1c/iniconfig-2.0.0-py3-none-any.whl ; python_version >= '3.12'",
]
```

On the fork state, we keep `ForkUrls` that check for conflicts after
forking, rejecting the third case because we added two packages of the
same name with different URLs.

We need to flatten out the requirements before transformation into
pubgrub requirements to get the full list of other requirements which
may contain a URL, which was changed in a previous PR: #4430.

## Complex Example

a:
```toml
dependencies = [
  # Force a split
  "anyio==4.3.0 ; python_version >= '3.12'",
  "anyio==4.2.0 ; python_version < '3.12'",
  # Include URLs transitively
  "b"
]
```
b:
```toml
dependencies = [
  # Only one is used in each split.
  "b1 ; python_version < '3.12'",
  "b2 ; python_version >= '3.12'",
  "b3 ; python_version >= '3.12'",
]
```
b1:
```toml
dependencies = [
  "iniconfig @ b3c12c6d70/iniconfig-1.1.1-py2.py3-none-any.whl",
]
```
b2:
```toml
dependencies = [
  "iniconfig @ 62565a6e1c/iniconfig-2.0.0-py3-none-any.whl",
]
```
b3:
```toml
dependencies = [
  "iniconfig @ e96292c7f7/iniconfig-1.1.0.tar.gz",
]
```

In this example, all packages are url requirements (directory
requirements) and the root package is `a`. We first split on `a`, `b`
being in each split. In the first fork, we reach `b1`, the fork URLs are
empty, we insert the iniconfig 1.1.1 URL, and then we skip over `b2` and
`b3` since the mark is disjoint with the fork markers. In the second
fork, we skip over `b1`, visit `b2`, insert the iniconfig 2.0.0 URL into
the again empty fork URLs, then visit `b3` and try to insert the
iniconfig 1.1.0 URL. At this point we find a conflict for the iniconfig
URL and error.

## Closing

The git tests are slow, but they make the best example for different URL
types i could find.

Part of #3927. This PR does not handle `Locals` or pre-releases yet.
2024-06-26 13:58:23 +02:00
Charlie Marsh
41f051db3b
Remove exclude newer methods on test context (#4535)
## Summary

Closes https://github.com/astral-sh/uv/issues/4531.
2024-06-25 23:45:35 +00:00
konsti
e6103dcab1
Deduplicate test command creation (#4512)
This PR refactors the command creation in the test suite to remove the
duplication.

**1)** We add the same set of test stubbing args to almost any uv
invocation in the tests:

```rust
command
    .arg("--cache-dir")
    .arg(self.cache_dir.path())
    .env("VIRTUAL_ENV", self.venv.as_os_str())
    .env("UV_NO_WRAP", "1")
    .env("HOME", self.home_dir.as_os_str())
    .env("UV_TOOLCHAIN_DIR", "")
    .env("UV_TEST_PYTHON_PATH", &self.python_path())
    .current_dir(self.temp_dir.path());

if cfg!(all(windows, debug_assertions)) {
    // TODO(konstin): Reduce stack usage in debug mode enough that the tests pass with the
    // default windows stack of 1MB
    command.env("UV_STACK_SIZE", (8 * 1024 * 1024).to_string());
}
```

Centralizing these into a `TestContext::add_shared_args` method removes
them from everywhere.

**2)** Prefix all `TextContext` methods of the pip interface with
`pip_`. This is now necessary due to `uv sync` vs. `uv pip sync`.

**3)** Move command creation in the various test files into dedicated
functions or methods to avoid repeating the arguments. Except for error
message tests, there should be at most one `Command::new(get_bin())`
call per test file. `EXCLUDE_NEWER` is exclusively used in
`TestContext`.

---

I'm considering adding a `TestCommand` on top of these changes (in
another PR) that holds a reference to the `TextContext`, has
`add_shared_args` as a method and uses `Fn(Self) -> Self` instead of
`Fn(&mut Self) -> Self` for methods to improved chaining.
2024-06-25 22:06:54 +00:00
Charlie Marsh
e1708689a9
Add a universal resolution mode to pip compile (#4505)
## Summary

This needs more tests and a few more changes, but checkpointing it for
now.
2024-06-25 21:28:50 +00:00
Charlie Marsh
5732209be3
Add support for --no-strip-markers in pip compile output (#4503)
## Summary

This is an intermediary change in enabling universal resolution for
`requirements.txt` files. To start, we need to be able to preserve
markers in the `requirements.txt` output _and_ propagate those markers,
such that if you have a dependency that's only included with a given
marker, the transitive dependencies respect that marker too.

Closes #1429.
2024-06-25 20:55:58 +00:00
Charlie Marsh
a81742c06b
Read persistent configuration from non-workspace pyproject.toml (#4526)
## Summary

If the user puts their configuration in a `pyproject.toml` that _isn't_
a valid workspace root (e.g., it's a Poetry file), we won't discover it,
because we only look in `uv.toml` files in that case. I think this is
somewhat debatable... We could choose to _require_ `uv.toml` there, but
as a user I'd probably expect it to work?

Closes https://github.com/astral-sh/uv/issues/4521.
2024-06-25 18:53:14 +00:00
Charlie Marsh
f07308823e
Add --emit-build-options flag to uv pip compile interface (#4463)
## Summary

Closes https://github.com/astral-sh/uv/issues/4420.
2024-06-24 12:25:01 +00:00
renovate[bot]
3251690327
Update Rust crate rustc-hash to v2 (#4461) 2024-06-24 01:13:05 +00:00
Zanie Blue
b22ee82f0d
Use "Prepared" instead of "Downloaded" in logs (#4394)
We download, build, and unzip packages in this stage. The current name
is very misleading.

Closes https://github.com/astral-sh/uv/issues/4011
2024-06-18 13:38:18 -05:00
Zanie Blue
76c26db444
Move virtual environment test context into main context (#4370)
It was becoming problematic that the virtual environment test context
diverged from the other one i.e. we had to implement filtering twice.
This combines the contexts and tweaks the `TestContext` API and
filtering mechanisms for Python versions. Combined with my previous
changes to the test context at #4364 and
https://github.com/astral-sh/uv/pull/4368 this finally unblocks the
snapshots for test cases in #4360 and
https://github.com/astral-sh/uv/pull/4362.
2024-06-18 09:11:47 -05:00
Charlie Marsh
f01ab57518
Add a --show-settings option for configuration testing (#4304)
## Summary

The fixtures here are pretty large, but it lets us test what we actually
care about (the resolved settings) rather than inferring the resolved
settings from behavior, which I think is a big improvement.

I also broke the tests down into more granular cases.
2024-06-14 03:14:23 +00:00
Charlie Marsh
cacd1a2b5a
Load configuration options from workspace root (#4295)
## Summary

In a workspace, we now read configuration from the workspace root.
Previously, we read configuration from the first `pyproject.toml` or
`uv.toml` file in path -- but in a workspace, that would often be the
_project_ rather than the workspace configuration.

We need to read configuration from the workspace root, rather than its
members, because we lock the workspace globally, so all configuration
applies to the workspace globally.

As part of this change, the `uv-workspace` crate has been renamed to
`uv-settings` and its purpose has been narrowed significantly (it no
longer discovers a workspace; instead, it just reads the settings from a
directory).

If a user has a `uv.toml` in their directory or in a parent directory
but is _not_ in a workspace, we will still respect that use-case as
before.

Closes #4249.
2024-06-14 01:26:20 +00:00
Charlie Marsh
e0a389032f
Add persistent configuration for non-pip APIs (#4294)
## Summary

This PR introduces top-level configuration for uv, such that you can do:

```toml
[tool.uv]
index-url = "https://test.pypi.org/simple"
```

And `uv pip compile`, `uv run`, `uv tool run`, etc., will all respect
that configuration.

The settings that were escalated to the top-level remain on
`tool.uv.pip` too, but they're only respected in `uv pip` commands. If
they're specified in both places, then the `pip` settings win out.

While making this change, I also wired up some of the global options,
like `connectivity` and `native_tls`, through to all the relevant
places.

Closes #4250.
2024-06-13 20:56:38 -04:00
Brian Mego
cd461f1243
Allow --no-binary with uv pip compile (#4301)
## Summary

This i still a draft, but it gets some of the work started on issue
#4064 , which requests --no-binary functionality similar to `uv pip
install` to exist on `uv pip compile`.

So far this has moved the command line shape from install and cloned it
over to compile. The actual functionality of respecting --no-binary
<package> and generating the resulting line in requirements.txt I could
use a hand with.

My understanding is we want to create a requirements.in file such as:

```
yt
```

Then when running `cargo run -- pip compile --no-binary yt
requirements.in`
we want the file to have this line in it:

```
yt==4.3.1 --no-binary yt
```

## Test Plan

Existing unit tests continue to pass.
No new unit tests have been created yet.
The new command line options do show up when testing with `cargo run --
pip compile -h`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-06-13 09:59:03 -04:00
Charlie Marsh
d8f1de6134
Use separate path types for directories and files (#4285)
## Summary

This is what I consider to be the "real" fix for #8072. We now treat
directory and path URLs as separate `ParsedUrl` types and
`RequirementSource` types. This removes a lot of `.is_dir()` forking
within the `ParsedUrl::Path` arms and makes some states impossible
(e.g., you can't have a `.whl` path that is editable). It _also_ fixes
the `direct_url.json` for direct URLs that refer to files. Previously,
we wrote out to these as if they were installed as directories, which is
just wrong.
2024-06-12 15:59:21 -04:00
Zanie Blue
b5c9003883
Fix pip_compile::missing_venv test (#4224)
A merge kerfuffle from #4222  and #4218 

Now we fail because we genuinely can't find any interpreters since tests
contexts are isolated by default. I'll improve the error message and
maybe add another test case once `main` is fixed.
2024-06-10 22:39:25 +00:00
Zanie Blue
98d1ea6bb0
Improve handling of missing interpreters during discovery (#4218)
Cherry-picked from https://github.com/astral-sh/uv/pull/4214

The first commit gets us some context on an IO error during queries:

Previously:

```
failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

Now:

```
Failed to query Python interpreter
    Caused by: failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

but really we shouldn't attempt to query a missing interpreter during
discovery anyway, so we improve handling of that too.
2024-06-10 22:26:34 +00:00
Charlie Marsh
125a4b220e
Improve static metadata extraction for Poetry projects (#4182)
## Summary

Adds handling for a few cases to improve interoperability with Poetry:

- If the `project` schema is invalid, we now raise a hard error, rather
than treating the metadata as dynamic and then falling back to the build
backend. This could cause problems, I'm not sure. It's stricter than
before.
- If the project contains `tool.poetry` but omits
`project.dependencies`, we now treat it as dynamic. We could go even
further and treat _any_ Poetry project as dynamic, but then we'd be
ignoring user-declared dependencies, which is also confusing.

Closes https://github.com/astral-sh/uv/issues/4142.
2024-06-10 10:26:38 -04:00
Charlie Marsh
d7cc622d6c
Accept file:// URLs for requirements.txt et all references (#4145)
## Summary

Closes https://github.com/astral-sh/uv/issues/4124.
2024-06-07 22:03:08 +00:00
Charlie Marsh
0acae9bd9c
Add support for development dependencies (#4036)
## Summary

Externally, development dependencies are currently structured as a flat
list of PEP 580-compatible requirements:

```toml
[tool.uv]
dev-dependencies = ["werkzeug"]
```

When locking, we lock all development dependencies; when syncing, users
can provide `--dev`.

Internally, though, we model them as dependency groups, similar to
Poetry, PDM, and [PEP 735](https://peps.python.org/pep-0735). This
enables us to change out the user-facing frontend without changing the
internal implementation, once we've decided how these should be exposed
to users.

A few important decisions encoded in the implementation (which we can
change later):

1. Groups are enabled globally, for all dependencies. This differs from
extras, which are enabled on a per-requirement basis. Note, however,
that we'll only discover groups for uv-enabled packages anyway.
2. Installing a group requires installing the base package. We rely on
this in PubGrub to ensure that we resolve to the same version (even
though we only expect groups to come from workspace dependencies anyway,
which are unique). But anyway, that's encoded in the resolver right now,
just as it is for extras.
2024-06-06 01:40:17 +00:00
Charlie Marsh
191f9556b7
Avoid building packages with dynamic versions (#4058)
## Summary

This PR separates "gathering the requirements" from the rest of the
metadata (e.g., version), which isn't required when installing a
package's _dependencies_ (as opposed to installing the package itself).
It thus ensures that we don't need to build a package when a static
`pyproject.toml` is provided in `pip compile`.

Closes https://github.com/astral-sh/uv/issues/4040.
2024-06-05 18:11:58 +00:00
Charlie Marsh
6d148da0ab
Preserve fragments when applying verbatim redirects (#4038)
## Summary

`echo
"git+https://github.com/pypa/sample-namespace-packages.git#subdirectory=pkg_resources/pkg_a"
| cargo run pip compile -` now resolves to a compliant URL.

Closes https://github.com/astral-sh/uv/issues/4037.
2024-06-05 03:53:23 +00:00
Di-Is
41ec302557
Fix a bug where no warning is output when parsing of workspace settings fails. (#4014)
## Summary

See #4013

`uv pip ...` command loads workspace settings from pyproject.toml and
uv.toml.

Although a warning is implemented to output a warning when parsing
fails, it is not actually output.


https://github.com/astral-sh/uv/blob/main/crates/uv-workspace/src/workspace.rs#L38-L61

The reason is that the flag to display warnings is enabled after loading
the workspace settings.

This PR turns on the warning output flag before loading the workspace.

## Test Plan

pyproject.toml for test

```toml
[project]
name = "sample"
version = "0.0.0"
dependencies = ["ruff"]

[tool.uv.pip]
# originally string type.
index-url = 1
```

command output (before modification)
```bash
uv pip compile pyproject.toml 

Resolved 1 package in 383ms
# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml
ruff==0.4.7
    # via sample (pyproject.toml)
```

command output (after modification)

```bash
uv pip compile pyproject.toml

warning: Failed to parse `pyproject.toml`: TOML parse error at line 7, column 13
  |
7 | index-url = true
  |             ^^^^
invalid type: boolean `true`, expected a string

Resolved 1 package in 107ms
# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml
ruff==0.4.7
    # via sample (pyproject.toml)
```
2024-06-04 09:21:19 -04:00
Di-Is
5c776939d2
Add override namespace to pyproject.toml/uv.toml (#3839)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

See #3834 .

This PR adds a new namespace, `override-dependencies`, to
pyproject.toml/uv.toml.
This namespace assumes that the dependencies you want to override are
written in the form of `requirements.txt`.


a example of pyproject.toml
```toml
[project]
name = "example"
version = "0.0.0"
dependencies = [
  "flask==3.0.0"
]

[tool.uv]
override-dependencies = [
  "werkzeug==2.3.0"
]
```

This will improve usability by allowing you to override dependencies
without having to specify the --override option when running `uv pip
compile/install`.

## Test Plan

added test to `crates/uv/tests/pip_compile.rs`.

---------

Co-authored-by: konstin <konstin@mailbox.org>
2024-06-03 12:15:51 +02:00
Charlie Marsh
8c11f99fdf
Discard fragments when parsing unnamed URLs (#3940)
## Summary

Closes https://github.com/astral-sh/uv/issues/3934.
2024-05-31 13:54:15 +00:00
konsti
081f20c53e
Add support for tool.uv into distribution building (#3904)
With the change, we remove the special casing of workspace dependencies
and resolve `tool.uv` for all git and directory distributions. This
gives us support for non-editable workspace dependencies and path
dependencies in other workspaces. It removes a lot of special casing
around workspaces. These changes are the groundwork for supporting
`tool.uv` with dynamic metadata.

The basis for this change is moving `Requirement` from
`distribution-types` to `pypi-types` and the lowering logic from
`uv-requirements` to `uv-distribution`. This changes should be split out
in separate PRs.

I've included an example workspace `albatross-root-workspace2` where
`bird-feeder` depends on `a` from another workspace `ab`. There's a
bunch of failing tests and regressed error messages that still need
fixing. It does fix the audited package count for the workspace tests.
2024-05-31 02:42:03 +00:00
Charlie Marsh
1fc6a59707
Remove special-casing for editable requirements (#3869)
## Summary

There are a few behavior changes in here:

- We now enforce `--require-hashes` for editables, like pip. So if you
use `--require-hashes` with an editable requirement, we'll reject it. I
could change this if it seems off.
- We now treat source tree requirements, editable or not (e.g., both `-e
./black` and `./black`) as if `--refresh` is always enabled. This
doesn't mean that we _always_ rebuild them; but if you pass
`--reinstall`, then yes, we always rebuild them. I think this is an
improvement and is close to how editables work today.

Closes #3844.

Closes #2695.
2024-05-28 15:49:34 +00:00
konsti
a89e146107
Initial workspace support (#3705)
Add workspace support when using `-r <path>/pyproject.toml` or `-e
<path>` in the pip interface. It is limited to all-editable
static-metadata workspaces, and tests only include a single main
workspace, ignoring path dependencies in another workspace. This can be
considered the MVP for workspace support: You can create a workspace,
you can install from it, but some options and conveniences are still
missing. I'll file follow-up tickets (support in lockfiles, support path
deps in other workspace, #3625)

There is also support in `uv run`, but we need
https://github.com/astral-sh/uv/issues/3700 first to properly support
using different current projects in the bluejay interface, currently the
resolution and therefore the lockfile depends on the current project.
I'd do this change first (it's big enough already), then #3700, and then
add workspace support properly to bluejay.

Fixes #3404
2024-05-28 07:41:53 +00:00
Charlie Marsh
688f4f43d4
Ignore unnamed requirements in preferences (#3826)
## Summary

We actually _already_ ignore these (preferences only apply to versions,
not URLs), it just happens later on. This PR thus just avoids crashing.
The behavior is unchanged.

Closes #3822.
2024-05-24 17:32:20 +00:00
Charlie Marsh
999d072ae9
Always include package names for Git and HTTPS dependencies (#3821)
## Summary

Related to https://github.com/astral-sh/uv/issues/3818. We should
_always_ include the package name if we know it's not a file path, even
if it starts with an environment variable.
2024-05-24 14:01:38 +00:00