Commit graph

556 commits

Author SHA1 Message Date
Charlie Marsh
9a7262c360
Avoid batch prefetching for un-optimized registries (#7226)
## Summary

We now track the discovered `IndexCapabilities` for each `IndexUrl`. If
we learn that an index doesn't support range requests, we avoid doing
any batch prefetching.

Closes https://github.com/astral-sh/uv/issues/7221.
2024-09-09 15:46:19 -04:00
Charlie Marsh
dcbaa1f486
Avoid iteration for singleton selections (#7195)
## Summary

If we have a singleton `Range`, we don't need to iterate over the map of
available ranges; instead, we can just get the singleton directly.

Closes #6131.
2024-09-09 13:43:57 +00:00
Charlie Marsh
dafa3596c5
Avoid distribution clones in requirements.txt graph (#7210) 2024-09-09 13:31:13 +00:00
Charlie Marsh
b738b35910
Prune unreachable packages from --universal output (#7209)
## Summary

Closes https://github.com/astral-sh/uv/issues/7196.
2024-09-09 09:20:25 -04:00
Charlie Marsh
4466402214
DRY up exclusion checks in selector (#7194) 2024-09-08 17:42:41 +00:00
Andrew Gallant
f6bc701ac3 uv-resolver: use new simplify/complexify marker routines
This finally gets rid of our hack for working around "hidden"
state. We no longer do a roundtrip marker serialization and
deserialization just to avoid the hidden state.
2024-09-07 13:46:02 -04:00
Charlie Marsh
7d49fbc753
Skip metadata fetch for --no-deps and pip sync (#7127)
## Summary

I think a better tradeoff here is to skip fetching metadata, even though
we can't validate the extras.

It will help with situations like
https://github.com/astral-sh/uv/issues/5073#issuecomment-2334235588 in
which, otherwise, we have to download the wheels twice.
2024-09-06 21:26:28 -04:00
Amos Wenger
5e1b9b1964
chore: Remove dep on derivative (#7133)
(This is part of #5711)

## Summary

@BurntSushi and I spotted that the `derivative` crate is only used for
one enum in the entire codebase — however, it's a proc macro, and we pay
for the cost of (re)compiling it in many different contexts.

This replaces it with a private `Inner` core which uses the regular std
derive macros — inlining and optimizations should make this equivalent
to the other implementation, and not too hard to maintain hopefully
(versus a manual impl of `PartialEq` and `Hash` which have to be kept in
sync.)

## Test Plan

Trust CI?
2024-09-06 17:46:56 -04:00
Charlie Marsh
6ae005b0d0
Add --no-emit-project and friends to uv export (#7110)
## Summary

Like `uv sync`, you can omit the current project (`--no-emit-project`),
a specific package (`--no-emit-package`), or the entire workspace
(`--no-emit-workspace`).

Closes https://github.com/astral-sh/uv/issues/6960.

Closes #6995.
2024-09-06 01:01:46 +00:00
Charlie Marsh
58b25b560c
Invalidate lockfile when member versions change (#7102)
## Summary

Closes https://github.com/astral-sh/uv/issues/7101.

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-09-05 17:14:57 -05:00
Charlie Marsh
29f53c3c63
Support --no-build and --no-binary in uv sync et al (#7100)
## Summary

This option already existed, but `--no-binary` always errored.

Closes https://github.com/astral-sh/uv/issues/7099.
2024-09-05 21:45:42 +00:00
konsti
750e8b1e2f
Remove propagate_markers (#7076)
Follow-up to #6959 and #6961: Use the reachability computation instead
of `propagate_markers` everywhere.

With `marker_reachability`, we have a function that computes for each
node the markers under which it is (`requirements.txt`, no markers
provided on installation) or can be (`uv.lock`, depending on the markers
provided on installation) included in the installation. Put differently:
If the marker computed by `marker_reachability` is not fulfilled for the
current platform, the package is never required on the current platform.

We compute the markers for each package in the graph, this includes the
virtual extra packages and the base packages. Since we know that each
virtual extra package depends on its base package (`foo[bar]` implied
`foo`), we only retain the base package marker in the `requirements.txt`
graph.

In #6959/#6961 we were only using it for pruning packages in `uv.lock`,
now we're also using it for the markers in `requirements.txt`.

I think this closes #4645, CC @bluss.
2024-09-05 16:52:21 +00:00
konsti
d5eb6eb12c
Move reachability function (#7091)
Split out from #7076
2024-09-05 16:44:21 +00:00
Charlie Marsh
d91f7ddf5f
Don't include registry hashes for URL distributions (#7086)
The opposite _can_ be true.
2024-09-05 11:42:07 -04:00
Charlie Marsh
a20fecedaa
Strip fragments from direct source URLs in lockfile (#7061)
## Summary

In resolving https://github.com/astral-sh/uv/issues/7059, I noticed that
we left the fragment on the `source = { url = "..." }`.
2024-09-05 02:05:39 +00:00
Charlie Marsh
1f7a9a7407
Use distribution hash over registry hash (#7060)
## Summary

We need to prioritize hashes for the distribution over hashes for the
related packages.

I think this needs to be redone entirely though. I can see other issues
with the current approach.

Closes https://github.com/astral-sh/uv/issues/7059.
2024-09-05 01:58:21 +00:00
Charlie Marsh
1ccc15e7bb
Use correct ordering semantics for narrowing upper-bounded Python requirements (#7031)
## Summary

We need to use different ordering semantics for upper and lower Python
bounds.

Closes https://github.com/astral-sh/uv/issues/6911.
2024-09-04 15:57:14 +00:00
konsti
c9787f9fd8
Prune unreachable wheels from lockfile (#6961)
When a package is included under a platform-specific marker, we know
that wheels that mismatch this marker can never be installed, so we drop
them from the lockfile.
2024-09-04 11:08:37 +02:00
konsti
b9f2bd155a
Prune unreachable packages from lockfile (#6959)
In transformers, we have:

* `tensorflow-text`: `tensorflow-macos; python_full_version >= '3.13'
and platform_machine == 'arm64' and platform_system == 'Darwin'`
* `tensorflow-macos`: `tensorflow-cpu-aws; (python_full_version < '3.10'
and platform_machine == 'aarch64' and platform_system == 'Linux') or
(python_full_version >= '3.13' and platform_machine == 'aarch64' and
platform_system == 'Linux') or (python_full_version >= '3.13' and
platform_machine == 'arm64' and platform_system == 'Linux')`
* `tensorflow-macos`: `tensorflow-intel; python_full_version >= '3.13'
and platform_system == 'Windows'`

This means that `tensorflow-cpu-aws` and `tensorflow-intel` can never be
installed, and we can drop them from the lockfile.
2024-09-04 10:57:42 +02:00
Andrew Gallant
2aeb51eebb uv-resolver: update snapshot tests with new Debug repr for Dependency
A `Dependency` now has both "simplified" and "complexified" markers, so
just update the snapshots to match the new reality.
2024-09-03 18:41:15 -04:00
Andrew Gallant
4ff057e108 uv-resolver: refactor how we deal with requires-python
This commit refactors how deal with `requires-python` so that instead of
simplifying markers of dependencies inside the resolver, we do it at the
edges of our system. When writing markers to output, we simplify when
there's an obvious `requires-python` context. And when reading markers
as input, we complexity markers with the relevant `requires-python`
constraint.
2024-09-03 18:41:15 -04:00
Andrew Gallant
94a0a0f2ee uv-resolver: rejigger 'trace_resolution'
When I first wrote this routine, it was intended to only emit a trace
for the final "unioned" resolution. But we actually moved that semantic
operation to the construction of the resolution *graph*. So there is no
unioned `Resolution` any more.

But this is still useful to see. So I changed this to just emit a trace
of *every* resolution right before constructing the graph.

It might be nice to also emit a trace of the unioned graph too. Or
perhaps we should do that instead if this proves too noisy. (Although
this is only emitted at TRACE level.)
2024-09-03 18:41:15 -04:00
konsti
9e34c42cec
Move unreachable wheels check (#6957)
Prep for fixing #6512. No functional changes.
2024-09-03 07:48:54 +00:00
Charlie Marsh
ccdf2d793b
Add --no-hashes to uv export (#6954)
## Summary

Closes https://github.com/astral-sh/uv/issues/6944.
2024-09-02 22:12:29 -04:00
Charlie Marsh
f9c04581e6
Allow uv sync --package without copying member pyproject.toml (#6943)
## Summary

Closes https://github.com/astral-sh/uv/issues/6935.
2024-09-02 21:01:50 +00:00
Charlie Marsh
8eef1a2314
Use lower-bound semantics for all Python compatibility comparisons (#6882)
## Summary

Right now, we have slightly different `requires-python` semantics for
`-p 3.11` vs. `-p 3.11 --universal`, and slightly different (wrong)
semantics for how we compare against the _installed_ Python version
(which doesn't ignore upper bounds, but should).

This PR rips it all out and replaces it with consistent semantics across
`uv lock`, `uv pip compile -p 3.11`, and `uv pip compile -p 3.11
--universal`. We now always ignore upper bounds.

Closes https://github.com/astral-sh/uv/issues/6859.

Closes https://github.com/astral-sh/uv/issues/5045.
2024-09-02 18:23:42 +00:00
Ed Morley
a5f1e1c765
Fix typos in docs, error messages and comments (#6910) 2024-09-01 11:37:43 +00:00
Charlie Marsh
3e647b139e
Fix is_disjoint check for supported environments (#6902) 2024-08-31 19:01:47 -04:00
konsti
00c98a82b0
Use from_range_bounds (#6879)
Not the most ergonomic api pubgrub has to offer, but better than rolling
our own.
2024-08-30 16:06:55 -04:00
Charlie Marsh
9f8ebca941
Error when user-provided environments are disjoint with Python (#6841) 2024-08-29 20:18:30 -04:00
Charlie Marsh
34d74501ac
Respect the user's upper-bound in requires-python (#6824)
## Summary

We now respect the user-provided upper-bound in for `requires-python`.
So, if the user has `requires-python = "==3.11.*"`, we won't explore
forks that have `python_version >= '3.12'`, for example.

However, we continue to _only_ compare the lower bounds when assessing
whether a dependency is compatible with a given Python range.

Closes https://github.com/astral-sh/uv/issues/6150.
2024-08-29 18:37:05 +00:00
Charlie Marsh
cbfc928a9c
Add uv export --format requirements.txt (#6778)
## Summary

The interface here is intentionally a bit more limited than `uv pip
compile`, because we don't want `requirements.txt` to be a system of
record -- it's just an export format. So, we don't write annotation
comments (i.e., which dependency is requested from which), we don't
allow writing extras, etc. It's just a flat list of requirements, with
their markers and hashes.

Closes #6007.

Closes #6668.

Closes #6670.
2024-08-29 17:46:42 +00:00
Charlie Marsh
c166e65ba6
Move lock.rs into its own module (#6775)
## Summary

Desperately need the ability to start splitting up code here.
2024-08-28 18:04:45 -04:00
Charlie Marsh
6a8da7dff8
Compare virtual members when invalidating lockfile (#6754)
## Summary

Whether a package is itself virtual isn't captured in the package
metadata, so we have to compare the sources.

Closes https://github.com/astral-sh/uv/issues/6749.
2024-08-28 15:11:16 +00:00
Charlie Marsh
53ef633c6d
Do not require workspace members to sync with --frozen (#6737)
## Summary

Closes https://github.com/astral-sh/uv/issues/6685.
2024-08-28 07:58:50 -04:00
Charlie Marsh
56cc0c9b3c
Avoid using editable tag in lockfile for non-package dependencies (#6728)
## Summary

Use a dedicated source type for non-package requirements. Also enables
us to support non-package `path` dependencies _and_ removes the need to
have the member `pyproject.toml` files available when we sync _and_
makes it explicit which dependencies are virtual vs. not (as evidenced
by the snapshot changes). All good things!
2024-08-28 01:19:05 +00:00
Charlie Marsh
3ee6ca31f4
Rename virtual workspace roots to non-project workspace roots (#6717)
## Summary

Closes https://github.com/astral-sh/uv/issues/6709.
2024-08-27 21:36:40 +00:00
Charlie Marsh
a7850d2a1c
Use separate types to represent raw vs. resolver markers (#6646)
## Summary

This is similar to https://github.com/astral-sh/uv/pull/6171 but more
expansive... _Anywhere_ that we test requirements for platform
compatibility, we _need_ to respect the resolver-friendly markers. In
fixing the motivating issue (#6621), I also realized that we had a bunch
of bugs here around `pip install` with `--python-platform` and
`--python-version`, because we always performed our `satisfy` and `Plan`
operations on the interpreter's markers, not the adjusted markers!

Closes https://github.com/astral-sh/uv/issues/6621.
2024-08-26 18:00:21 +00:00
Charlie Marsh
2ec7c69861
Respect extras and markers on virtual dev dependencies (#6620)
## Summary

Closes https://github.com/astral-sh/uv/issues/6617.
2024-08-26 00:31:42 +00:00
Charlie Marsh
7fa265a11b
Use relative paths for --find-links and local registries (#6566)
## Summary

See: https://github.com/astral-sh/uv/issues/6458
2024-08-25 02:41:47 +00:00
Charlie Marsh
1eb97c91fd
Remove FileLocation::Path variant (#6577)
## Summary

This is redundant now that we support `file://` URLs.
2024-08-24 07:52:43 -04:00
Charlie Marsh
f7835243c5
Only use relative paths in lockfile (#6490)
For users who were using absolute paths in the `pyproject.toml`
previously, this is a behavior change: We now convert all absolute paths
in `path` entries to relative paths. Since i assume that no-one relies
on absolute path in their lockfiles - they are intended to be portable -
I'm tagging this as a bugfix.

Closes https://github.com/astral-sh/uv/pull/6438
Fixes https://github.com/astral-sh/uv/issues/6371
2024-08-23 22:19:10 -04:00
Charlie Marsh
19a7f3ec07
Collapse extras on dev dependencies (#6383)
## Summary

It turns out we weren't applying the collapse logic here, so dev deps
with extras were repeated. This was generally ok... unless we ended up
_dropping_ an extra, in which case, you now have a duplicate.

Closes https://github.com/astral-sh/uv/issues/6380.
2024-08-21 22:36:51 +00:00
Charlie Marsh
c485727e00
Remove duplicated lockfile invalidation logs (#6340)
## Summary

These got moved to the caller, so they're all duplicated right now.
2024-08-21 15:29:33 +00:00
Charlie Marsh
d627dea51e
Preserve Git username for SSH dependencies (#6335)
## Summary

We're gonna work on a more comprehensive review of whether we should
preserve the username here, but for now, `git@` is effectively a
convention for GitHub and GitLab etc.

Closes https://github.com/astral-sh/uv/issues/6305.

## Test Plan

I guess we don't have infrastructure for testing SSH private keys right
now, but...

```
❯ cargo run init foo
❯ cd foo
❯ cargo run add git+ssh://git@github.com/astral-sh/mkdocs-material-insiders.git
```
2024-08-21 11:22:45 -04:00
Charlie Marsh
2e02d579a0
Invalidate uv.lock when virtual dev-dependencies change (#6291)
## Summary

For non-virtual workspaces, these are covered by the _members_. But for
virtual workspaces, they aren't captured anywhere else in the lock. So,
we weren't invalidating `uv.lock` when the dev dependencies changed,
which led to a panic.

Closes https://github.com/astral-sh/uv/issues/6288
2024-08-21 01:25:38 +00:00
Andrew Gallant
33480d61eb switch to jiff from chrono (#6205)
This PR migrates uv's use of `chrono` to `jiff`.

I did most of this work a while back as one of my tests to ensure Jiff
could actually be used in a real world project. I decided to revive
this because I noticed that `reqwest-retry` dropped its Chrono
dependency,
which is I believe the only other thing requiring Chrono in uv.
(Although, we use a fork of `reqwest-middleware` at present, and that
hasn't been updated to latest upstream yet. I wasn't quite sure of the
process we have for that.)

In course of doing this, I actually made two changes to uv:

First is that the lock file now writes an RFC 3339 timestamp for
`exclude-newer`. Previously, we were using Chrono's `Display`
implementation for this which is a non-standard but "human readable"
format. I think the right thing to do here is an RFC 3339 timestamp.

Second is that, in addition to an RFC 3339 timestamp, `--exclude-newer`
used to accept a "UTC date." But this PR changes it to a "local date."
That is, a date in the user's system configured time zone. I think
this makes more sense than a UTC date, but one alternative is to drop
support for a date and just rely on an RFC 3339 timestamp. The main
motivation here is that automatically assuming UTC is often somewhat
confusing, since just writing an unqualified date like `2024-08-19` is
often assumed to be interpreted relative to the writer's "local" time.
2024-08-20 11:31:46 -05:00
Charlie Marsh
3395d24959
Allow user to constrain supported lock environments (#6210)
## Summary

The strategy here is: if the user provides supported environments, we
use those as the initial forks when resolving. As a result, we never add
or explore branches that are disjoint with the supported environments.
(If the supported environments change, we ignore the lockfile entirely,
so we don't have to worry about any interactions between supported
environments and the preference forks.)

Closes https://github.com/astral-sh/uv/issues/6184.
2024-08-20 13:28:04 +00:00
Charlie Marsh
d02c202eb2
Rename environment-markers to resolution-markers (#6240)
## Summary

I probably won't land https://github.com/astral-sh/uv/pull/6210 for the
release, but I do want to make one breaking change to prep for it:
renaming `environment-markers` to `resolution-markers` (or
`solution-markers`?) so that it's delineated from the user-defined
markers in that PR.
2024-08-20 09:14:55 -04:00
Andrew Gallant
c14e30ad09 pep508: fix doc test
Indented blocks in Markdown are treated as code blocks, and rustdoc
treats all unadorned code blocks as Rust doctests. Since this wasn't
intended as a doctest and isn't valid Rust, it makes `cargo test --doc`
fail. We fix this by using an explicit code block labeled as `text`.
2024-08-19 16:29:20 -07:00