mirrors/uv
1
0
Fork 0
mirror of https://github.com/astral-sh/uv.git synced 2025-11-01 20:31:12 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
uv/crates
History
konsti d9dbb8a4af
Support conflicting URL in separate forks (#4435) 
Downstack PR: #4481

## Introduction

We support forking the dependency resolution to support conflicting
registry requirements for different platforms, say on package range is
required for an older python version while a newer is required for newer
python versions, or dependencies that are different per platform. We
need to extend this support to direct URL requirements.

```toml
dependencies = [
  "iniconfig @ 62565a6e1c/iniconfig-2.0.0-py3-none-any.whl ; python_version >= '3.12'",
  "iniconfig @ b3c12c6d70/iniconfig-1.1.1-py2.py3-none-any.whl ; python_version < '3.12'"
]
```

This did not work because `Urls` was built on the assumption that there
is a single allowed URL per package. We collect all allowed URL ahead of
resolution by following direct URL dependencies (including path
dependencies) transitively, i.e. a registry distribution can't require a
URL.

## The same package can have Registry and URL requirements

Consider the following two cases:

requirements.in:
```text
werkzeug==2.0.0
werkzeug @ 960bb4017c/Werkzeug-2.0.0-py3-none-any.whl
```
pyproject.toml:
```toml
dependencies = [
  "iniconfig == 1.1.1 ; python_version < '3.12'",
  "iniconfig @ git+https://github.com/pytest-dev/iniconfig@93f5930e668c0d1ddf4597e38dd0dea4e2665e7a ; python_version >= '3.12'",
]
```

In the first case, we want the URL to override the registry dependency,
in the second case we want to fork and have one branch use the registry
and the other the URL. We have to know about this in
`PubGrubRequirement::from_registry_requirement`, but we only fork after
the current method.

Consider the following case too:

a:
```
c==1.0.0
b @ https://b.zip
```
b:
```
c @ https://c_new.zip ; python_version >= '3.12'",
c @ https://c_old.zip ; python_version < '3.12'",
```

When we convert the requirements of `a`, we can't know the url of `c`
yet. The solution is to remove the `Url` from `PubGrubPackage`: The
`Url` is redundant with `PackageName`, there can be only one url per
package name per fork. We now do the following: We track the urls from
requirements in `PubGrubDependency`. After forking, we call
`add_package_version_dependencies` where we apply override URLs, check
if the URL is allowed and check if the url is unique in this fork. When
we request a distribution, we ask the fork urls for the real URL. Since
we prioritize url dependencies over registry dependencies and skip
packages with `Urls` entries in pre-visiting, we know that when fetching
a package, we know if it has a url or not.

## URL conflicts

pyproject.toml (invalid):
```toml
dependencies = [
  "iniconfig @ e96292c7f7/iniconfig-1.1.0.tar.gz",
  "iniconfig @ b3c12c6d70/iniconfig-1.1.1-py2.py3-none-any.whl ; python_version < '3.12'",
  "iniconfig @ 62565a6e1c/iniconfig-2.0.0-py3-none-any.whl ; python_version >= '3.12'",
]
```

On the fork state, we keep `ForkUrls` that check for conflicts after
forking, rejecting the third case because we added two packages of the
same name with different URLs.

We need to flatten out the requirements before transformation into
pubgrub requirements to get the full list of other requirements which
may contain a URL, which was changed in a previous PR: #4430.

## Complex Example

a:
```toml
dependencies = [
  # Force a split
  "anyio==4.3.0 ; python_version >= '3.12'",
  "anyio==4.2.0 ; python_version < '3.12'",
  # Include URLs transitively
  "b"
]
```
b:
```toml
dependencies = [
  # Only one is used in each split.
  "b1 ; python_version < '3.12'",
  "b2 ; python_version >= '3.12'",
  "b3 ; python_version >= '3.12'",
]
```
b1:
```toml
dependencies = [
  "iniconfig @ b3c12c6d70/iniconfig-1.1.1-py2.py3-none-any.whl",
]
```
b2:
```toml
dependencies = [
  "iniconfig @ 62565a6e1c/iniconfig-2.0.0-py3-none-any.whl",
]
```
b3:
```toml
dependencies = [
  "iniconfig @ e96292c7f7/iniconfig-1.1.0.tar.gz",
]
```

In this example, all packages are url requirements (directory
requirements) and the root package is `a`. We first split on `a`, `b`
being in each split. In the first fork, we reach `b1`, the fork URLs are
empty, we insert the iniconfig 1.1.1 URL, and then we skip over `b2` and
`b3` since the mark is disjoint with the fork markers. In the second
fork, we skip over `b1`, visit `b2`, insert the iniconfig 2.0.0 URL into
the again empty fork URLs, then visit `b3` and try to insert the
iniconfig 1.1.0 URL. At this point we find a conflict for the iniconfig
URL and error.

## Closing

The git tests are slow, but they make the best example for different URL
types i could find.

Part of #3927. This PR does not handle `Locals` or pre-releases yet.
2024-06-26 13:58:23 +02:00
..
bench Respect index strategy in source distribution builds (#4468) 2024-06-24 12:03:38 +00:00
cache-key Remove some unused pub use exports (#3930) 2024-05-30 22:26:52 -04:00
distribution-filename Make .egg-info filename parsing spec compliant (#4533) 2024-06-25 23:49:43 +00:00
distribution-types Make .egg-info filename parsing spec compliant (#4533) 2024-06-25 23:49:43 +00:00
install-wheel-rs Remove useless #[allow(clippy::too_many_arguments)] (#4529) 2024-06-25 19:09:59 +00:00
once-map Avoid race condition in OnceMap (#3987) 2024-06-03 12:25:58 -04:00
pep440-rs Enable workspace lint configuration in remaining crates (#4329) 2024-06-18 03:02:28 +00:00
pep508-rs Support conflicting URL in separate forks (#4435) 2024-06-26 13:58:23 +02:00
platform-tags fix: add missing ppc64le alias for powerpc64le (#3963) 2024-06-02 13:15:24 -04:00
pypi-types Support conflicting URL in separate forks (#4435) 2024-06-26 13:58:23 +02:00
requirements-txt Allow local index references in requirements.txt files (#4525) 2024-06-25 18:06:37 +00:00
uv Support conflicting URL in separate forks (#4435) 2024-06-26 13:58:23 +02:00
uv-auth Enable workspace lint configuration in remaining crates (#4329) 2024-06-18 03:02:28 +00:00
uv-build Create temporary environments in dedicated cache bucket (#4223) 2024-06-10 23:17:35 +00:00
uv-cache Represent build tag as u64 (#4253) 2024-06-11 21:40:08 +00:00
uv-cli Add a universal resolution mode to pip compile (#4505) 2024-06-25 21:28:50 +00:00
uv-client Avoid panic for invalid, non-base index URLs (#4527) 2024-06-25 18:32:58 +00:00
uv-configuration Flatten requirements eagerly in get_dependencies (#4430) 2024-06-25 21:13:47 +00:00
uv-dev Respect index strategy in source distribution builds (#4468) 2024-06-24 12:03:38 +00:00
uv-dispatch Rename SitePackages::from_environment for clarity (#4497) 2024-06-24 23:32:45 +00:00
uv-distribution Use shared client in Git fetch implementation (#4487) 2024-06-24 17:09:29 -04:00
uv-extract Fix nightly cfg checker warnings (#3932) 2024-05-31 09:35:52 +00:00
uv-fs Add guard to replace_symlink on Windows (#4519) 2024-06-25 13:47:40 -05:00
uv-git Use shared client in Git fetch implementation (#4487) 2024-06-24 17:09:29 -04:00
uv-installer Rename SitePackages::from_environment for clarity (#4497) 2024-06-24 23:32:45 +00:00
uv-macros Enable workspace lint configuration in remaining crates (#4329) 2024-06-18 03:02:28 +00:00
uv-normalize Enable workspace lint configuration in remaining crates (#4329) 2024-06-18 03:02:28 +00:00
uv-requirements Remove useless #[allow(clippy::too_many_arguments)] (#4529) 2024-06-25 19:09:59 +00:00
uv-resolver Support conflicting URL in separate forks (#4435) 2024-06-26 13:58:23 +02:00
uv-settings Add a universal resolution mode to pip compile (#4505) 2024-06-25 21:28:50 +00:00
uv-state Add persistent storage of installed toolchains (#3797) 2024-05-27 03:54:49 +00:00
uv-toolchain More precise locking with --prefix option (#4506) 2024-06-25 06:47:52 -04:00
uv-trampoline Support x86 windows (#3873) 2024-05-28 16:07:39 +00:00
uv-types Respect index strategy in source distribution builds (#4468) 2024-06-24 12:03:38 +00:00
uv-version Bump version to 0.2.15 (#4475) 2024-06-24 10:04:09 -05:00
uv-virtualenv Use relative path for lib64 symlink (#4268) 2024-06-12 09:36:27 -04:00
uv-warnings Fix a bug where no warning is output when parsing of workspace settings fails. (#4014) 2024-06-04 09:21:19 -04:00
README.md Rename uv-interpreter crate to uv-toolchain (#4120) 2024-06-07 13:59:14 -05:00

README.md

Crates

bench

Functionality for benchmarking uv.

cache-key

Generic functionality for caching paths, URLs, and other resources across platforms.

distribution-filename

Parse built distribution (wheel) and source distribution (sdist) filenames to extract structured metadata.

distribution-types

Abstractions for representing built distributions (wheels) and source distributions (sdists), and the sources from which they can be downloaded.

install-wheel-rs

Install built distributions (wheels) into a virtual environment.]

once-map

A waitmap-like concurrent hash map for executing tasks exactly once.

pep440-rs

Utilities for interacting with Python version numbers and specifiers.

pep508-rs

Utilities for interacting with PEP 508 dependency specifiers.

platform-host

Functionality for detecting the current platform (operating system, architecture, etc.).

platform-tags

Functionality for parsing and inferring Python platform tags as per PEP 425.

uv

Command-line interface for the uv package manager.

uv-build

A PEP 517-compatible build frontend for uv.

uv-cache

Functionality for caching Python packages and associated metadata.

uv-client

Client for interacting with PyPI-compatible HTTP APIs.

uv-dev

Development utilities for uv.

uv-dispatch

A centralized struct for resolving and building source distributions in isolated environments. Implements the traits defined in uv-types.

uv-distribution

Client for interacting with built distributions (wheels) and source distributions (sdists). Capable of fetching metadata, distribution contents, etc.

uv-extract

Utilities for extracting files from archives.

uv-fs

Utilities for interacting with the filesystem.

uv-git

Functionality for interacting with Git repositories.

uv-installer

Functionality for installing Python packages into a virtual environment.

uv-toolchain

Functionality for detecting and leveraging the current Python interpreter.

uv-normalize

Normalize package and extra names as per Python specifications.

uv-package

Types and functionality for working with Python packages, e.g., parsing wheel files.

uv-requirements

Utilities for reading package requirements from pyproject.toml and requirements.txt files.

uv-resolver

Functionality for resolving Python packages and their dependencies.

uv-types

Shared traits for uv, to avoid circular dependencies.

pypi-types

General-purpose type definitions for types used in PyPI-compatible APIs.

uv-virtualenv

A venv replacement to create virtual environments in Rust.

uv-warnings

User-facing warnings for uv.

requirements-txt

Functionality for parsing requirements.txt files.