diff --git a/docs/concepts/resolution.md b/docs/concepts/resolution.md index 08c0e4b70..95fbcfd89 100644 --- a/docs/concepts/resolution.md +++ b/docs/concepts/resolution.md @@ -1,50 +1,126 @@ # Resolution -Dependency resolution is the process of taking your requirements and converting them to a list of -package versions that fulfil your requirements and the requirements of all included packages. +Resolution is the process of taking a list of requirements and converting them to a list of package +versions that fulfill the requirements. Resolution requires recursively searching for compatible +versions of packages, ensuring that the requested requirements are fulfilled and that the +requirements of the requested packages are compatible. -## Overview +## Dependencies -Imagine you have the following dependency tree: +Most projects and packages have dependencies. Dependencies are other packages that are needed in +order for the current package to work. A package defines its dependencies as _requirements_, roughly +a combination of a package name and acceptable versions. The dependencies defined by the current +project are called _direct dependencies_. The requirements added by each dependency of the current +project are called _indirect_ or _transitive dependencies_. -- Your project depends on `foo>=1,<3` and `bar>=1,<3`. -- `foo` has two versions, 1.0.0 and 2.0.0. `foo` 2.0.0 depends on `lib==2.0.0`, `foo` 1.0.0 has no - dependencies. -- `bar` has two versions, 1.0.0 and 2.0.0. `bar` 2.0.0 depends on `lib==1.0.0`, `bar` 1.0.0 has no - dependencies. +!!! note + + See the [dependency specifiers + page](https://packaging.python.org/en/latest/specifications/dependency-specifiers/) + in the Python Packaging documentation for details about dependencies. + +## Basic examples + +To help demonstrate the resolution process, consider the following dependencies: + + +- The project depends on `foo` and `bar`. +- `foo` has one version, 1.0.0: + - `foo 1.0.0` depends on `lib>=1.0.0`. +- `bar` has one version, 1.0.0: + - `bar 1.0.0` depends on `lib>=2.0.0`. - `lib` has two versions, 1.0.0 and 2.0.0. Both versions have no dependencies. -We can't install both `foo` 2.0.0 and `bar` 2.0.0 because they conflict on the version of `lib`, so -the resolver will pick either `foo` 1.0.0 or `bar` 1.0.0. Both are valid solutions, at the resolvers -choice. +In this example, the resolver must find a set of package versions which satisfies the project +requirements. Since there is only one version of both `foo` and `bar`, those will be used. The +resolution must also include the transitive dependencies, so a version of `lib` must be chosen. +`foo 1.0.0` allows all of the available versions of `lib`, but `bar 1.0.0` requires `lib>=2.0.0` so +`lib 2.0.0` must be used. -## Platform-specific and universal resolution +In some resolutions, there is more than one solution. Consider the following dependencies: -uv supports two modes of resolution: Platform-specific and universal (platform-independent). + +- The project depends on `foo` and `bar`. +- `foo` has two versions, 1.0.0 and 2.0.0: + - `foo 1.0.0` has no dependencies. + - `foo 2.0.0` depends on `lib==2.0.0`. +- `bar` has two versions, 1.0.0 and 2.0.0: + - `bar 1.0.0` has no dependencies. + - `bar 2.0.0` depends on `lib==1.0.0` +- `lib` has two versions, 1.0.0 and 2.0.0. Both versions have no dependencies. -Like `pip` and `pip-tools`, `uv pip compile` produces a resolution that's only known to be -compatible with the current operating system, architecture, Python version and Python interpreter. -`uv pip compile --universal` and the [project](../guides/projects.md) interface on the other hand -will solve to a host-agnostic universal resolution that can be used across platforms. +In this example, some version of both `foo` and `bar` must be picked, however, determining which +version requires considering the dependencies of each version of `foo` and `bar`. `foo 2.0.0` and +`bar 2.0.0` cannot be installed together because they conflict on their required version of `lib`, +so the resolver must select either `foo 1.0.0` or `bar 1.0.0`. Both are valid solutions, and +different resolution algorithms may give either result. -For universal resolution, you need to configure the minimum required python version. For -`uv pip compile --universal`, you can pass `--python-version`, otherwise the current Python version +## Platform markers + +Markers allow attaching an expression to requirements that indicate when the dependency should be +used. For example `bar; python_version<"3.9"` can be used to only require `bar` on Python 3.8 and +older. + +Markers are used to adjust a package's dependencies dependending on the current environment or +platform. For example, markers can be used to change dependencies based on the operating system, the +CPU architecture, the Python version, the Python implementation, and more. + +!!! note + + See the [environment + markers](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers) + section in the Python Packaging documentation for more details about markers. + +Markers are important for resolution because their values change the required dependencies. +Typically, Python package resolvers use the markers of the _current_ platform to determine which +dependencies to use since the package is often being _installed_ on the current platform. However, +for _locking_ dependencies this is problematic — the lockfile would only work for developers using +the same platform the lockfile was created on. To solve this problem, platform-independent, or +"universal" resolvers exist. + +uv supports both [platform-specific](#platform-specific-resolution) and +[universal](#universal-resolution) resolution. + +## Universal resolution + +uv's lockfile (`uv.lock`) is created with a universal resolution and is portable across platforms. +This ensures that dependencies are locked for everyone working on the project, regardless of +operating system, architecture, and Python version. The uv lockfile is created and modified by +[project](../concepts/projects.md) commands such as `uv lock`, `uv sync`, and `uv add`. + +universal resolution is also available in uv's pip interface, i.e., +[`uv pip compile`](../pip/compile.md), with the `--universal` flag. The resulting requirements file +will contain markers to indicate which platform each dependency is relevant for. + +During universal resolution, a package may be listed multiple times with different versions or URLs +if different versions are needed for different platforms — the markers determine which version will +be used. A universal resolution is often more constrained than a platform-specific resolution, since +we need to take the requirements for all markers into account. + +During universal resolution, a minimum Python version must be specified. Project commands read the +minimum required version from `project.requires-python` in the `pyproject.toml`. When using the pip +interface, provide a value with the `--python-version` option, otherwise the current Python version will be treated as a lower bound. For example, `--universal --python-version 3.9` writes a universal -resolution for Python 3.9 and later. Project commands such as `uv sync` or `uv lock` read -`project.requires-python` from your `pyproject.toml`. +resolution for Python 3.9 and later. Setting the minimum Python version is important because all package versions we select have to be -compatible with the python range. For example, a universal resolution of `numpy<2` with +compatible with the Python version range. For example, a universal resolution of `numpy<2` with `--python-version 3.8` resolves to `numpy==1.24.4`, while `--python-version 3.9` resolves to -`numpy==1.26.4`, as `numpy` releases after 1.26.4 require at least Python 3.9. Note that we only -consider the lower bound of any Python requirement. +`numpy==1.26.4`, as `numpy` releases after 1.26.4 require at Python 3.9+. Note that we only consider +the lower bound of any Python requirement, upper bounds are always ignored. -In platform-specific mode, the `uv pip` interface also supports resolving for specific alternate -platforms and Python versions with `--python-platform` and `--python-version`. For example, if -you're running Python 3.12 on macOS, but want to resolve for Linux with Python 3.10, you can run -`uv pip compile --python-platform linux --python-version 3.10 requirements.in` to produce a -`manylinux2014`-compatible resolution. In this mode, `--python-version` is the exact python version -to use, not a lower bound. +## Platform-specific resolution + +By default, uv's pip interface, i.e., [`uv pip compile`](../pip/compile.md), produces a resolution +that is platform-specific, like `pip-tools`. There is no way to use platform-specific resolution in +the uv's project interface. + +uv also supports resolving for specific, alternate platforms and Python versions with the +`--python-platform` and `--python-version` options. For example, if using Python 3.12 on macOS, +`uv pip compile --python-platform linux --python-version 3.10 requirements.in` can be used to +produce a resolution for Python 3.10 on Linux instead. Unlike universal resolution, during +platform-specific resolution, the provided `--python-version` is the exact python version to use, +not a lower bound. !!! note @@ -55,34 +131,28 @@ to use, not a lower bound. compatible with any machine running on the target `--python-platform`, which should be sufficient for most use cases, but may lose fidelity for complex package and platform combinations. -In universal mode, a package may be listed multiple times with different versions or URLs. In this -case, uv determined that we need different versions to be compatible different platforms, and the -markers decides on which platform we use which version. A universal resolution is often more -constrained than a platform-specific resolution, since we need to take the requirements for all -markers into account. +## Dependency preferences -If an output file is used with `uv pip` or `uv.lock` exist with the project commands, we try to -resolve to the versions present there, considering them preferences in the resolution. The same -applies to version already installed to the active virtual environments. You can override this with -`--upgrade`. +If resolution output file exists, i.e. a uv lockfile (`uv.lock`) or a requirements output file +(`requirements.txt`), uv will _prefer_ the dependency versions listed there. Similarly, if +installing a package into a virtual environment, uv will prefer the already installed version if +present. This means that locked or installed versions will not change unless an incompatible version +is requested or an upgrade is explicitly requested with `--upgrade`. ## Resolution strategy By default, uv tries to use the latest version of each package. For example, -`uv pip install flask>=2.0.0` will install the latest version of Flask (at time of writing: -`3.0.0`). If you have `flask>=2.0.0` as a dependency of your library, you will only test `flask` -3.0.0 this way, but not if you are actually still compatible with `flask` 2.0.0. +`uv pip install flask>=2.0.0` will install the latest version of Flask, e.g., 3.0.0. If +`flask>=2.0.0` is a dependency of the project, only `flask` 3.0.0 will be used. This is important, +for example, because running tests will not check that the the project is actually compatible with +its stated lower bound of `flask` 2.0.0. With `--resolution lowest`, uv will install the lowest possible version for all dependencies, both -direct and indirect (transitive). Alternatively, `--resolution lowest-direct` will opt for the -lowest compatible versions for all direct dependencies, while using the latest compatible versions -for all other dependencies. uv will always use the latest versions for build dependencies. +direct and indirect (transitive). Alternatively, `--resolution lowest-direct` will use the lowest +compatible versions for all direct dependencies, while using the latest compatible versions for all +other dependencies. uv will always use the latest versions for build dependencies. -For libraries, we recommend separately running tests with `--resolution lowest` or -`--resolution lowest-direct` in continuous integration to ensure compatibility with the declared -lower bounds. - -As an example, given the following `requirements.in` file: +For example, given the following `requirements.in` file: ```text title="requirements.in" flask>=2.0.0 @@ -128,6 +198,10 @@ werkzeug==2.0.0 # via flask ``` +When publishing libraries, it is recommended to separately run tests with `--resolution lowest` or +`--resolution lowest-direct` in continuous integration to ensure compatibility with the declared +lower bounds. + ## Pre-release handling By default, uv will accept pre-release versions during dependency resolution in two cases: @@ -136,46 +210,53 @@ By default, uv will accept pre-release versions during dependency resolution in (e.g., `flask>=2.0.0rc1`). 1. If _all_ published versions of a package are pre-releases. -If dependency resolution fails due to a transitive pre-release, uv will prompt the user to re-run -with `--prerelease allow`, to allow pre-releases for all dependencies. +If dependency resolution fails due to a transitive pre-release, uv will prompt use of +`--prerelease allow` to allow pre-releases for all dependencies. -Alternatively, you can add the transitive dependency to your `requirements.in` file with a -pre-release specifier (e.g., `flask>=2.0.0rc1`) to opt in to pre-release support for that specific -dependency. +Alternatively, the transitive dependency can be added as a [constraint](#dependency-constraints) or +direct dependency (i.e. in `requirements.in` or `pyproject.toml`) with a pre-release version +specifier (e.g., `flask>=2.0.0rc1`) to opt-in to pre-release support for that specific dependency. Pre-releases are [notoriously difficult](https://pubgrub-rs-guide.netlify.app/limitations/prerelease_versions) to model, and are a frequent source of bugs in other packaging tools. uv's pre-release handling is -_intentionally_ limited and _intentionally_ requires user opt-in for pre-releases, to ensure -correctness. +_intentionally_ limited and requires user opt-in for pre-releases to ensure correctness. -For more, see [Pre-release compatibility](../pip/compatibility.md#pre-release-compatibility). +For more details, see +[Pre-release compatibility](../pip/compatibility.md#pre-release-compatibility). -## Constraints +## Dependency constraints -Like `pip`, uv supports constraints files (`--constraint constraints.txt`), which allows users to -narrow the set of acceptable versions for a given package. A constraint files is like a regular -requirements files, but it doesn't add packages, it only constrains their version range when they -are depended on by a regular requirement. +uv supports constraints files (`--constraint constraints.txt`), like pip, which narrow the set of +acceptable versions for the given packages. Constraint files are like a regular requirements files, +but they do not add packages to the requirements — they only take affect if the package is requested +in a direct or transitive dependency. Constraints are often useful for reducing the range of +available versions for a transitive dependency without adding a direct requirement on the package. -## Overrides +## Dependency overrides -Sometimes, the requirements in one of your (transitive) dependencies are too strict, and you want to -install a version of a package that you know to work, but wouldn't be allowed regularly. Overrides -allow you to lie to the resolver, replacing all other requirements for that package with the -override. They break the usual rules of version resolving and should only be used as last resort -measure. +Sometimes, the requirements defined by a dependency are too strict, and a working version of a +package is not allowed (often, causing resolution to fail). uv allows overriding requirements +defined by dependencies to unblock resolution. -For example, if a transitive dependency declares `pydantic>=1.0,<2.0`, but the user knows that the -package is compatible with `pydantic>=2.0`, the user can override the declared dependency with -`pydantic>=2.0,<3` to allow the resolver to continue. +For example, if a transitive dependency declares the requirement `pydantic>=1.0,<2.0`, but _works_ +with `pydantic>=2.0`, the user can override the declared dependency with `pydantic>=1.0,<3` to allow +the resolver to installer a newer version of `pydantic`. -Overrides are passed to `uv pip` as `--override` with an overrides file with the same syntax as -requirements or constraints files. In `pyproject.toml`, you can set `tool.uv.override-dependencies` -to a list of requirements. If you provide multiple overrides for the same package, we apply them -simultaneously, while markers are applied as usual. +Constraints and direct dependency declarations will _not_ result in the same behavior as an +override, as they can only add _additional_ constraints to the allowed version instead of +_replacing_ existing constraints. As with constraints, overrides do not _add_ a dependency on the +package and only take affect if the package is requested in a direct or transitive dependency. -## Time-restricted reproducible resolutions +In a `pyproject.toml`, use `tool.uv.override-dependencies` to define a list of overrides. In the +pip-compatible interface, the `--override` option can be used to pass files with the same format as +constraints files. + +If multiple overrides are provided for the same package, they must be differentiated with +[markers](#platform-markers). If a package has a dependency with a marker, it is replaced +unconditionally when using overrides — it does not matter if the marker evaluates to true or false. + +## Reproducible resolutions uv supports an `--exclude-newer` option to limit resolution to distributions published before a specific date, allowing reproduction of installations regardless of new package releases. The date