ba73231164
This is a minimal redux of #10861 to be compatible with `uv pip`. This implements the interface described in: https://github.com/pypa/pip/pull/13065#issuecomment-2544000876 for `uv pip install` and `uv pip compile`. Namely `--group <[path:]name>`, where `path` when not defined defaults to `pyproject.toml`. In that interface they add `--group` to `pip install`, `pip download`, and `pip wheel`. Notably we do not define `uv pip download` and `uv pip wheel`, so for parity we only need to implement `uv pip install`. However, we also support `uv pip compile` which is not part of pip itself, and `--group` makes sense there too. ---- The behaviour of `--group` for `uv pip` commands makes sense for the cases upstream pip supports, but has confusing meanings in cases that only we support (because reading pyproject.tomls is New Tech to them but heavily supported by us). **Specifically case (h) below is a concerning footgun, and case (e) below may get complaints from people who aren't well-versed in dependency-groups-as-they-pertain-to-wheels.** ## Only Group Flags Group flags on their own work reasonably and uncontroversially, except perhaps that they don't do very clever automatic project discovery. a) `uv pip install --group path/to/pyproject.toml:mygroup` pulls up `path/to/project.toml` and installs all the packages listed by its `mygroup` dependency-group (essentially treating it like another kind of requirements.txt). In this regard it functions similarly to `--only-group` in the rest of uv's interface. b) `uv pip install --group mygroup` is just sugar for `uv pip install --group pyproject.toml:mygroup` (**note that no project discovery occurs**, upstream pip simply hardcodes the path "pyproject.toml" here and we reproduce that.) c) `uv pip install --group a/pyproject.toml:groupx --group b/pyproject.toml:groupy`, and any other instance of multiple `--group` flags, can be understood as completely independent requests for the given groups at the given files. ## Groups With Named Packages Groups being mixed with named packages also work in a fairly unsurprising way, especially if you understand that things like dependency-groups are not really supposed to exist on pypi, they're just for local development. d) `uv pip install mypackage --group path/to/pyproject.toml:mygroup` much like multiple instances of `--group` the two requests here are essentially completely independent: pleases install `mypackage`, and please also install `path/to/pyproject.toml:mygroup`. e) `uv pip install mypackage --group mygroup` is exactly the same, but this is where it becomes possible for someone to be a little confused, as you might think `mygroup` is supposed to refer to `mypackage` in some way (it can't). But no, it's sourcing `pyproject.toml:mygroup` from the current working directory. ## Groups With Requirements/Sourcetrees/Editables Requirements and sourcetrees are where I expect users to get confused. It behaves *exactly* the same as it does in the previous sections but you would absolutely be forgiven for expecting a different behaviour. *Especially* because `--group` with the rest of uv *does* do something different. f) `uv pip install -r a/pyproject.toml --group b/pyproject.toml:mygroup` is again just two independent requests (install `a/pyproject.toml`'s dependencies, and `b/pyproject.toml`'s `mygroup`). g) `uv pip install -r pyproject.toml --group mygroup` is exactly like the previous case but *incidentally* the two requests refer to the same file. What the user wanted to happen is almost certainly happening, but they are likely getting "lucky" here that they're requesting something simple. h) `uv pip install -r a/pyproject.toml --group mygroup` is again exactly the same but the user is likely to get surprised and upset as this invocation actually sources two different files (install `a/pyproject.toml`'s dependencies, and `pyproject.toml`'s `mygroup`)! I would expect most people to assume the `--group` flag here is covering all applicable requirements/sourcetrees/editables, but no, it continues to be a totally independent reference to a file with a hardcoded relative path. ------ Fixes https://github.com/astral-sh/uv/issues/8590 Fixes https://github.com/astral-sh/uv/issues/8969
7.2 KiB
Locking environments
Locking is to take a dependency, e.g., ruff, and write an exact version to use to a file. When
working with many dependencies, it is useful to lock the exact versions so the environment can be
reproduced. Without locking, the versions of dependencies could change over time, when using a
different tool, or across platforms.
Locking requirements
uv allows dependencies to be locked in the requirements.txt format. It is recommended to use the
standard pyproject.toml to define dependencies, but other dependency formats are supported as
well. See the documentation on declaring dependencies for more details on how to
define dependencies.
To lock dependencies declared in a pyproject.toml:
$ uv pip compile pyproject.toml -o requirements.txt
Note by default the uv pip compile output is just displayed and --output-file / -o argument is
needed to write to a file.
To lock dependencies declared in a requirements.in:
$ uv pip compile requirements.in -o requirements.txt
To lock dependencies declared in multiple files:
$ uv pip compile pyproject.toml requirements-dev.in -o requirements-dev.txt
uv also supports legacy setup.py and setup.cfg formats. To lock dependencies declared in a
setup.py:
$ uv pip compile setup.py -o requirements.txt
To lock dependencies from stdin, use -:
$ echo "ruff" | uv pip compile -
To lock with optional dependencies enabled, e.g., the "foo" extra:
$ uv pip compile pyproject.toml --extra foo
To lock with all optional dependencies enabled:
$ uv pip compile pyproject.toml --all-extras
Note extras are not supported with the requirements.in format.
To lock a dependency group in the current project directory's pyproject.toml, for example the
group foo:
$ uv pip compile --group foo
!!! important
A `--group` flag has to be added to pip-tools' `pip compile`, [although they're considering it](https://github.com/jazzband/pip-tools/issues/2062). We expect to support whatever syntax and semantics they adopt.
To specify the project directory where groups should be sourced from:
$ uv pip compile --project some/path/ --group foo --group bar
Alternatively, you can specify a path to a pyproject.toml for each group:
$ uv pip compile --group some/path/pyproject.toml:foo --group other/pyproject.toml:bar
!!! note
`--group` flags do not apply to other specified sources. For instance,
`uv pip compile some/path/pyproject.toml --group foo` sources `foo`
from `./pyproject.toml` and **not** `some/path/pyproject.toml`.
Upgrading requirements
When using an output file, uv will consider the versions pinned in an existing output file. If a dependency is pinned it will not be upgraded on a subsequent compile run. For example:
$ echo "ruff==0.3.0" > requirements.txt
$ echo "ruff" | uv pip compile - -o requirements.txt
# This file was autogenerated by uv via the following command:
# uv pip compile - -o requirements.txt
ruff==0.3.0
To upgrade a dependency, use the --upgrade-package flag:
$ uv pip compile - -o requirements.txt --upgrade-package ruff
To upgrade all dependencies, there is an --upgrade flag.
Syncing an environment
Dependencies can be installed directly from their definition files or from compiled
requirements.txt files with uv pip install. See the documentation on
installing packages from files for more details.
When installing with uv pip install, packages that are already installed will not be removed
unless they conflict with the lockfile. This means that the environment can have dependencies that
aren't declared in the lockfile, which isn't great for reproducibility. To ensure the environment
exactly matches the lockfile, use uv pip sync instead.
To sync an environment with a requirements.txt file:
$ uv pip sync requirements.txt
To sync an environment with a pyproject.toml file:
$ uv pip sync pyproject.toml
Adding constraints
Constraints files are requirements.txt-like files that only control the version of a requirement
that's installed. However, including a package in a constraints file will not trigger the
installation of that package. Constraints can be used to add bounds to dependencies that are not
dependencies of the current project.
To define a constraint, define a bound for a package:
pydantic<2.0
To use a constraints file:
$ uv pip compile requirements.in --constraint constraints.txt
Note that multiple constraints can be defined in each file and multiple files can be used.
uv will also read constraint-dependencies from the pyproject.toml at the workspace root, and
append them to those specified in the constraints file.
Adding build constraints
Similar to constraints, but specifically for build-time dependencies, including those required
when building runtime dependencies.
Build constraint files are requirements.txt-like files that only control the version of a
build-time requirement. However, including a package in a build constraints file will not trigger
its installation at build time; instead, constraints apply only when the package is required as a
direct or transitive build-time dependency. Build constraints can be used to add bounds to
dependencies that are not explicitly declared as build-time dependencies of the current project.
For example, if a package defines its build dependencies as follows:
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
Build constraints could be used to ensure that a specific version of setuptools is used for every
package in the workspace:
setuptools==75.0.0
uv will also read build-constraint-dependencies from the pyproject.toml at the workspace root,
and append them to those specified in the build constraints file.
Overriding dependency versions
Overrides files are requirements.txt-like files that force a specific version of a requirement to
be installed, regardless of the requirements declared by any constituent package, and regardless of
whether this would be considered an invalid resolution.
While constraints are additive, in that they're combined with the requirements of the constituent packages, overrides are absolute, in that they completely replace the requirements of the constituent packages.
Overrides are most often used to remove upper bounds from a transitive dependency. For example, if
a requires c>=1.0,<2.0 and b requires c>=2.0 and the current project requires a and b
then the dependencies cannot be resolved.
To define an override, define the new requirement for the problematic package:
c>=2.0
To use an overrides file:
$ uv pip compile requirements.in --override overrides.txt
Now, resolution can succeed. However, note that if a is correct that it does not support
c>=2.0 then a runtime error will likely be encountered when using the packages.
Note that multiple overrides can be defined in each file and multiple files can be used.