Fixes#11793
On Windows, trying to read a file inside what is not a directory but
another file results in a not found error, while on Unix we get a not a
directory error. We check explicitly if something included in a
workspace glob is a non-directory to fix the behavior on Windows.
## Summary
Resolves#6913.
Add `tool.uv.build-constraint-dependencies` to pyproject.toml.
The changes are analogous to the constraint-dependencies feature
implemented in #5248.
Add documentation for `build-constraint-dependencies`
## Test Plan
Add tests for `uv lock`, `uv add`, `uv pip install` and `uv pip
compile`.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This PR revives https://github.com/astral-sh/uv/pull/10017, which might
be viable now that we _don't_ enforce any platforms by default.
The basic idea here is that users can mark certain platforms as required
(empty, by default). When resolving, we ensure that the specified
platforms have wheel coverage, backtracking if not.
For example, to require that we include a version of PyTorch that
supports Intel macOS:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = ["torch>1.13"]
[tool.uv]
required-platforms = [
"sys_platform == 'darwin' and platform_machine == 'x86_64'"
]
```
Other than that, the forking is identical to past iterations of this PR.
This would give users a way to resolve the tail of issues in #9711, but
with manual opt-in to supporting specific platforms.
I think `UV_PROJECT_ENVIRONMENT` is too complicated for use-cases where
the user wants to sync to the active environment. I don't see a
compelling reason not to make opt-in easier. I see a lot of questions
about how to deal with this warning in the issue tracker, but it seems
painful to collect them here for posterity.
A notable behavior here — we'll treat this as equivalent to
`UV_PROJECT_ENVIRONMENT` so... if you point us to a valid virtual
environment that needs to be recreated for some reason (e.g., new Python
version request), we'll happily delete it and start over.
## Summary
If members define disjoint Python requirements, we should error. Right
now, it seems that it maps to unbounded and leads to weird behavior.
Closes https://github.com/astral-sh/uv/issues/10835.
## Summary
This PR introduces a `LockTarget`, which is peer to `InstallTarget` and
enables us to capture the common functionality necessary to support
locking.
For now, to minimize changes, only the `Workspace` target is
implemented. In a future PR, I'll add a `Script` target for both locking
and installing.
## Summary
This is necessary for some future improvements to non-`[project]`
workspaces and PEP 723 scripts. It's not "breaking", but it will
invalidate lockfiles for non-`[project]` workspaces. I think that's
okay, since we consider those legacy right now, and they're really rare.
## Summary
Today, our dependency group implementation is a little awkward... For
each package `P`, we check if `P` contains dependencies for each enabled
group, then add a dependency on `P` with the group enabled. There are a
few issues here:
1. It's sort of backwards... We add a dependency from the base package
`P` to `P` with the group enabled. Then `P` with the group enabled adds
a dependency on the base package.
2. We can't, e.g., enable different groups for different packages. (We
don't have a way for users to specify this on the CLI, but there's no
reason that it should be _impossible_ in the resolver.)
3. It's inconsistent with how extras work, which leads to confusing
differences in the resolver.
Instead, our internal requirement type can now include dependency
groups, which makes dependency groups look much, much more like extras
in the resolver.
## Summary
A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.
IIUC, this _does_ require bumping our MSRV.
## Summary
We still only respect overrides and constraints in the workspace root --
which we may want to change -- but overrides and constraints are now
correctly lowered.
Closes https://github.com/astral-sh/uv/issues/8148.
## Summary
This PR enables something like the "final boss" of PyTorch setups --
explicit support for CPU vs. GPU-enabled variants via extras:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = []
[project.optional-dependencies]
cpu = [
"torch==2.5.1+cpu",
]
gpu = [
"torch==2.5.1",
]
[tool.uv.sources]
torch = [
{ index = "torch-cpu", extra = "cpu" },
{ index = "torch-gpu", extra = "gpu" },
]
[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
[tool.uv]
conflicts = [
[
{ extra = "cpu" },
{ extra = "gpu" },
],
]
```
It builds atop the conflicting extras work to allow sources to be marked
as specific to a dedicated extra being enabled or disabled.
As part of this work, sources now have an `extra` field. If a source has
an `extra`, it means that the source is only applied to the requirement
when defined within that optional group. For example, `{ index =
"torch-cpu", extra = "cpu" }` above only applies to
`"torch==2.5.1+cpu"`.
The `extra` field does _not_ mean that the source is "enabled" when the
extra is activated. For example, this wouldn't work:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = ["torch"]
[tool.uv.sources]
torch = [
{ index = "torch-cpu", extra = "cpu" },
{ index = "torch-gpu", extra = "gpu" },
]
[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
```
In this case, the sources would effectively be ignored. Extras are
really confusing... but I think this is correct? We don't want enabling
or disabling extras to affect resolution information that's _outside_ of
the relevant optional group.
## Summary
These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
## Summary
Align uv's workspace discovery with red knots (see
https://github.com/astral-sh/ruff/pull/14308#issuecomment-2474296308)
* Detect nested workspace inside the current workspace rather than
testing if the current workspace is a member of any outer workspace.
* Detect packages with identical names.
## Test Plan
I added two integration tests. I also back ported the tests to main to
verify that both these invalid workspaces aren't catched by uv today.
That makes this, technically, a breaking change but I would consider the
change a bug fix.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Since this is intended to support _both_ groups and extras, it doesn't
make sense to just name it for groups. And since there isn't really a
word that encapsulates both "extra" and "group," we just fall back to
the super general "conflicts."
We'll rename the variables and other things in the next commit.
This PR adds support for conflicting extras. For example, consider
some optional dependencies like this:
```toml
[project.optional-dependencies]
project1 = ["numpy==1.26.3"]
project2 = ["numpy==1.26.4"]
```
These dependency specifications are not compatible with one another.
And if you ask uv to lock these, you'll get an unresolvable error.
With this PR, you can now add this to your `pyproject.toml` to get
around this:
```toml
[tool.uv]
conflicting-groups = [
[
{ package = "project", extra = "project1" },
{ package = "project", extra = "project2" },
],
]
```
This will make the universal resolver create additional forks
internally that keep the dependencies from the `project1` and
`project2` extras separate. And we make all of this work by reporting
an error at **install** time if one tries to install with two or more
extras that have been declared as conflicting. (If we didn't do this,
it would be possible to try and install two different versions of the
same package into the same environment.)
This PR does *not* add support for conflicting **groups**, but it is
intended to add support in a follow-up PR.
Closes#6981Fixes#8024
Ref #6729, Ref #6830
This should also hopefully unblock
https://github.com/dagster-io/dagster/pull/23814, but in my testing, I
did run into other problems (specifically, with `pywin`). But it does
resolve the problem with incompatible dependencies in two different
extras once you declare `test-airflow-1` and `test-airflow-2` as
conflicting for `dagster-airflow`.
NOTE: This PR doesn't make `conflicting-groups` public yet. And in a
follow-up PR, I plan to switch the name to `conflicts` instead of
`conflicting-groups`, since it will be able to accept conflicting extras
_and_ conflicting groups.
Uses #6369 for test coverage.
Updates version file discovery to search up into parent directories.
Also refactors Python request determination to avoid duplicating the
user request / version file / workspace lookup logic in every command
(this supersedes the work started in
https://github.com/astral-sh/uv/pull/6372).
There is a bit of remaining work here, mostly around documentation.
There are some edge-cases where we don't use the refactored request
utility, like `uv build` — I'm not sure how I'm going to handle that yet
as it needs a separate root directory.
## Summary
This PR improves the interaction of `--frozen` such that we reduce the
dependency on the `pyproject.toml` and increase the dependency on the
`uv.lock`. Specifically, we now read the list of workspace members from
the `uv.lock` rather than the `pyproject.toml`, which means we don't
need to discover the member `pyproject.toml` files in order to perform a
`uv sync --frozen --all-packages`.
## Summary
This PR enables `uv sync --all-packages` to sync all packages in a
workspace. It removes a common use-case for the legacy non-`[project]`
packages that we're trying to move away from.
Closes https://github.com/astral-sh/uv/issues/8724.
## Summary
It turns out that when locking, we were only taking the groups from the
root `pyproject.toml` into account, and ignoring groups that were only
defined in a workspace member.
## Summary
We already support `tool.uv.dev-dependencies` in the legacy
non-`[project]` projects. This adds equivalent support for
`[dependency-groups]`, e.g.:
```toml
[tool.uv.workspace]
[dependency-groups]
lint = ["ruff"]
```
## Summary
This PR adds a first-class API for defining registry indexes, beyond our
existing `--index-url` and `--extra-index-url` setup.
Specifically, you now define indexes like so in a `uv.toml` or
`pyproject.toml` file:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
You can also provide indexes via `--index` and `UV_INDEX`, and override
the default index with `--default-index` and `UV_DEFAULT_INDEX`.
### Index priority
Indexes are prioritized in the order in which they're defined, such that
the first-defined index has highest priority.
Indexes are also inherited from parent configuration (e.g., the
user-level `uv.toml`), but are placed after any indexes in the current
project, matching our semantics for other array-based configuration
values.
You can mix `--index` and `--default-index` with the legacy
`--index-url` and `--extra-index-url` settings; the latter two are
merely treated as unnamed `[[tool.uv.index]]` entries.
### Index pinning
If an index includes a name (which is optional), it can then be
referenced via `tool.uv.sources`:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[tool.uv.sources]
torch = { index = "pytorch" }
```
If an index is marked as `explicit = true`, it can _only_ be used via
such references, and will never be searched implicitly:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
explicit = true
[tool.uv.sources]
torch = { index = "pytorch" }
```
Indexes defined outside of the current project (e.g., in the user-level
`uv.toml`) can _not_ be explicitly selected.
(As of now, we only support using a single index for a given
`tool.uv.sources` definition.)
### Default index
By default, we include PyPI as the default index. This remains true even
if the user defines a `[[tool.uv.index]]` -- PyPI is still used as a
fallback. You can mark an index as `default = true` to (1) disable the
use of PyPI, and (2) bump it to the bottom of the prioritized list, such
that it's used only if a package does not exist on a prior index:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
default = true
```
### Name reuse
If a name is reused, the higher-priority index with that name is used,
while the lower-priority indexes are ignored entirely.
For example, given:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[[tool.uv.index]]
name = "pytorch"
url = "https://test.pypi.org/simple"
```
The `https://test.pypi.org/simple` index would be ignored entirely,
since it's lower-priority than `https://download.pytorch.org/whl/cu121`
but shares the same name.
Closes#171.
## Future work
- Users should be able to provide authentication for named indexes via
environment variables.
- `uv add` should automatically write `--index` entries to the
`pyproject.toml` file.
- Users should be able to provide multiple indexes for a given package,
stratified by platform:
```toml
[tool.uv.sources]
torch = [
{ index = "cpu", markers = "sys_platform == 'darwin'" },
{ index = "gpu", markers = "sys_platform != 'darwin'" },
]
```
- Users should be able to specify a proxy URL for a given index, to
avoid writing user-specific URLs to a lockfile:
```toml
[[tool.uv.index]]
name = "test"
url = "https://private.org/simple"
proxy = "http://<omitted>/pypi/simple"
```
## Summary
This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.
I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.
This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.
I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.
I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.
Did I miss any? Are their initial docs cohesive?
Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.
## Test Plan
Existing tests
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html
Before that, there were 91 separate integration tests binary.
(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
## Summary
We can't rely on reading these from the `pyproject.toml`; instead, we
resolve the project metadata (which will typically just require reading
the `pyproject.toml`, but will go through our standard metadata paths).
Closes https://github.com/astral-sh/uv/issues/8071.
## Summary
This PR enables users to provide multiple source entries in
`tool.uv.sources`, e.g.:
```toml
[tool.uv.sources]
httpx = [
{ git = "https://github.com/encode/httpx", tag = "0.27.2", marker = "sys_platform == 'darwin'" },
{ git = "https://github.com/encode/httpx", tag = "0.24.1", marker = "sys_platform == 'linux'" },
]
```
The implementation is relatively straightforward: when we lower the
requirement, we now return an iterator rather than a single requirement.
In other words, the above is transformed into two requirements:
```txt
httpx @ git+https://github.com/encode/httpx@0.27.2 ; sys_platform == 'darwin'
httpx @ git+https://github.com/encode/httpx@0.24.1 ; sys_platform == 'linux'
```
We verify (at deserialization time) that the markers are
non-overlapping.
Closes https://github.com/astral-sh/uv/issues/3397.
## Summary
When syncing a lockfile, we need to respect credentials defined in the
`pyproject.toml`, even if they won't be used for resolution.
Unfortunately, this includes credentials in `tool.uv.sources`,
`tool.uv.dev-dependencies`, `project.dependencies`, and
`project.optional-dependencies`.
Closes https://github.com/astral-sh/uv/issues/7453.
## Summary
We were only applying exclusions when discovering the root, apparently.
Our logic now matches the original intent, which is...
- `exclude` always post-filters `members`.
- We don't treat globs any differently than non-globs.
The one confusing setup that falls out of this is that given:
```toml
members = ["foo/bar/baz"]
exclude = ["foo/bar"]
```
`foo/bar/baz` **would** be included. To exclude it, you would need:
```toml
members = ["foo/bar/baz"]
exclude = ["foo/bar/*"]
```
Closes https://github.com/astral-sh/uv/issues/7071.
Allows configuration of the (currently hard-coded) path to the virtual
environment in projects using the `UV_PROJECT_ENVIRONMENT` environment
variable.
If empty, we'll ignore it. If a relative path, it will be resolved
relative to the workspace root. If an absolute path, we'll use that.
This feature targets use in Docker images and CI. The variable is
intended to be set once in an isolated system and used for all uv
operations.
We do not expose a CLI option or configuration file setting — we may
pursue those later but I see them as lower priority. I think a
system-level environment variable addresses the most pressing use-cases
here.
This doesn't special-case the system environment. Which means that you
can use this to write to the system Python environment. I would
generally strongly recommend against doing so. The insightful comment
from @edmorley at
https://github.com/astral-sh/uv/issues/5229#issuecomment-2312702902
provides some context on why. More generally, `uv sync` will remove
packages from the environment by default. This means that if the system
environment contains any packages relevant to the operation of the
system (that are not dependencies of your project), `uv sync` will break
it. I'd only use this in Docker or CI, if anywhere. Virtual environments
have lots of benefits, and it's only [one line to "activate"
them](https://docs.astral.sh/uv/guides/integration/docker/#using-the-environment).
If you are considering using this feature to use Docker bind mounts for
developing in containers, I would highly recommend reading our [Docker
container development
documentation](https://docs.astral.sh/uv/guides/integration/docker/#developing-in-a-container)
first. If the solutions there do not work for you, please open an issue
describing your use-case and why.
We do not read `VIRTUAL_ENV` and do not have plans to at this time.
Reading `VIRTUAL_ENV` is high-risk, because users can easily leave an
environment active and use the uv project interface today. Reading
`VIRTUAL_ENV` would be a breaking change. Additionally, uv is
intentionally moving away from the concept of "active environments" and
I don't think syncing to an "active" environment is the right behavior
while managing projects. I plan to add a warning if `VIRTUAL_ENV` is
set, to avoid confusion in this area (see
https://github.com/astral-sh/uv/pull/6864).
This does not directly enable centrally managed virtual environments. If
you set `UV_PROJECT_ENVIRONMENT` to an absolute path and use it across
multiple projects, they will clobber each other's environments. However,
you could use this with something like `direnv` to achieve "centrally
managed" environments. I intend to build a prototype of this eventually.
See #1495 for more details on this use-case.
Lots of discussion about this feature in:
- https://github.com/astral-sh/rye/issues/371
- https://github.com/astral-sh/rye/pull/1222
- https://github.com/astral-sh/rye/issues/1211
- https://github.com/astral-sh/uv/issues/5229
- https://github.com/astral-sh/uv/issues/6669
- https://github.com/astral-sh/uv/issues/6612
Follow-ups:
- #6835
- https://github.com/astral-sh/uv/pull/6864
- Document this in the project concept documentation (can probably
re-use some of this post)
Closes https://github.com/astral-sh/uv/issues/6669
Closes https://github.com/astral-sh/uv/issues/5229
Closes https://github.com/astral-sh/uv/issues/6612
## Summary
If you're in a directory, and there's workspace above it, we check if
the directory is excluded from the workspace members... But not if it's
_included_ in the first place.
Closes https://github.com/astral-sh/uv/issues/6732.
