Respect build constraints in uv sync (#12502)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on linux (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions

## Summary

There are still a few missing sites that we need to audit:

- `uv tool install` (https://github.com/astral-sh/uv/issues/12496)
- `uv tool run` (https://github.com/astral-sh/uv/issues/12496)
- The `--with` dependencies in `uv run --with`
(https://github.com/astral-sh/uv/issues/12505)

Closes #12441.
This commit is contained in:
Charlie Marsh 2025-03-27 17:11:50 -04:00 committed by GitHub
parent 50cf7d19b0
commit 9e10f83ce7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 273 additions and 13 deletions

View file

@ -78,7 +78,7 @@ impl Requirement {
self.source.is_editable()
}
/// Convert the requirement to a [`Requirement`] relative to the given path.
/// Convert to a [`Requirement`] with a relative path based on the given root.
pub fn relative_to(self, path: &Path) -> Result<Self, io::Error> {
Ok(Self {
source: self.source.relative_to(path)?,
@ -86,6 +86,15 @@ impl Requirement {
})
}
/// Convert to a [`Requirement`] with an absolute path based on the given root.
#[must_use]
pub fn to_absolute(self, path: &Path) -> Self {
Self {
source: self.source.to_absolute(path),
..self
}
}
/// Return the hashes of the requirement, as specified in the URL fragment.
pub fn hashes(&self) -> Option<Hashes> {
let RequirementSource::Url { ref url, .. } = self.source else {
@ -593,6 +602,37 @@ impl RequirementSource {
}),
}
}
/// Convert the source to a [`RequirementSource`] with an absolute path based on the given root.
#[must_use]
pub fn to_absolute(self, root: &Path) -> Self {
match self {
RequirementSource::Registry { .. }
| RequirementSource::Url { .. }
| RequirementSource::Git { .. } => self,
RequirementSource::Path {
install_path,
ext,
url,
} => Self::Path {
install_path: uv_fs::normalize_path_buf(root.join(install_path)).into_boxed_path(),
ext,
url,
},
RequirementSource::Directory {
install_path,
editable,
r#virtual,
url,
..
} => Self::Directory {
install_path: uv_fs::normalize_path_buf(root.join(install_path)).into_boxed_path(),
editable,
r#virtual,
url,
},
}
}
}
impl Display for RequirementSource {

View file

@ -19,7 +19,7 @@ use tracing::debug;
use url::Url;
use uv_cache_key::RepositoryUrl;
use uv_configuration::BuildOptions;
use uv_configuration::{BuildOptions, Constraints};
use uv_distribution::{DistributionDatabase, FlatRequiresDist};
use uv_distribution_filename::{
BuildTag, DistExtension, ExtensionError, SourceDistExtension, WheelFilename,
@ -674,6 +674,17 @@ impl Lock {
&self.manifest.dependency_groups
}
/// Returns the build constraints that were used to generate this lock.
pub fn build_constraints(&self, root: &Path) -> Constraints {
Constraints::from_requirements(
self.manifest
.build_constraints
.iter()
.cloned()
.map(|requirement| requirement.to_absolute(root)),
)
}
/// Return the workspace root used to generate this lock.
pub fn root(&self) -> Option<&Package> {
self.packages.iter().find(|package| {
@ -931,6 +942,26 @@ impl Lock {
manifest_table.insert("overrides", value(overrides));
}
if !self.manifest.build_constraints.is_empty() {
let build_constraints = self
.manifest
.build_constraints
.iter()
.map(|requirement| {
serde::Serialize::serialize(
&requirement,
toml_edit::ser::ValueSerializer::new(),
)
})
.collect::<Result<Vec<_>, _>>()?;
let build_constraints = match build_constraints.as_slice() {
[] => Array::new(),
[requirement] => Array::from_iter([requirement]),
build_constraints => each_element_on_its_line_array(build_constraints.iter()),
};
manifest_table.insert("build-constraints", value(build_constraints));
}
if !self.manifest.dependency_groups.is_empty() {
let mut dependency_groups = Table::new();
for (extra, requirements) in &self.manifest.dependency_groups {
@ -1188,6 +1219,7 @@ impl Lock {
requirements: &[Requirement],
constraints: &[Requirement],
overrides: &[Requirement],
build_constraints: &[Requirement],
dependency_groups: &BTreeMap<GroupName, Vec<Requirement>>,
dependency_metadata: &DependencyMetadata,
indexes: Option<&IndexLocations>,
@ -1279,6 +1311,27 @@ impl Lock {
}
}
// Validate that the lockfile was generated with the same build constraints.
{
let expected: BTreeSet<_> = build_constraints
.iter()
.cloned()
.map(|requirement| normalize_requirement(requirement, root))
.collect::<Result<_, _>>()?;
let actual: BTreeSet<_> = self
.manifest
.build_constraints
.iter()
.cloned()
.map(|requirement| normalize_requirement(requirement, root))
.collect::<Result<_, _>>()?;
if expected != actual {
return Ok(SatisfiesResult::MismatchedBuildConstraints(
expected, actual,
));
}
}
// Validate that the lockfile was generated with the dependency groups.
{
let expected: BTreeMap<GroupName, BTreeSet<Requirement>> = dependency_groups
@ -1685,6 +1738,8 @@ pub enum SatisfiesResult<'lock> {
MismatchedConstraints(BTreeSet<Requirement>, BTreeSet<Requirement>),
/// The lockfile uses a different set of overrides.
MismatchedOverrides(BTreeSet<Requirement>, BTreeSet<Requirement>),
/// The lockfile uses a different set of build constraints.
MismatchedBuildConstraints(BTreeSet<Requirement>, BTreeSet<Requirement>),
/// The lockfile uses a different set of dependency groups.
MismatchedDependencyGroups(
BTreeMap<GroupName, BTreeSet<Requirement>>,
@ -1765,6 +1820,9 @@ pub struct ResolverManifest {
/// The overrides provided to the resolver.
#[serde(default)]
overrides: BTreeSet<Requirement>,
/// The build constraints provided to the resolver.
#[serde(default)]
build_constraints: BTreeSet<Requirement>,
/// The static metadata provided to the resolver.
#[serde(default)]
dependency_metadata: BTreeSet<StaticMetadata>,
@ -1778,6 +1836,7 @@ impl ResolverManifest {
requirements: impl IntoIterator<Item = Requirement>,
constraints: impl IntoIterator<Item = Requirement>,
overrides: impl IntoIterator<Item = Requirement>,
build_constraints: impl IntoIterator<Item = Requirement>,
dependency_groups: impl IntoIterator<Item = (GroupName, Vec<Requirement>)>,
dependency_metadata: impl IntoIterator<Item = StaticMetadata>,
) -> Self {
@ -1786,6 +1845,7 @@ impl ResolverManifest {
requirements: requirements.into_iter().collect(),
constraints: constraints.into_iter().collect(),
overrides: overrides.into_iter().collect(),
build_constraints: build_constraints.into_iter().collect(),
dependency_groups: dependency_groups
.into_iter()
.map(|(group, requirements)| (group, requirements.into_iter().collect()))
@ -1813,6 +1873,11 @@ impl ResolverManifest {
.into_iter()
.map(|requirement| requirement.relative_to(root))
.collect::<Result<BTreeSet<_>, _>>()?,
build_constraints: self
.build_constraints
.into_iter()
.map(|requirement| requirement.relative_to(root))
.collect::<Result<BTreeSet<_>, _>>()?,
dependency_groups: self
.dependency_groups
.into_iter()
@ -2375,11 +2440,8 @@ impl Package {
url.set_query(None);
// Reconstruct the `GitUrl` from the `GitSource`.
let git_url = uv_git_types::GitUrl::from_commit(
url,
GitReference::from(git.kind.clone()),
git.precise,
)?;
let git_url =
GitUrl::from_commit(url, GitReference::from(git.kind.clone()), git.precise)?;
// Reconstruct the PEP 508-compatible URL from the `GitSource`.
let url = Url::from(ParsedGitUrl {

View file

@ -117,6 +117,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -124,6 +124,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -116,6 +116,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -195,6 +195,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -195,6 +195,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -220,6 +220,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -195,6 +195,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -97,6 +97,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -93,6 +93,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -83,6 +83,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -83,6 +83,7 @@ Ok(
dependency_groups: {},
constraints: {},
overrides: {},
build_constraints: {},
dependency_metadata: {},
},
},

View file

@ -5,7 +5,7 @@ use std::str::FromStr;
use itertools::Either;
use rustc_hash::FxHashSet;
use uv_configuration::{DependencyGroupsWithDefaults, ExtrasSpecification};
use uv_configuration::{Constraints, DependencyGroupsWithDefaults, ExtrasSpecification};
use uv_distribution_types::Index;
use uv_normalize::PackageName;
use uv_pypi_types::{DependencyGroupSpecifier, LenientRequirement, VerbatimParsedUrl};
@ -236,6 +236,10 @@ impl<'lock> InstallTarget<'lock> {
}
}
pub(crate) fn build_constraints(&self) -> Constraints {
self.lock().build_constraints(self.install_path())
}
/// Validate the extras requested by the [`ExtrasSpecification`].
#[allow(clippy::result_large_err)]
pub(crate) fn validate_extras(self, extras: &ExtrasSpecification) -> Result<(), ProjectError> {

View file

@ -624,8 +624,6 @@ async fn do_lock(
.build();
let hasher = HashStrategy::Generate(HashGeneration::Url);
let build_constraints = Constraints::from_requirements(build_constraints.iter().cloned());
// TODO(charlie): These are all default values. We should consider whether we want to make them
// optional on the downstream APIs.
let build_hasher = HashStrategy::default();
@ -647,7 +645,7 @@ async fn do_lock(
let build_dispatch = BuildDispatch::new(
&client,
cache,
build_constraints,
Constraints::from_requirements(build_constraints.iter().cloned()),
interpreter,
index_locations,
&flat_index,
@ -679,6 +677,7 @@ async fn do_lock(
&dependency_groups,
&constraints,
&overrides,
&build_constraints,
&conflicts,
environments,
required_environments,
@ -837,6 +836,7 @@ async fn do_lock(
requirements,
constraints,
overrides,
build_constraints,
dependency_groups,
dependency_metadata.values().cloned(),
)
@ -889,6 +889,7 @@ impl ValidatedLock {
dependency_groups: &BTreeMap<GroupName, Vec<Requirement>>,
constraints: &[Requirement],
overrides: &[Requirement],
build_constraints: &[Requirement],
conflicts: &Conflicts,
environments: Option<&SupportedEnvironments>,
required_environments: Option<&SupportedEnvironments>,
@ -1066,6 +1067,7 @@ impl ValidatedLock {
requirements,
constraints,
overrides,
build_constraints,
dependency_groups,
dependency_metadata,
indexes,
@ -1144,6 +1146,13 @@ impl ValidatedLock {
);
Ok(Self::Preferable(lock))
}
SatisfiesResult::MismatchedBuildConstraints(expected, actual) => {
debug!(
"Ignoring existing lockfile due to mismatched build constraints:\n Requested: {:?}\n Existing: {:?}",
expected, actual
);
Ok(Self::Preferable(lock))
}
SatisfiesResult::MismatchedDependencyGroups(expected, actual) => {
debug!(
"Ignoring existing lockfile due to mismatched dependency groups:\n Requested: {:?}\n Existing: {:?}",

View file

@ -11,7 +11,7 @@ use uv_auth::UrlAuthPolicies;
use uv_cache::Cache;
use uv_client::{FlatIndexClient, RegistryClientBuilder};
use uv_configuration::{
Concurrency, Constraints, DependencyGroups, DependencyGroupsWithDefaults, DryRun, EditableMode,
Concurrency, DependencyGroups, DependencyGroupsWithDefaults, DryRun, EditableMode,
ExtrasSpecification, HashCheckingMode, InstallOptions, PreviewMode,
};
use uv_dispatch::BuildDispatch;
@ -644,9 +644,11 @@ pub(super) async fn do_sync(
BuildIsolation::SharedPackage(venv, no_build_isolation_package)
};
// Read the build constraints from the lockfile.
let build_constraints = target.build_constraints();
// TODO(charlie): These are all default values. We should consider whether we want to make them
// optional on the downstream APIs.
let build_constraints = Constraints::default();
let build_hasher = HashStrategy::default();
// Extract the hashes from the lockfile.

View file

@ -8924,3 +8924,135 @@ fn locked_version_coherence() -> Result<()> {
Ok(())
}
/// `uv sync` should respect build constraints. In this case, `json-merge-patch` should _not_ fail
/// to build, despite the fact that `setuptools==78.0.1` is the most recent version and _does_ fail
/// to build that package.
///
/// See: <https://github.com/astral-sh/uv/issues/12434>
#[test]
fn sync_build_constraints() -> Result<()> {
let context = TestContext::new("3.12").with_exclude_newer("2025-03-24T19:00:00Z");
let pyproject_toml = context.temp_dir.child("pyproject.toml");
pyproject_toml.write_str(
r#"
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["json-merge-patch"]
[tool.uv]
build-constraint-dependencies = ["setuptools<78"]
"#,
)?;
uv_snapshot!(context.filters(), context.sync().arg("--no-binary-package").arg("json-merge-patch"), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ json-merge-patch==0.2
");
let lock = context.read("uv.lock");
insta::with_settings!(
{
filters => context.filters(),
},
{
assert_snapshot!(
lock, @r#"
version = 1
revision = 1
requires-python = ">=3.12"
[options]
exclude-newer = "2025-03-24T19:00:00Z"
[manifest]
build-constraints = [{ name = "setuptools", specifier = "<78" }]
[[package]]
name = "json-merge-patch"
version = "0.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/39/62/3b783faabac9a099877397d8f7a7cc862a03fbf9fb1b90d414ea7c6bb096/json-merge-patch-0.2.tar.gz", hash = "sha256:09898b6d427c08754e2a97c709cf2dfd7e28bd10c5683a538914975eab778d39", size = 3081 }
[[package]]
name = "project"
version = "0.1.0"
source = { virtual = "." }
dependencies = [
{ name = "json-merge-patch" },
]
[package.metadata]
requires-dist = [{ name = "json-merge-patch" }]
"#
);
}
);
fs_err::remove_dir_all(&context.cache_dir)?;
fs_err::remove_dir_all(&context.venv)?;
// We should also be able to read from the lockfile.
uv_snapshot!(context.filters(), context.sync().arg("--locked"), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Using CPython 3.12.[X] interpreter at: [PYTHON-3.12]
Creating virtual environment at: .venv
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ json-merge-patch==0.2
");
// Modify the build constraints.
pyproject_toml.write_str(
r#"
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["json-merge-patch"]
[tool.uv]
build-constraint-dependencies = ["setuptools<77"]
"#,
)?;
// This should fail, given that the build constraints have changed.
uv_snapshot!(context.filters(), context.sync().arg("--locked"), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
Resolved 2 packages in [TIME]
error: The lockfile at `uv.lock` needs to be updated, but `--locked` was provided. To update the lockfile, run `uv lock`.
");
// Changing the build constraints should lead to a re-resolve.
uv_snapshot!(context.filters(), context.sync(), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 2 packages in [TIME]
Audited 1 package in [TIME]
");
Ok(())
}