Treat already-installed base environment packages as preferences in uv run --with (#13284)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / smoke test | macos (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on linux (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | free-threaded python on github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / integration test | uv_build (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | x86-64 python3.13 on windows aarch64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions

## Summary

If a script has some requirements, and you provide `--with`, we
currently ignore any constraints from those requirements. We might want
to treat them as hard constraints in the future. For now, though, we
just treat them as preferences -- so we _prefer_ those versions, but
don't require them to match and still run the `--with` resolution in
isolation.

Closes https://github.com/astral-sh/uv/issues/13173.
This commit is contained in:
Charlie Marsh 2025-05-04 19:24:57 -04:00 committed by GitHub
parent ea4284c041
commit 2c567a64b9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 139 additions and 33 deletions

View file

@ -4,7 +4,7 @@ use std::str::FromStr;
use rustc_hash::FxHashMap;
use tracing::trace;
use uv_distribution_types::IndexUrl;
use uv_distribution_types::{IndexUrl, InstalledDist};
use uv_normalize::PackageName;
use uv_pep440::{Operator, Version};
use uv_pep508::{MarkerTree, VerbatimUrl, VersionOrUrl};
@ -115,6 +115,21 @@ impl Preference {
}))
}
/// Create a [`Preference`] from an installed distribution.
pub fn from_installed(dist: &InstalledDist) -> Option<Self> {
let InstalledDist::Registry(dist) = dist else {
return None;
};
Some(Self {
name: dist.name.clone(),
version: dist.version.clone(),
marker: MarkerTree::TRUE,
index: PreferenceIndex::Any,
fork_markers: vec![],
hashes: HashDigests::empty(),
})
}
/// Return the [`PackageName`] of the package for this [`Preference`].
pub fn name(&self) -> &PackageName {
&self.name

View file

@ -35,8 +35,8 @@ use uv_python::{
use uv_requirements::upgrade::{read_lock_requirements, LockedRequirements};
use uv_requirements::{NamedRequirementsResolver, RequirementsSpecification};
use uv_resolver::{
FlatIndex, Lock, OptionsBuilder, PythonRequirement, RequiresPython, ResolverEnvironment,
ResolverOutput,
FlatIndex, Lock, OptionsBuilder, Preference, PythonRequirement, RequiresPython,
ResolverEnvironment, ResolverOutput,
};
use uv_scripts::Pep723ItemRef;
use uv_settings::PythonInstallMirrors;
@ -1632,27 +1632,42 @@ pub(crate) async fn resolve_names(
Ok(requirements)
}
#[derive(Debug, Clone)]
pub(crate) enum PreferenceSource<'lock> {
/// The preferences should be extracted from a lockfile.
Lock {
lock: &'lock Lock,
install_path: &'lock Path,
},
/// The preferences will be provided directly as [`Preference`] entries.
Entries(Vec<Preference>),
}
#[derive(Debug, Clone)]
pub(crate) struct EnvironmentSpecification<'lock> {
/// The requirements to include in the environment.
requirements: RequirementsSpecification,
/// The lockfile from which to extract preferences, along with the install path.
lock: Option<(&'lock Lock, &'lock Path)>,
/// The preferences to respect when resolving.
preferences: Option<PreferenceSource<'lock>>,
}
impl From<RequirementsSpecification> for EnvironmentSpecification<'_> {
fn from(requirements: RequirementsSpecification) -> Self {
Self {
requirements,
lock: None,
preferences: None,
}
}
}
impl<'lock> EnvironmentSpecification<'lock> {
/// Set the [`PreferenceSource`] for the specification.
#[must_use]
pub(crate) fn with_lock(self, lock: Option<(&'lock Lock, &'lock Path)>) -> Self {
Self { lock, ..self }
pub(crate) fn with_preferences(self, preferences: PreferenceSource<'lock>) -> Self {
Self {
preferences: Some(preferences),
..self
}
}
}
@ -1765,17 +1780,22 @@ pub(crate) async fn resolve_environment(
let upgrade = Upgrade::default();
// If an existing lockfile exists, build up a set of preferences.
let LockedRequirements { preferences, git } = spec
.lock
.map(|(lock, install_path)| read_lock_requirements(lock, install_path, &upgrade))
.transpose()?
.unwrap_or_default();
let preferences = match spec.preferences {
Some(PreferenceSource::Lock { lock, install_path }) => {
let LockedRequirements { preferences, git } =
read_lock_requirements(lock, install_path, &upgrade)?;
// Populate the Git resolver.
for ResolvedRepositoryReference { reference, sha } in git {
debug!("Inserting Git reference into resolver: `{reference:?}` at `{sha}`");
state.git().insert(reference, sha);
}
// Populate the Git resolver.
for ResolvedRepositoryReference { reference, sha } in git {
debug!("Inserting Git reference into resolver: `{reference:?}` at `{sha}`");
state.git().insert(reference, sha);
}
preferences
}
Some(PreferenceSource::Entries(entries)) => entries,
None => vec![],
};
// Resolve the flat indexes from `--find-links`.
let flat_index = {

View file

@ -31,7 +31,7 @@ use uv_python::{
VersionFileDiscoveryOptions,
};
use uv_requirements::{RequirementsSource, RequirementsSpecification};
use uv_resolver::{Installable, Lock};
use uv_resolver::{Installable, Lock, Preference};
use uv_scripts::Pep723Item;
use uv_settings::PythonInstallMirrors;
use uv_shell::runnable::WindowsRunnable;
@ -49,8 +49,9 @@ use crate::commands::project::lock::LockMode;
use crate::commands::project::lock_target::LockTarget;
use crate::commands::project::{
default_dependency_groups, script_specification, update_environment,
validate_project_requires_python, EnvironmentSpecification, ProjectEnvironment, ProjectError,
ScriptEnvironment, ScriptInterpreter, UniversalState, WorkspacePython,
validate_project_requires_python, EnvironmentSpecification, PreferenceSource,
ProjectEnvironment, ProjectError, ScriptEnvironment, ScriptInterpreter, UniversalState,
WorkspacePython,
};
use crate::commands::reporters::PythonDownloadReporter;
use crate::commands::run::run_to_completion;
@ -898,9 +899,14 @@ hint: If you are running a script with `{}` in the shebang, you may need to incl
};
// If necessary, create an environment for the ephemeral requirements or command.
let base_site_packages = SitePackages::from_interpreter(&base_interpreter)?;
let ephemeral_env = match spec {
None => None,
Some(spec) if can_skip_ephemeral(&spec, &base_interpreter, &settings) => None,
Some(spec)
if can_skip_ephemeral(&spec, &base_interpreter, &base_site_packages, &settings) =>
{
None
}
Some(spec) => {
debug!("Syncing ephemeral requirements");
@ -909,12 +915,24 @@ hint: If you are running a script with `{}` in the shebang, you may need to incl
.as_ref()
.map(|(lock, path)| lock.build_constraints(path));
// Read the preferences.
let spec = EnvironmentSpecification::from(spec).with_preferences(
if let Some((lock, install_path)) = base_lock.as_ref() {
// If we have a lockfile, use the locked versions as preferences.
PreferenceSource::Lock { lock, install_path }
} else {
// Otherwise, extract preferences from the base environment.
PreferenceSource::Entries(
base_site_packages
.iter()
.filter_map(Preference::from_installed)
.collect::<Vec<_>>(),
)
},
);
let result = CachedEnvironment::from_spec(
EnvironmentSpecification::from(spec).with_lock(
base_lock
.as_ref()
.map(|(lock, install_path)| (lock, install_path.as_ref())),
),
spec,
build_constraints.unwrap_or_default(),
&base_interpreter,
&settings,
@ -1115,13 +1133,10 @@ hint: If you are running a script with `{}` in the shebang, you may need to incl
/// Returns `true` if we can skip creating an additional ephemeral environment in `uv run`.
fn can_skip_ephemeral(
spec: &RequirementsSpecification,
base_interpreter: &Interpreter,
interpreter: &Interpreter,
site_packages: &SitePackages,
settings: &ResolverInstallerSettings,
) -> bool {
let Ok(site_packages) = SitePackages::from_interpreter(base_interpreter) else {
return false;
};
if !(settings.reinstall.is_none() && settings.reinstall.is_none()) {
return false;
}
@ -1130,7 +1145,7 @@ fn can_skip_ephemeral(
&spec.requirements,
&spec.constraints,
&spec.overrides,
&base_interpreter.resolver_marker_environment(),
&interpreter.resolver_marker_environment(),
) {
// If the requirements are already satisfied, we're done.
Ok(SatisfiesResult::Fresh {

View file

@ -5078,3 +5078,59 @@ fn run_pep723_script_with_constraints_lock() -> Result<()> {
Ok(())
}
/// If a `--with` requirement overlaps with a non-locked script requirement, respect the environment
/// site-packages as preferences.
///
/// See: <https://github.com/astral-sh/uv/issues/13173>
#[test]
fn run_pep723_script_with_constraints() -> Result<()> {
let context = TestContext::new("3.12");
let test_script = context.temp_dir.child("main.py");
test_script.write_str(indoc! { r#"
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "iniconfig<2",
# ]
# ///
import iniconfig
print("Hello, world!")
"#
})?;
let pyproject_toml = context.temp_dir.child("pyproject.toml");
pyproject_toml.write_str(indoc! { r#"
[project]
name = "foo"
version = "1.0.0"
requires-python = ">=3.10"
dependencies = [
"iniconfig",
]
"#
})?;
uv_snapshot!(context.filters(), context.run().arg("--with").arg(".").arg("main.py"), @r"
success: true
exit_code: 0
----- stdout -----
Hello, world!
----- stderr -----
Resolved 1 package in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==1.1.1
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 2 packages in [TIME]
+ foo==1.0.0 (from file://[TEMP_DIR]/)
+ iniconfig==1.1.1
");
Ok(())
}