We currently treat path sources as virtual if they do not specify a
build system, which is surprising behavior. This PR updates the behavior
to treat path sources as packages unless the path source is explicitly
marked as `package = false` or its own `tool.uv.package` is set to
`false`.
Closes#12015
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
By default, `uv venv <venv-name>` currently removes the `<venv-name`>
directory if it exists. This can be surprising behavior: not everyone
expects an existing environment to be overwritten. This PR updates the
default to fail if a non-empty `<venv-name>` directory already exists
and neither `--allow-existing` nor the new `-c/--clear` option is
provided (if a TTY is detected, it prompts first). If it's not a TTY,
then uv will only warn and not fail for now — we'll make this an error
in the future. I've also added a corresponding `UV_VENV_CLEAR` env var.
I've chosen to use `--clear` instead of `--force` for this option
because it is used by the `venv` module and `virtualenv` and will be
familiar to users. I also think its meaning is clearer in this context
than `--force` (which could plausibly mean force overwrite just the
virtual environment files, which is what our current `--allow-existing`
option does).
Closes#1472.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
In the case of `uv sync` all we really need to do is handle the
`OutdatedEnvironment` error (precisely the error we yield only on
dry-runs when everything Works but we determine things are outdated) in
`OperationDiagnostic::report` (the post-processor on all
`operations::install` calls) because any diagnostic handled by that gets
downgraded to from status 2 to status 1 (although I don't know if that's
really intentional or a random other bug in our status handling... but I
figured it's best to highlight that other potential status code
incongruence than not rely on it 😄).
Fixes#12603
---------
Co-authored-by: John Mumm <jtfmumm@gmail.com>
We weren't following our usual "destructure all the options" pattern in
this function, and several "this isn't actually read from uv.toml"
fields slipped through the cracks over time since folks forgot it
existed.
Fixes part of #14308, although we could still try to make the warning in
FilesystemOptions more accurate?
You could argue this is a breaking change, but I think it ultimately
isn't really, because we were already silently ignoring these fields.
Now we properly error.
If a user specifies `-e /path/to/dir` and `/path/to/dir` in a `uv pip
install` command, we want the editable to "win" (rather than erroring
due to conflicting URLs). Unfortunately, this behavior meant that when
you requested a package as editable and non-editable in conflicting
groups, the editable version was _always_ used. This PR modifies the
requisite types to use `Option<bool>` rather than `bool` for the
`editable` field, so we can determine whether a requirement was
explicitly requested as editable, explicitly requested as non-editable,
or not specified (as in the case of `/path/to/dir` in a
`requirements.txt` file). In the latter case, we allow editables to
override the "unspecified" requirement.
If a project includes a path dependency twice, once with `editable =
true` and once without any `editable` annotation, those are now
considered conflicting URLs, and lead to an error, so I've marked this
change as breaking.
Closes https://github.com/astral-sh/uv/issues/14139.
If `--workspace` is provided, we add all paths as workspace members.
If `--no-workspace` is provided, we add all paths as direct path
dependencies.
If neither is provided, then we add any paths that are under the
workspace root as workspace members, and the rest as direct path
dependencies.
Closes#14524.
While reviewing https://github.com/astral-sh/uv/pull/14107, @oconnor663
pointed out a bug where we allow `uv python pin --rm` to delete the
global pin without the `--global` flag. I think that shouldn't be
allowed? I'm not 100% certain though.
Adds environment variables for
https://github.com/astral-sh/uv/pull/14612 and
https://github.com/astral-sh/uv/pull/14614
We can't use the Clap `BoolishValueParser` here, and the reasoning is a
little hard to explain. If we used `UV_PYTHON_INSTALL_NO_BIN`, as is our
typical pattern, it'd work, but here we allow opt-in to hard errors with
`UV_PYTHON_INSTALL_BIN=1` and I don't think we should have both
`UV_PYTHON_INSTALL_BIN` and `UV_PYTHON_INSTALL_NO_BIN`.
Consequently, this pull request introduces a new `EnvironmentOptions`
abstraction which allows us to express semantics that Clap cannot —
which we probably want anyway because we have an increasing number of
environment variables we're parsing downstream, e.g., #14544 and #14369.
## Summary
When refactoring the addition PR I accidentally introduced a bug where
the debug message would not be output if the default value is used.
cc @zanieb
## Summary
When installing packages on _very_ slow/overloaded systems it'spossible
to trigger bytecode compilation timeouts, which tends to happen in
environments such as Qemu (especially without KVM/virtio), but also on
systems that are simply overloaded. I've seen this in my Nix builds if I
for example am compiling a Linux kernel at the same time as a few other
concurrent builds.
By making the bytecode compilation timeout adjustable you can work
around such issues. I plan to set `UV_COMPILE_BYTECODE_TIMEOUT=0` in the
[pyproject.nix
builders](https://pyproject-nix.github.io/pyproject.nix/build.html) to
make them more reliable.
- Related issues
* https://github.com/astral-sh/uv/issues/6105
## Test Plan
Only manual testing was applied in this instance. There is no existing
automated tests for bytecode compilation timeout afaict.
Closes#14262
## Description
Adds `UV_LIBC` environment variable and implements check within
`Libc::from_env` as recommended here:
https://github.com/astral-sh/uv/issues/14262#issuecomment-3014600313
Gave this a few passes to make sure I follow dev practices within uv as
best I am able. Feel free to call out anything that could be improved.
## Test Plan
Planned to simply run existing test suite. Open to adding more tests
once implementation is validated due to my limited Rust experience.
## Summary
We validate the `uv.toml` when it's discovered automatically, but not
when provided via `--config-file`. The same limitations exist, though --
I think the lack of enforcement is just an oversight.
Closes https://github.com/astral-sh/uv/issues/14650.
Currently we treat all spawn failures as fatal, because they indicate a
broken interpreter. In this case, I think we should just skip these
broken interpreters — though I don't know the root cause of why it's
broken yet.
Closes https://github.com/astral-sh/uv/issues/14637
See
1394758502
Part of #14296
This is the same as `uv tool update-shell` but handles the case where
the Python bin directory is configured to a different path.
```
❯ UV_PYTHON_BIN_DIR=/tmp/foo cargo run -q -- python install --preview 3.13.3
Installed Python 3.13.3 in 1.75s
+ cpython-3.13.3-macos-aarch64-none
warning: `/tmp/foo` is not on your PATH. To use installed Python executables, run `export PATH="/tmp/foo:$PATH"` or `uv python update-shell`.
❯ UV_PYTHON_BIN_DIR=/tmp/foo cargo run -q -- python update-shell
Created configuration file: /Users/zb/.zshenv
Restart your shell to apply changes
❯ cat /Users/zb/.zshenv
# uv
export PATH="/tmp/foo:$PATH"
❯ UV_TOOL_BIN_DIR=/tmp/bar cargo run -q -- tool update-shell
Updated configuration file: /Users/zb/.zshenv
Restart your shell to apply changes
❯ cat /Users/zb/.zshenv
# uv
export PATH="/tmp/foo:$PATH"
# uv
export PATH="/tmp/bar:$PATH"
```
Previously, if installation of executables into the bin directory failed
we'd with a non-zero code. However, if we make this behavior the default
we don't want it to be fatal. There's a `--bin` opt-in to _require_
successful executable installation and a `--no-bin` opt-out to silence
the warning / opt-out of installation entirely.
Part of https://github.com/astral-sh/uv/issues/14296 — we need this
before we can stabilize the behavior.
In #14614 we do the same for writing entries to the Windows registry.
## Summary
You can now override the cache control headers for the Simple API, file
downloads, or both:
```toml
[[tool.uv.index]]
name = "example"
url = "https://example.com/simple"
cache-control = { api = "max-age=600", files = "max-age=365000000, immutable" }
```
Closes https://github.com/astral-sh/uv/issues/10444.
## Summary
There's some inconsistent behaviour in handling symlinks when
`cache-key` is a glob or a file path. This PR attempts to address that.
- When cache-key is a path,
[`Path::metadata()`](https://doc.rust-lang.org/std/path/struct.Path.html#method.metadata)
is used to check if it's a file or not. According to the docs:
> This function will traverse symbolic links to query information about
the destination file.
So, if the target file is a symlink, it will be resolved and the
metadata will be queried for the underlying file.
- When cache-key is a glob, `globwalk` is used, specifically allowing
for symlinks:
```rust
.file_type(globwalk::FileType::FILE | globwalk::FileType::SYMLINK)
```
- However, without enabling link following, `DirEntry::metadata()` will
return an equivalent of `Path::symlink_metadata()` (and not
`Path::metadata()`), which will have a file type that looks like
```rust
FileType {
is_file: false,
is_dir: false,
is_symlink: true,
..
}
```
- Then, there's a check for `metadata.is_file()` which fails and
complains that the target entry "is a directory when file was expected".
- TLDR: glob cache-keys don't work with symlinks.
## Solutions
Option 1 (current PR): follow symlinks.
Option 2 (also doable): don't follow symlinks, but resolve the resulting
target entry manually in case its file type is a symlink. However, this
would be a little weird and unobvious in that we resolve files but not
directories for some reason. Also, symlinking directories is pretty
useful if you want to symlink directories of local dependencies that are
not under the project's path.
## Test Plan
This has been tested manually:
```rust
fn main() {
for follow_links in [false, true] {
let walker = globwalk::GlobWalkerBuilder::from_patterns(".", &["a/*"])
.file_type(globwalk::FileType::FILE | globwalk::FileType::SYMLINK)
.follow_links(follow_links)
.build()
.unwrap();
let entry = walker.into_iter().next().unwrap().unwrap();
dbg!(&entry);
dbg!(entry.file_type());
dbg!(entry.path_is_symlink());
dbg!(entry.path());
let meta = entry.metadata().unwrap();
dbg!(meta.is_file());
}
let path = std::path::PathBuf::from("./a/b");
dbg!(path.metadata().unwrap().file_type());
dbg!(path.symlink_metadata().unwrap().file_type());
}
```
Current behaviour (glob cache-key, don't follow links):
```
[src/main.rs:9:9] &entry = DirEntry("./a/b")
[src/main.rs:10:9] entry.file_type() = FileType {
is_file: false,
is_dir: false,
is_symlink: true,
..
}
[src/main.rs:11:9] entry.path_is_symlink() = true
[src/main.rs:12:9] entry.path() = "./a/b"
[src/main.rs:14:9] meta.is_file() = false
```
Glob cache-key, follow links:
```
[src/main.rs:9:9] &entry = DirEntry("./a/b")
[src/main.rs:10:9] entry.file_type() = FileType {
is_file: true,
is_dir: false,
is_symlink: false,
..
}
[src/main.rs:11:9] entry.path_is_symlink() = true
[src/main.rs:12:9] entry.path() = "./a/b"
[src/main.rs:14:9] meta.is_file() = true
```
Using `path.metadata()` for a non-glob cache key:
```
[src/main.rs:18:5] path.metadata().unwrap().file_type() = FileType {
is_file: true,
is_dir: false,
is_symlink: false,
..
}
[src/main.rs:19:5] path.symlink_metadata().unwrap().file_type() = FileType {
is_file: false,
is_dir: false,
is_symlink: true,
..
}
```
This is a continuation of the work in
* #12405
I have:
* moved to an architecture where the human output is derived from the
json structs to centralize more of the printing state/logic
* cleaned up some of the names/types
* added tests
* removed the restriction that this output is --dry-run only
I have not yet added package info, which was TBD in their design.
---------
Co-authored-by: x0rw <mahdi.svt5@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-authored-by: John Mumm <jtfmumm@gmail.com>
We've seen a few cases of uv.exe exiting with an exception code as its
exit status and no user-visible output (#14563 in the field, and #13812
in CI). It seems that recent versions of Windows no longer show dialog
boxes on access violations (what UNIX calls segfaults) or similar
errors. Something is probably sent to Windows Error Reporting, and we
can maybe sign up to get the crashes from Microsoft, but the user
experience of seeing uv exit with no output is poor, both for end users
and during development. While it's possible to opt out of this behavior
or set up a debugger, this isn't the default configuration. (See
https://superuser.com/q/1246626 for some pointers.)
In order to get some output on a crash, we need to install our own
default handler for unhandled exceptions (or call all our code inside a
Structured Exception Handling __try/__catch block, which is complicated
on Rust). This is the moral equivalent of a segfault handler on Windows;
the kernel creates a new stack frame and passes arguments to it with
some processor state.
This commit adds a relatively simple exception handler that leans on
Rust's own backtrace implementation and also displays some minimal
information from the exception itself. This should be enough info to
communicate that something went wrong and let us collect enough
information to attempt to debug. There are also a handful of (non-Rust)
open-source libraries for this like Breakpad and Crashpad (both from
Google) and crashrpt.
The approach here, of using SetUnhandledExceptionFilter, seems to be the
standard one taken by other such libraries. Crashpad also seems to try
to use a newer mechanism for an out-of-tree DLL to report the crash:
https://issues.chromium.org/issues/42310037
If we have serious problems with memory corruption, it might be worth
adopting some third-party library that has already implemented this
approach. (In general, the docs of other crash reporting libraries are
worth skimming to understand how these things ought to work.)
Co-authored-by: samypr100 <3933065+samypr100@users.noreply.github.com>
## Summary
(Related PR: #13438 - would be nice to have it merged as well since it
touches on the same globwalker code)
There's a few issues with `cache-key` globs, which this PR attempts to
address:
- As of the current state, parent or absolute paths are not allowed,
which is not obvious and is not documented. E.g., cache-key paths of the
form `{file = "../dep/**"}` will be essentially ignored.
- Absolute glob patterns also don't work (funnily enough, there's logic
in `globwalk` itself that attempts to address it in
[`globwalk::glob_builder()`](8973fa2bc5/src/lib.rs (L415)),
which serves as inspiration to some parts of this PR).
- The reason for parent paths being ignored is the way globwalker is
currently being triggered in `uv-cache-info`: the base directory is
being walked over completely and each entry is then being matched to one
of the provided match patterns.
- This may also end up being very inefficient if you have a huge root
folder with thousands of files: if your match patterns are `a/b/*.rs`
and `a/c/*.py` then instead of walking over the root directory, you can
just walk over `a/b` and `a/c` and match the relevant patterns there.
- Why supporting parent paths may be important to the point of being a
blocker: in large codebases with python projects depending on other
local non-python projects (e.g. rust crates), cache-keys can be very
useful to track dependency on the source code of the latter (e.g.
`cache-keys = [{ file = "../../crates/some-dep/**" }]`.
- TLDR: parent/absolute cache-key globs don't work, glob walk can be
slow.
## Solution
- In this PR, user-provided glob patterns are first clustered
(LCP-style) into pattern groups with longest common path prefix; each of
these groups can then be walked over separately.
- Pattern groups do not overlap, so we would never walk over the same
directory twice (unless there's symlinks pointing to same folders).
- Paths are not canonicalized nor virtually normalized (which is
impossible on Unix without FS access), so the method is symlink-safe
(i.e. we don't treat `a/b/..` as `a`) and should work fine with #13438.
- Because of LCP logic, the minimal amount of directory space will be
traversed to cover all patterns.
- Absolute glob patterns will now work.
- Parent-relative glob patterns will now work.
- Glob walking will be more efficient in some cases.
## Possible improvements
- Efficiency can be further greatly improved if we limit max depth for
globwalk. Currently, a simple ".toml" will deep-traverse the whole
folder. Essentially, max depth can be always set to either N or
infinity. If a pattern at a pivot node contains `**`, we collect all
children nodes from the subtree into the same group and don't limit max
depth; otherwise, we set max depth to the length of the glob pattern.
This wouldn't change correctness though and can we done separately if
needed.
- If this is considered important enough, docs can be updated to
indicate that parent and absolute globs are supported (and symlinks are
resolved, if the relevant PR is algo merged in).
## Test Plan
- Glob splitting and clustering tests are included in the PR.
- Relative and absolute glob cache-keys were tested in an actual
codebase.
