Commit graph

283 commits

Author SHA1 Message Date
Charlie Marsh
e3f345ce09
Validate lockfile (rather than re-resolve) in uv lock (#6091)
## Summary

Historically, in order to "resolve from a lockfile", we've taken the
lockfile, used it to pre-populate the in-memory metadata index, then run
a resolution. If the resolution didn't match our existing resolution, we
re-resolved from scratch.

This was an appealing approach because (in theory) it didn't require any
dedicated logic beyond pre-populating the index. However, it's proven to
be _really_ hard to get right, because it's a stricter requirement than
we need. We just need the current lockfile to _satisfy_ the requirements
provided by the user. We don't actually need a second resolution to
produce the exact same result. And it's not uncommon that this second
resolution differs, because we seed it with preferences, which
fundamentally changes its course. We've worked hard to minimize those
"instabilities", but they're still present.

The approach here is intended to be much simpler. Instead of resolving
from the lockfile, we just check if the current resolution satisfies the
state of the workspace. Specifically, we check if the lockfile (1)
contains all the relevant members, and (2) matches the metadata for all
dependencies, recursively. (We skip registry dependencies, assuming that
they're immutable.)

This may actually be too conservative, since we can have resolutions
that satisfy the requirements, even if the requirements have changed
slightly. But we want to bias towards correctness for now.

My hope is that this scheme will be more performant, simpler, and more
robust.

Closes https://github.com/astral-sh/uv/issues/6063.
2024-08-14 20:00:15 -04:00
Charlie Marsh
1bbb05dca7
Invalidate uv.lock if registry sources are removed (#6026)
## Summary

Now, if you resolve against a registry, then swap it out for another, we
won't reuse the lockfile. (If you don't provide any registry
configuration, then we won't enforce this, so that `uv lock --index-url
foo` and `uv lock` is stable.)

Closes https://github.com/astral-sh/uv/issues/5920.
2024-08-13 23:42:04 +00:00
konsti
1e6b021506
Update packse to 0.3.34 (#5954)
Preparation for #5905
2024-08-09 09:04:17 +00:00
konsti
4038c9a6af
Rename distribution to packages in lockfile (#5861)
Currently, the entry for a package+version+source table is called
`distribution`. That is incorrect, the `sdist` and `wheel` fields inside
of that table are distributions, the table itself is for a package. We
also align ourselves closer with PEP 751.

I went through `lock.rs` and renamed all occurrences of "distribution"
that actually referred to a "package".

This change invalidates all existing lockfiles.

Bikeshedding: Do we call it `package` or `packages`? See also
https://github.com/python/peps/pull/3877

`package` is nice because it looks like a header:

```toml
[[package]]
name = "anyio"
version = "4.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
    { name = "idna" },
    { name = "sniffio" },
]
sdist = { url = "3970183622/anyio-4.3.0.tar.gz", hash = "sha256:f75253795a87df48568485fd18cdd2a3fa5c4f7c5be8e5e36637733fce06fed6", size = 159642 }
wheels = [
    { url = "2f20c40b45/anyio-4.3.0-py3-none-any.whl", hash = "sha256:048e05d0f6caeed70d731f3db756d35dcc1f35747c8c403364a8332c630441b8", size = 85584 },
]
```

`packages` is nice because the field is not a single entry, but a list.

2/3 for https://github.com/astral-sh/uv/issues/4893

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-08-08 11:25:06 -04:00
konsti
ae6b59365f
Only textwrap json packse scenarios with packse 0.3.32 (#5810)
Companion change to https://github.com/astral-sh/packse/pull/205 to
correctly format lock scenario doc comments.

Updates packse to 0.3.32.
2024-08-08 15:49:50 +02:00
Charlie Marsh
0f63173400
Add --python argument to benchmark script (#5844)
## Summary

Makes it easy to benchmark on different Python versions. Used this for
transformers, where some projects don't support Python 3.12.
2024-08-06 21:45:46 -04:00
Charlie Marsh
f3cc8e4790
Minor improvements to benchmarking setup (#5843) 2024-08-06 23:47:45 +00:00
Charlie Marsh
097aa929b7
Skip files when detecting workspace members (#5735)
## Summary

Closes https://github.com/astral-sh/uv/issues/5724.
Closes https://github.com/astral-sh/uv/issues/5725.
2024-08-02 19:44:43 +00:00
konsti
db371560bc
Use prettier to format the documentation (#5708)
To enforce the 100 character line limit in markdown files introduced in
https://github.com/astral-sh/uv/pull/5635, and to automate the
formatting of markdown files, i've added prettier and formatted our
markdown files with it.

I've excluded the changelog and the generated references documentation
from this for having too many changes, but we can also include them.

I'm not particular on which style we use. My main motivations are
(major) not having to reflow markdown files myself anymore and (minor)
consistence between all markdown files. I've chosen prettier for similar
reason as we chose black, it's a single good style that's automated and
shared in the community. I do prefer prettier's style of not breaking
inside of a link name though.

This PR is in two parts, the first adds prettier to CI and documents
using it, while the second actually formats the docs. When merge
conflicts arise, we can drop the last commit and regenerate it with `npx
prettier --prose-wrap always --write BENCHMARKS.md CONTRIBUTING.md
README.md STYLE.md docs/*.md docs/concepts/**/*.md docs/guides/**/*.md
docs/pip/**/*.md`.

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-08-02 08:58:31 -05:00
Charlie Marsh
7efed62798
Add a dedicated error for members with mismatched Python requirements (#5695)
## Summary

Gives you a nice error message if you attempt to sync with, e.g., `-p
3.8` when that version is supported by at least one workspace member,
but your project's minimum requirement is `>=3.12`

Closes https://github.com/astral-sh/uv/issues/5662.
2024-08-01 21:37:13 +00:00
konsti
9a1a2118e1
Add transformers pyproject.toml to test requirements (#5657)
This `pyproject.toml`, translated from
0e4a1c3401/setup.py,
has found a number of bugs in blue jay, so i'm adding it to uv as a
development test case. (The file is too large to make it a test by
itself.)
2024-08-01 13:25:20 +02:00
konsti
2247b0f540
Check idempotence in packse lock scenarios (#5485)
Add tests for the instabilities fix.

Part of
https://github.com/astral-sh/uv/issues/5180#issuecomment-2247696198

Closes #5180
2024-07-31 15:39:16 +00:00
Charlie Marsh
5d727cb0af
Deprecate the --isolated flag (#5466)
## Summary

This PR deprecates the `--isolated` flag. The treatment varies across
the APIs:

- For non-preview APIs, we warn but treat it as equivalent to
`--no-config`.
- For preview APIs, we warn and ignore it, with two exceptions...
- For `tool run` and `run` specifically, we don't even warn, because we
can't differentiate the command-specific `--isolated` from the global
`--isolated`.
2024-07-30 22:40:38 +00:00
Zanie Blue
8545ae2312
Rename more use of "lock file" to "lockfile" (#5629) 2024-07-30 19:09:43 +00:00
konsti
194904b340
Redact packse version in snapshots follow-up (#5563)
I thought i had this included in #5483 but i had it on the wrong branch.
2024-07-29 15:22:20 +00:00
konsti
0f87d174b9
Redact packse version in snapshots (#5483)
Every packse version update is currently causing a huge diff (the size
of the `lock_scenarios.rs` diff in this PR). By redacting the version
from the snapshots, we will only have the actual change in the diff and
not the redundant version change noise.

The second commit moves all remaining packse url arg values to
`common/mod.rs`, which acts as a single source of truth for the packse
version.
2024-07-29 17:04:46 +02:00
Charlie Marsh
a5866f44c4
Add a no-op resolution benchmark (#5558) 2024-07-29 10:37:12 -04:00
Charlie Marsh
3626d08cca
Enable benchmarking of uv tool and pipx (#5531)
## Summary

Closes https://github.com/astral-sh/uv/issues/5263.
2024-07-28 23:27:14 +00:00
Charlie Marsh
44a77a04d0
Move bench directory to benchmark (#5529)
## Summary

Removes the legacy `benchmark` directory (we'll always have it in Git)
and renames `bench` to `benchmark` for clarity. Fixes a variety of
commands and references.
2024-07-28 22:03:52 +00:00
Charlie Marsh
9732922929
Migrate benchmark setup to uv (#5526) 2024-07-28 21:37:57 +00:00
Charlie Marsh
efbc9fb78d
Add support for benchmarking uv sync and uv lock (#5524)
## Summary

This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.

Part of: https://github.com/astral-sh/uv/issues/5263.

## Test Plan

For example:

```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
2024-07-28 21:09:08 +00:00
Ahmed Ilyas
e8d7c0cb58
Editable installs for uv tool (#5454)
## Summary

Resolves #5436. 

## Test Plan

`cargo test` 

```console
❯ ./target/debug/uv tool install -e ~/black
warning: `uv tool install` is experimental and may change without warning
Resolved 6 packages in 894ms
   Built black @ file:///Users/ahmedilyas/black
Prepared 1 package in 468ms
Installed 6 packages in 6ms
 + black==24.4.3.dev23+g7e2afc9 (from file:///Users/ahmedilyas/black)
 + click==8.1.7
 + mypy-extensions==1.0.0
 + packaging==24.1
 + pathspec==0.12.1
 + platformdirs==4.2.2
Installed 2 executables: black, blackd
```

venv has the `.pth` files.
```console
❯ eza /Users/ahmedilyas/Library/Application\ Support/uv/tools/black/lib/python3.12/site-packages/
_black.pth       _virtualenv.py                         click                  mypy_extensions-1.0.0.dist-info  packaging                 pathspec                   platformdirs
_virtualenv.pth  black-24.4.3.dev23+g7e2afc9.dist-info  click-8.1.7.dist-info  mypy_extensions.py               packaging-24.1.dist-info  pathspec-0.12.1.dist-info  platformdirs-4.2.2.dist-info
```

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-26 16:30:15 -04:00
konsti
f1eda3590b
Update to packse 0.3.31 (#5441)
Update packse to 0.3.31, adding the instability scenarios.
2024-07-26 15:39:29 +00:00
Zanie Blue
42e76e2545
Prefer "lockfile" to "lock file" (#5427)
Closes https://github.com/astral-sh/uv/issues/5415
2024-07-25 09:22:36 -05:00
Charlie Marsh
f992532f78
Fix benchmark image in documentation (#5102) 2024-07-16 13:54:06 +00:00
Andrew Gallant
b6ad41fd8e scenarios: bump to packse 0.3.30 2024-07-15 10:09:01 -07:00
Charlie Marsh
b5f0e0729e
Use versioned bages when uploading to PyPI (#5039) 2024-07-13 18:12:45 +00:00
Charlie Marsh
b629ab89c5
Set absolute URLs prior to uploading to PyPI (#5038)
## Summary

Closes https://github.com/astral-sh/uv/issues/5030.
2024-07-13 17:29:21 +00:00
Charlie Marsh
6a6168ec78
Update SchemaStore script (#5024)
## Summary

Small mistakes I noticed while running.

## Test Plan

`python scripts/update_schemastore.py`
2024-07-12 17:49:38 -04:00
Charlie Marsh
a4417eba4a
Enable projects to opt-out of workspace management (#4565)
## Summary

You can now add `managed = false` under `[tool.uv]` in a
`pyproject.toml` to explicitly opt out of the project and workspace
APIs.

If a project sets `managed = false`, we will (1) _not_ discover it as a
workspace root, and (2) _not_ discover it as a workspace member (similar
to using `exclude` in the workspace parent).

Closes https://github.com/astral-sh/uv/issues/4551.
2024-07-01 16:17:43 -04:00
Charlie Marsh
41f051db3b
Remove exclude newer methods on test context (#4535)
## Summary

Closes https://github.com/astral-sh/uv/issues/4531.
2024-06-25 23:45:35 +00:00
konsti
e6103dcab1
Deduplicate test command creation (#4512)
This PR refactors the command creation in the test suite to remove the
duplication.

**1)** We add the same set of test stubbing args to almost any uv
invocation in the tests:

```rust
command
    .arg("--cache-dir")
    .arg(self.cache_dir.path())
    .env("VIRTUAL_ENV", self.venv.as_os_str())
    .env("UV_NO_WRAP", "1")
    .env("HOME", self.home_dir.as_os_str())
    .env("UV_TOOLCHAIN_DIR", "")
    .env("UV_TEST_PYTHON_PATH", &self.python_path())
    .current_dir(self.temp_dir.path());

if cfg!(all(windows, debug_assertions)) {
    // TODO(konstin): Reduce stack usage in debug mode enough that the tests pass with the
    // default windows stack of 1MB
    command.env("UV_STACK_SIZE", (8 * 1024 * 1024).to_string());
}
```

Centralizing these into a `TestContext::add_shared_args` method removes
them from everywhere.

**2)** Prefix all `TextContext` methods of the pip interface with
`pip_`. This is now necessary due to `uv sync` vs. `uv pip sync`.

**3)** Move command creation in the various test files into dedicated
functions or methods to avoid repeating the arguments. Except for error
message tests, there should be at most one `Command::new(get_bin())`
call per test file. `EXCLUDE_NEWER` is exclusively used in
`TestContext`.

---

I'm considering adding a `TestCommand` on top of these changes (in
another PR) that holds a reference to the `TextContext`, has
`add_shared_args` as a method and uses `Fn(Self) -> Self` instead of
`Fn(&mut Self) -> Self` for methods to improved chaining.
2024-06-25 22:06:54 +00:00
Andrew Gallant
10c950a551 scenarios: bump to packse 0.3.29 2024-06-20 07:21:45 -04:00
Zanie Blue
549d7dfe37
Add test case for wheel installation with different path (#4396)
Regression test for #4391 / https://github.com/astral-sh/uv/pull/4393
2024-06-19 09:39:55 -05:00
Zanie Blue
0b0a0472ad
Request Python 3.12 in the release script (#4384)
Otherwise, if another version is active we fail because Rooster requires
3.12
2024-06-18 10:44:42 -05:00
Andrew Gallant
bb40d13603 scenarios: bump to packse 0.3.27 2024-06-17 09:30:37 -04:00
Andrew Gallant
eea1bf9bf2 scenarios: bump to packse 0.3.26 2024-06-17 09:30:37 -04:00
Markus Hofbauer
9647eed697
Rename README to README.md (#4315)
Make use of GitHub's markdown rendering

<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
2024-06-13 16:43:11 -05:00
Zanie Blue
434aef0325
Move the preview changelog so the GitHub Release shows stable changes (#4290) 2024-06-12 21:49:12 +00:00
Zanie Blue
d963a0f085
Improve local testing docs for packse (#4279) 2024-06-12 13:19:10 -05:00
Andrew Gallant
910168a219 scenarios: bump packse to 0.3.24 2024-06-12 13:30:47 -04:00
Andrew Gallant
1e282da04a scripts/scenarios: add requires-python to packse lock test template
packse has the ability to specify a project wide Requires-Python
constraint, but our lock template wasn't forwarding this to the
corresponding pyproject.toml. This update makes that happen.
2024-06-12 13:30:47 -04:00
Zanie Blue
daee30aeb1
Add changelog for preview changes (#4251)
I tweaked rooster to allow sections to be overridden from the CLI so we
can generate a separate preview changelog

See https://github.com/zanieb/rooster/pull/43 for the rooster changes
needed

I tested `./scripts/release.sh` as well.
2024-06-11 19:16:29 -04:00
Andrew Gallant
c46fa74e65
make universal resolver fork only when markers are disjoint (#4135)
The basic idea here is to make it so forking can only ever result in a
resolution that, for a particular marker environment, will only install
at most one version of a package. We can guarantee this by ensuring we
only fork on conflicting dependency specifications only when their
corresponding markers are completely disjoint. If they aren't, then
resolution _must_ find a single version of the package in the
intersection of the two dependency specifications.

A test for this case has been added to packse here:
https://github.com/astral-sh/packse/pull/182. Previously, that test
would result in a resolution with two different unconditional versions
of the same package. With this change, resolution fails (as it should).

A commit-by-commit review should be helpful here, since the first commit
is a refactor to make the second commit a bit more digestible.
2024-06-07 23:40:55 +00:00
Charlie Marsh
e5f95186de
Default to current Python minor if Requires-Python is absent (#4070)
## Summary

If `Requires-Python` is omitted in `uv lock` or `uv run`, we now warn
and default to `>=` the current minor version.

Closes https://github.com/astral-sh/uv/issues/4050.
2024-06-05 20:45:50 +00:00
Andrew Gallant
42b584c668 scripts/scenarios: update packse to 0.3.17 2024-06-04 14:24:59 -04:00
Andrew Gallant
7b736fc238 scripts/scenarios: update 'generate.py' to handle universal tests
This commit adds a template and does some light surgery on `generate.py`
to make use of that template. In particular, the universal tests require
using the "workspace"-aware version of `uv`, so we can't use the
existing `uv pip {compile,install}` tests.
2024-06-04 14:24:59 -04:00
Andrew Gallant
459966a132
uv/tests: update packse tests (#4015)
This is just the result of running

    ./scripts/sync_scenarios.sh

From the root of the `uv` repository.

When I initially ran this, it produced some tests with snapshots that
weren't being updated. It turned out this was because the tests weren't
running, as they were gated behind the `python-patch` feature. In this
commit, we add `python-patch` to our `cargo insta` command, which should
update all relevant snapshots.

There are still some superfluous updates as a result of a spell checker
being run on generated files, but
2024-06-04 13:56:20 -04:00
konsti
da7d5549a3
Don't copy gitignored files in workspace tests (#4012)
The workspace test directories can be used both in tests and directly
for developing/debugging. In the latter, we shouldn't copy the venv and
the lockfile when running tests. Using the ignore crate over manual
recursion we exclude those files.
2024-06-04 12:58:07 +00:00
Charlie Marsh
1fc6a59707
Remove special-casing for editable requirements (#3869)
## Summary

There are a few behavior changes in here:

- We now enforce `--require-hashes` for editables, like pip. So if you
use `--require-hashes` with an editable requirement, we'll reject it. I
could change this if it seems off.
- We now treat source tree requirements, editable or not (e.g., both `-e
./black` and `./black`) as if `--refresh` is always enabled. This
doesn't mean that we _always_ rebuild them; but if you pass
`--reinstall`, then yes, we always rebuild them. I think this is an
improvement and is close to how editables work today.

Closes #3844.

Closes #2695.
2024-05-28 15:49:34 +00:00