Commit graph

21 commits

Author SHA1 Message Date
Charlie Marsh
42dcea0ee2
Bump MSRV to 1.84 (#12670)
## Summary

Closes https://github.com/astral-sh/uv/issues/12649.
2025-04-04 11:49:26 -04:00
Ankit Saini
ef95d79bfa
Fix version string truncation while generating cache_key (#11830)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Follow up for https://github.com/astral-sh/uv/pull/11738

I missed this while reviewing the truncation changes. 

`format!("{:.N}", value)` only truncates if the `fmt::Display`
implementation supports it (by reading `f.precision()` in trait
implementation).

So in our case `format!("{:.N}", version.to_string())` will work but not
`format!("{:.N}", version)` unless `Version` supports it.

Since we only need it once, I am just truncating after the string is
created.

## Test Plan

<!-- How was it tested? -->
2025-02-27 08:48:43 -05:00
Ankit Saini
fb35875f24
Use hash instead of full wheel name in wheels bucket (#11738)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->


## Summary
Closes #2410 
<!-- What's the purpose of the change? What does it do, and why? -->
This changes the name of files in `wheels` bucket to use a hash instead
of the wheel name as to not exceed maximum file length limit on various
systems.

This only addresses the primary concern of #2410. It still does _not_
address:
- Path limit of 260 on windows:
https://github.com/astral-sh/uv/issues/2410#issuecomment-2062020882
To solve this we need to opt-in to longer path limits on windows
([ref](https://github.com/astral-sh/uv/issues/2410#issuecomment-2150532658)),
but I think that is a separate issue and should be a separate MR.
- Exceeding filename limit while building a wheel from source
distribution
As per my understanding, this is out of uv's control. Name of the output
wheel will be decided by build-backend used by the project. For wheels
built from source distribution, pip also uses the wheel names in cache.
So I have not touched `sdists` cache.


I have added a `filename: WheelFileName` field in `Archive`, so we can
use it while indexing instead of relying on the filename on disk.
Another way to do this was to read `.dist-info/WHEEL` and
`.dist-info/METADATA` and build `WheelFileName` but that seems less
robust and will be slower.
## Test Plan

<!-- How was it tested? -->
Tested by installing `yt-dlp`, `httpie` and `sqlalchemy` and verifying
that cache files in `wheels` bucket use hash.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-02-26 22:41:57 +00:00
Charlie Marsh
c37af945b3
Avoid using owned String in deserializers (#11764)
## Summary

This is the pattern I see in a variety of crates, and I believe this is
preferred if you don't _need_ an owned `String`, since you can avoid the
allocation. This could be pretty impactful for us?
2025-02-25 14:28:16 +00:00
Charlie Marsh
4d5041dc00 Use files instead of junctions on Windows (#11269)
Instead of using junctions, we can just write files that contain (as the
file contents) the target path. This requires a little more finesse in
that, as readers, we need to know where to expect these. But it also
means we get to avoid junctions, which have led to a variety of
confusing behaviors. Further, `replace_symlink` should now be on atomic
on Windows.

Closes #11263.
2025-02-13 16:17:49 -06:00
Charlie Marsh
dce7b9da13
Store unsupported tags in wheel filename (#10665)
Some checks are pending
CI / check system | alpine (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86_64 (push) Blocked by required conditions
CI / check system | python3.10 on windows (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | conda3.11 on linux (push) Blocked by required conditions
CI / check system | conda3.8 on linux (push) Blocked by required conditions
CI / check system | conda3.11 on macos (push) Blocked by required conditions
CI / check system | conda3.8 on macos (push) Blocked by required conditions
CI / check system | conda3.11 on windows (push) Blocked by required conditions
CI / check system | conda3.8 on windows (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
## Summary

We can retain the small-size advantage of our new tags by moving the
"unknown tag" case into `WheelTagLarge`. This ensures that we can still
represent unknown tags, but avoid paying the cost for them.
2025-01-17 04:41:53 +00:00
Charlie Marsh
ed34d37e3c
Avoid failing when deserializing unknown tags (#10655)
Some checks are pending
CI / integration test | uv publish (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86_64 (push) Blocked by required conditions
CI / check system | python3.10 on windows (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on linux (push) Blocked by required conditions
CI / check system | conda3.8 on linux (push) Blocked by required conditions
CI / check system | conda3.11 on macos (push) Blocked by required conditions
CI / check system | conda3.8 on macos (push) Blocked by required conditions
CI / check system | conda3.11 on windows (push) Blocked by required conditions
CI / check system | conda3.8 on windows (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
## Summary

Closes
https://github.com/astral-sh/uv/issues/10654#issuecomment-2594022975.
2025-01-15 18:03:28 -05:00
Charlie Marsh
d20a48a5b4
Use memchr for wheel parsing (#10620)
## Summary

Before:

```
wheelname_parsing/numpy-compatible
                        time:   [106.90 ns 107.86 ns 108.86 ns]
                        thrpt:  [402.97 MiB/s 406.71 MiB/s 410.39 MiB/s]
                 change:
                        time:   [-4.5360% -3.7694% -2.9179%] (p = 0.00 < 0.05)
                        thrpt:  [+3.0056% +3.9170% +4.7515%]
                        Performance has improved.
wheelname_parsing/flyte-short-incompatible
                        time:   [81.439 ns 82.209 ns 83.015 ns]
                        thrpt:  [390.59 MiB/s 394.42 MiB/s 398.15 MiB/s]
                 change:
                        time:   [+6.2344% +7.5385% +8.8928%] (p = 0.00 < 0.05)
                        thrpt:  [-8.1666% -7.0101% -5.8685%]
                        Performance has regressed.
wheelname_parsing/flyte-short-compatible
                        time:   [78.909 ns 79.456 ns 80.031 ns]
                        thrpt:  [357.49 MiB/s 360.08 MiB/s 362.57 MiB/s]
                 change:
                        time:   [+3.2653% +4.1733% +5.1062%] (p = 0.00 < 0.05)
                        thrpt:  [-4.8582% -4.0061% -3.1620%]
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
wheelname_parsing/flyte-long-incompatible
                        time:   [353.35 ns 357.49 ns 361.77 ns]
                        thrpt:  [340.06 MiB/s 344.13 MiB/s 348.17 MiB/s]
                 change:
                        time:   [+1.4846% +2.0228% +2.6504%] (p = 0.00 < 0.05)
                        thrpt:  [-2.5820% -1.9827% -1.4629%]
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
wheelname_parsing/flyte-long-compatible
                        time:   [256.47 ns 258.12 ns 260.17 ns]
                        thrpt:  [417.88 MiB/s 421.20 MiB/s 423.90 MiB/s]
                 change:
                        time:   [+0.4079% +1.8252% +3.6270%] (p = 0.02 < 0.05)
                        thrpt:  [-3.5001% -1.7925% -0.4063%]
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
```

After:

```
wheelname_parsing_fastest/numpy-compatible
                        time:   [61.500 ns 61.904 ns 62.350 ns]
                        thrpt:  [703.60 MiB/s 708.66 MiB/s 713.32 MiB/s]
                 change:
                        time:   [+0.9879% +1.4542% +1.9311%] (p = 0.00 < 0.05)
                        thrpt:  [-1.8945% -1.4334% -0.9782%]
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe
wheelname_parsing_fastest/flyte-short-incompatible
                        time:   [49.341 ns 49.538 ns 49.769 ns]
                        thrpt:  [651.50 MiB/s 654.54 MiB/s 657.16 MiB/s]
                 change:
                        time:   [+5.8750% +6.3656% +6.8338%] (p = 0.00 < 0.05)
                        thrpt:  [-6.3967% -5.9847% -5.5490%]
                        Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
  5 (5.00%) low severe
  1 (1.00%) low mild
  6 (6.00%) high mild
  5 (5.00%) high severe
wheelname_parsing_fastest/flyte-short-compatible
                        time:   [49.425 ns 49.789 ns 50.193 ns]
                        thrpt:  [570.01 MiB/s 574.63 MiB/s 578.86 MiB/s]
                 change:
                        time:   [+5.1267% +5.7418% +6.3476%] (p = 0.00 < 0.05)
                        thrpt:  [-5.9687% -5.4300% -4.8767%]
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high mild
wheelname_parsing_fastest/flyte-long-incompatible
                        time:   [295.81 ns 298.01 ns 301.04 ns]
                        thrpt:  [408.66 MiB/s 412.82 MiB/s 415.89 MiB/s]
                 change:
                        time:   [+0.5553% +1.0842% +1.7059%] (p = 0.00 < 0.05)
                        thrpt:  [-1.6772% -1.0726% -0.5523%]
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
wheelname_parsing_fastest/flyte-long-compatible
                        time:   [214.80 ns 216.10 ns 217.50 ns]
                        thrpt:  [499.87 MiB/s 503.10 MiB/s 506.15 MiB/s]
                 change:
                        time:   [+0.9003% +1.3207% +1.8268%] (p = 0.00 < 0.05)
                        thrpt:  [-1.7940% -1.3035% -0.8923%]
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe
```

(Ignore the percent changes; they're not relative to one another.)

So it's like >40% faster for the small case and >15% faster for the
large case.
2025-01-15 11:39:25 -05:00
Charlie Marsh
a7fe84aa03
Avoid allocating when stripping source distribution extension (#10625) 2025-01-15 03:29:08 +00:00
Charlie Marsh
279043f864
Include build tag in rendered wheel filenames (#10599)
## Summary

I don't think this had an impact in practice, but it is "wrong" to omit
these. Confirmed that the cache (for example) now includes the build tag
(as in, `mkl_fft-1.3.8-72-cp310-cp310-manylinux2014_x86_64`).
2025-01-14 15:01:45 +00:00
Charlie Marsh
24a5920739
Reduce WheelFilename to 48 bytes (#10583)
## Summary

Based on some advice from @konstin.
2025-01-14 14:49:17 +00:00
Charlie Marsh
faa4481ccc
Shrink size of platform tag enum (#10546)
Some checks are pending
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86_64 (push) Blocked by required conditions
CI / check system | python3.10 on windows (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on linux (push) Blocked by required conditions
CI / check system | conda3.8 on linux (push) Blocked by required conditions
CI / check system | conda3.11 on macos (push) Blocked by required conditions
CI / check system | conda3.8 on macos (push) Blocked by required conditions
CI / check system | conda3.11 on windows (push) Blocked by required conditions
CI / check system | conda3.8 on windows (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
## Summary

Reduces it from 56 bytes to 16 bytes.
2025-01-14 03:13:59 +00:00
Charlie Marsh
5c91217488
Use structured wheel tags everywhere (#10542)
## Summary

This PR extends the thinking in #10525 to platform tags, and then uses
the structured tag enums everywhere, rather than passing around strings.
I think this is a big improvement! It means we're no longer doing ad hoc
tag parsing all over the place.
2025-01-14 01:39:39 +00:00
Charlie Marsh
e0e8ba582a
Use structured types for parsing and formatting language and ABI tags (#10525)
## Summary

I need to be able to do non-lexicographic comparisons between tags
(e.g., so I can sort `cp313` as greater than `cp39`). It ended up being
easiest to just create structured types for all the tags we support,
with `FromStr` and `Display` implementations.

We don't currently store these in `Tags` or in `WheelFilename`. We may
want to, since they're really small (and `Copy`), but I need to
benchmark to determine whether parsing these in `WheelFilename` is
prohibitively slow.
2025-01-13 19:49:43 -05:00
Charlie Marsh
9e948b7363
Remove resolved build tag TODO (#10526) 2025-01-11 23:12:11 +00:00
Charlie Marsh
3aaa9594be
Add test coverage for build tag prioritization (#9680)
Some checks are pending
CI / integration test | uv publish (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86_64 (push) Blocked by required conditions
CI / check system | python3.10 on windows (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on linux (push) Blocked by required conditions
CI / check system | conda3.8 on linux (push) Blocked by required conditions
CI / check system | conda3.11 on macos (push) Blocked by required conditions
CI / check system | conda3.8 on macos (push) Blocked by required conditions
CI / check system | conda3.11 on windows (push) Blocked by required conditions
CI / check system | conda3.8 on windows (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
## Summary

See: https://github.com/astral-sh/uv/pull/3781 and
https://github.com/astral-sh/uv/pull/9677.
2024-12-06 09:32:19 -05:00
konsti
566c178276
uv build: Catch version mismatch between sdist and wheel (#9633)
When building a wheel from a source distribution or both a source
distribution and a wheel, the versions in their filenames must be the
same.

By inspecting the filenames, we also assert that the filenames from the
build a valid (we don't enforce normalization though, just that uv can
parse them).

Note that we're not yet checking that also the `pyproject.toml` version,
if declared, and METADATA version matches.
2024-12-04 15:21:01 +01:00
Charlie Marsh
d08bfee718
Remove separate test files in favor of same-file mod tests (#9199)
## Summary

These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
2024-11-18 20:11:46 +00:00
Charlie Marsh
29e1b15473
Add uv tree --outdated (#8893)
## Summary

Similar to `pip list --outdated`, but for `uv tree`.

## Test Plan

Looks like:

```
foo v0.1.0
└── flask v2.0.0 (latest: v3.0.3)
    ├── click v8.1.7
    ├── itsdangerous v2.2.0
    ├── jinja2 v3.1.4
    │   └── markupsafe v3.0.2
    └── werkzeug v3.1.2
        └── markupsafe v3.0.2
```

With `(latest: v3.0.3)` in bold cyan.
2024-11-07 14:10:46 -06:00
Amos Wenger
715f28fd39
chore: Move all integration tests to a single binary (#8093)
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html

Before that, there were 91 separate integration tests binary.

(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
2024-10-11 16:41:35 +02:00
Charlie Marsh
14507a1793
Add uv- prefix to all internal crates (#7853)
## Summary

Brings more consistency to the repo and ensures that all crates
automatically show up in `--verbose` logging.
2024-10-01 20:15:32 -04:00