mirrors/uv
1
0
Fork 0
mirror of https://github.com/astral-sh/uv.git synced 2025-08-04 19:08:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
uv/crates
History
Charlie Marsh 85162d1111
Parallelize wheel installations with Rayon (#84) 
It looks like using _either_ async Rust with a `JoinSet` _or_
parallelizing a fixed threadpool with Rayon provide about a ~5% speed-up
over our current serial approach:

```console
❯ hyperfine --runs 30 --warmup 5 --prepare "./target/release/puffin venv .venv" \
  "./target/release/rayon sync ./scripts/benchmarks/requirements-large.txt" \
  "./target/release/async sync ./scripts/benchmarks/requirements-large.txt" \
  "./target/release/main sync ./scripts/benchmarks/requirements-large.txt"
Benchmark 1: ./target/release/rayon sync ./scripts/benchmarks/requirements-large.txt
  Time (mean ± σ):     295.7 ms ±  16.9 ms    [User: 28.6 ms, System: 263.3 ms]
  Range (min … max):   249.2 ms … 315.9 ms    30 runs

Benchmark 2: ./target/release/async sync ./scripts/benchmarks/requirements-large.txt
  Time (mean ± σ):     296.2 ms ±  20.2 ms    [User: 36.1 ms, System: 340.1 ms]
  Range (min … max):   258.0 ms … 359.4 ms    30 runs

Benchmark 3: ./target/release/main sync ./scripts/benchmarks/requirements-large.txt
  Time (mean ± σ):     306.6 ms ±  19.5 ms    [User: 25.3 ms, System: 220.5 ms]
  Range (min … max):   269.6 ms … 332.2 ms    30 runs

Summary
  './target/release/rayon sync ./scripts/benchmarks/requirements-large.txt' ran
    1.00 ± 0.09 times faster than './target/release/async sync ./scripts/benchmarks/requirements-large.txt'
    1.04 ± 0.09 times faster than './target/release/main sync ./scripts/benchmarks/requirements-large.txt'
```

It's much easier to just parallelize with Rayon and avoid async in the
underlying wheel code, so this PR takes that approach for now.
2023-10-10 23:46:30 -04:00
..
gourgeist Add puffin venv command to create virtual environments (#83) 2023-10-10 13:46:25 -04:00
install-wheel-rs Parallelize wheel installations with Rayon (#84) 2023-10-10 23:46:30 -04:00
pep440-rs Enable release builds via cargo-dist (#79) 2023-10-09 20:48:55 +00:00
pep508-rs Enable release builds via cargo-dist (#79) 2023-10-09 20:48:55 +00:00
platform-host Add a basic test for the resolver (#86) 2023-10-11 03:30:53 +00:00
platform-tags Add a basic test for the resolver (#86) 2023-10-11 03:30:53 +00:00
puffin-cli Add a separate dist-info name struct (#85) 2023-10-10 23:21:18 +00:00
puffin-client Add a basic test for the resolver (#86) 2023-10-11 03:30:53 +00:00
puffin-installer Parallelize wheel installations with Rayon (#84) 2023-10-10 23:46:30 -04:00
puffin-interpreter Rework puffin sync output to summarize (#81) 2023-10-10 03:29:09 +00:00
puffin-package Add a separate dist-info name struct (#85) 2023-10-10 23:21:18 +00:00
puffin-resolver Add a basic test for the resolver (#86) 2023-10-11 03:30:53 +00:00
wheel-filename Enable release builds via cargo-dist (#79) 2023-10-09 20:48:55 +00:00
README.md Use local copy of install-wheel-rs (#34) 2023-10-07 01:43:55 +00:00

README.md

Crates

pep440-rs

Utilities for interacting with Python version numbers and specifiers.

pep508-rs

Utilities for interacting with PEP 508 dependency specifiers.

platform-host

Functionality for detecting the current platform (operating system, architecture, etc.).

platform-tags

Functionality for parsing and inferring Python platform tags as per PEP 425.

puffin-cli

Command-line interface for the Puffin package manager.

puffin-client

Client for interacting with PyPI-compatible HTTP APIs.

puffin-installer

Functionality for installing Python packages into a virtual environment.

puffin-interpreter

Functionality for detecting and leveraging the current Python interpreter.

puffin-package

Types and functionality for working with Python packages, e.g., parsing wheel files.

puffin-resolver

Functionality for resolving Python packages and their dependencies.

wheel-filename

Functionality for parsing wheel filenames as per PEP 427.