uv/crates
Charlie Marsh 2a846e76b7
Store unzipped wheels in a cache (#49)
This PR massively speeds up the case in which you need to install wheels
that already exist in the global cache.

The new strategy is as follows:

- Download the wheel into the content-addressed cache.
- Unzip the wheel into the cache, but ignore content-addressing. It
turns out that writing to `cacache` for every file in the zip added a
ton of overhead, and I don't see any actual advantages to doing so.
Instead, we just unzip the contents into a directory at, e.g.,
`~/.cache/puffin/django-4.1.5`.
- (The unzip itself is now parallelized with Rayon.)
- When installing the wheel, we now support unzipping from a directory
instead of a zip archive. This required duplicating and tweaking a few
functions.
- When installing the wheel, we now use reflinks (or copy-on-write
links). These have a few fantastic properties: (1) they're extremely
cheap to create (on macOS, they are allegedly faster than hard links);
(2) they minimize disk space, since we avoid copying files entirely in
the vast majority of cases; and (3) if the user then edits a file
locally, the cache doesn't get polluted. Orogene, Bun, and soon pnpm all
use reflinks.

Puffin is now ~15x faster than `pip` for the common case of installing
cached data into a fresh environment.

Closes https://github.com/astral-sh/puffin/issues/21.

Closes https://github.com/astral-sh/puffin/issues/39.
2023-10-08 04:04:48 +00:00
..
install-wheel-rs Store unzipped wheels in a cache (#49) 2023-10-08 04:04:48 +00:00
pep440-rs Use local versions of PEP 440 and PEP 508 crates (#32) 2023-10-07 00:16:44 +00:00
pep508-rs Use local versions of PEP 440 and PEP 508 crates (#32) 2023-10-07 00:16:44 +00:00
platform-host Use local copy of install-wheel-rs (#34) 2023-10-07 01:43:55 +00:00
platform-tags Use local copy of install-wheel-rs (#34) 2023-10-07 01:43:55 +00:00
puffin-cli Surface error when unable to find package (#45) 2023-10-07 19:43:12 +00:00
puffin-client Surface error when unable to find package (#45) 2023-10-07 19:43:12 +00:00
puffin-installer Store unzipped wheels in a cache (#49) 2023-10-08 04:04:48 +00:00
puffin-interpreter Skip already-installed dependencies during sync command (#43) 2023-10-07 19:26:45 +00:00
puffin-package Skip already-installed dependencies during sync command (#43) 2023-10-07 19:26:45 +00:00
puffin-resolver Surface error when unable to find package (#45) 2023-10-07 19:43:12 +00:00
wheel-filename Use local copy of install-wheel-rs (#34) 2023-10-07 01:43:55 +00:00
README.md Use local copy of install-wheel-rs (#34) 2023-10-07 01:43:55 +00:00

Crates

pep440-rs

Utilities for interacting with Python version numbers and specifiers.

pep508-rs

Utilities for interacting with PEP 508 dependency specifiers.

platform-host

Functionality for detecting the current platform (operating system, architecture, etc.).

platform-tags

Functionality for parsing and inferring Python platform tags as per PEP 425.

puffin-cli

Command-line interface for the Puffin package manager.

puffin-client

Client for interacting with PyPI-compatible HTTP APIs.

puffin-installer

Functionality for installing Python packages into a virtual environment.

puffin-interpreter

Functionality for detecting and leveraging the current Python interpreter.

puffin-package

Types and functionality for working with Python packages, e.g., parsing wheel files.

puffin-resolver

Functionality for resolving Python packages and their dependencies.

wheel-filename

Functionality for parsing wheel filenames as per PEP 427.