Regenerate benchmarks (#1305)

This commit is contained in:
Charlie Marsh 2024-02-15 12:54:04 -05:00 committed by GitHub
parent 2586f655bb
commit 55808a451f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 10 additions and 13 deletions

View file

@ -26,7 +26,7 @@ Benchmarking package installation (e.g., `uv pip sync`) with a warm cache. This
to removing and recreating a virtual environment, and then populating it with dependencies that
you've installed previously on the same machine.
![install-warm](https://github.com/astral-sh/ruff/assets/1309177/b6cb8d48-52e0-45c2-ae15-0a3f69ec3263)
![install-warm](./assets/png/install-warm.png)
## Cold Installation
@ -34,7 +34,7 @@ Benchmarking package installation (e.g., `uv pip sync`) with a cold cache. This
to running `uv pip sync` on a new machine or in CI (assuming that the package manager cache is
not shared across runs).
![install-cold](https://github.com/astral-sh/ruff/assets/1309177/ed86c193-582f-4163-b369-f12ec3905c3c)
![install-cold](./assets/png/install-cold.png)
## Warm Resolution
@ -42,7 +42,7 @@ Benchmarking dependency resolution (e.g., `uv pip compile`) with a warm cache, b
lockfile. This is equivalent to blowing away an existing `requirements.txt` file to regenerate it
from a `requirements.in` file.
![resolve-warm](https://github.com/astral-sh/ruff/assets/1309177/a4ca9d23-1148-4103-abe7-a35fa488409d)
![resolve-warm](./assets/png/resolve-warm.png)
## Cold Resolution
@ -50,7 +50,7 @@ Benchmarking dependency resolution (e.g., `uv pip compile`) with a cold cache. T
equivalent to running `uv pip compile` on a new machine or in CI (assuming that the package
manager cache is not shared across runs).
![resolve-cold](https://github.com/astral-sh/ruff/assets/1309177/556ac7aa-0a6a-4f94-b0d9-90b25461de7b)
![resolve-cold](./assets/png/resolve-cold.png)
## Reproduction
@ -64,27 +64,28 @@ The benchmark script itself has a several requirements:
- A virtual environment with the script's own dependencies installed (`uv venv && uv pip sync scripts/bench/requirements.txt`).
- The [`hyperfine`](https://github.com/sharkdp/hyperfine) command-line tool installed on your system.
To benchmark uv's resolution against pip-compile and Poetry:
To benchmark resolution against pip-compile, Poetry, and PDM:
```shell
python -m scripts.bench \
--uv \
--poetry \
--pdm \
--pip-compile \
--benchmark resolve-warm \
--benchmark resolve-warm --benchmark resolve-cold \
scripts/requirements/trio.in \
--json
```
To benchmark uv's installation against pip-sync and Poetry:
To benchmark installation against pip-sync, Poetry, and PDM:
```shell
python -m scripts.bench \
--uv \
--poetry \
--pdm \
--pip-sync \
--benchmark install-warm \
scripts/requirements/compiled/trio.txt \
--benchmark install-warm --benchmark install-cold \
--json
```

BIN
assets/png/install-cold.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 145 KiB

BIN
assets/png/install-warm.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 175 KiB

BIN
assets/png/resolve-cold.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

BIN
assets/png/resolve-warm.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

View file

@ -2186,10 +2186,6 @@ fn sync_editable() -> Result<()> {
.iter()
.chain(&[
(filter_path.as_str(), "requirements.txt"),
(
r"file://.*/../../scripts/editable-installs/poetry_editable",
"file://[TEMP_DIR]/../../scripts/editable-installs/poetry_editable",
),
(&workspace_url, "file://[WORKSPACE_DIR]"),
])
.copied()