mirror of
https://github.com/astral-sh/uv.git
synced 2025-08-04 10:58:28 +00:00
2a53e789b0
Add a `--compile` option to `pip install` and `pip sync`. I chose to implement this as a separate pass over the entire venv. If we wanted to compile during installation, we'd have to make sure that writing is exclusive, to avoid concurrent processes writing broken `.pyc` files. Additionally, this ensures that the entire site-packages are bytecode compiled, even if there are packages that aren't from this `uv` invocation. The disadvantage is that we do not update RECORD and rely on this comment from [PEP 491](https://peps.python.org/pep-0491/): > Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD. If this is a problem we can change it to run during installation and write RECORD entries. Internally, this is implemented as an async work-stealing subprocess worker pool. The producer is a directory traversal over site-packages, sending each `.py` file to a bounded async FIFO queue/channel. Each worker has a long-running python process. It pops the queue to get a single path (or exists if the channel is closed), then sends it to stdin, waits until it's informed that the compilation is done through a line on stdout, and repeat. This is fast, e.g. installing `jupyter plotly` on Python 3.12 it processes 15876 files in 319ms with 32 threads (vs. 3.8s with a single core). The python processes internally calls `compileall.compile_file`, the same as pip. Like pip, we ignore and silence all compilation errors (https://github.com/astral-sh/uv/issues/1559). There is a 10s timeout to handle the case when the workers got stuck. For the reviewers, please check if i missed any spots where we could deadlock, this is the hardest part of this PR. I've added `uv-dev compile <dir>` and `uv-dev clear-compile <dir>` commands, mainly for my own benchmarking. I don't want to expose them in `uv`, they almost certainly not the correct workflow and we don't want to support them. Fixes #1788 Closes #1559 Closes #1928 |
||
|---|---|---|
| .. | ||
| src | ||
| .gitignore | ||
| builder.dockerfile | ||
| Cargo.toml | ||
| test_sdist_building.sh | ||