Replace Python bootstrapping script with Rust implementation (#2842)

See https://github.com/astral-sh/uv/issues/2617

Note this also includes:
- #2918 
- #2931 (pending)

A first step towards Python toolchain management in Rust.

First, we add a new crate to manage Python download metadata:

- Adds a new `uv-toolchain` crate
- Adds Rust structs for Python version download metadata
- Duplicates the script which downloads Python version metadata
- Adds a script to generate Rust code from the JSON metadata
- Adds a utility to download and extract the Python version

I explored some alternatives like a build script using things like
`serde` and `uneval` to automatically construct the code from our
structs but deemed it to heavy. Unlike Rye, I don't generate the Rust
directly from the web requests and have an intermediate JSON layer to
speed up iteration on the Rust types.

Next, we add add a `uv-dev` command `fetch-python` to download Python
versions per the bootstrapping script.

- Downloads a requested version or reads from `.python-versions`
- Extracts to `UV_BOOTSTRAP_DIR`
- Links executables for path extension

This command is not really intended to be user facing, but it's a good
PoC for the `uv-toolchain` API. Hash checking (via the sha256) isn't
implemented yet, we can do that in a follow-up.

Finally, we remove the `scripts/bootstrap` directory, update CI to use
the new command, and update the CONTRIBUTING docs.

<img width="1023" alt="Screenshot 2024-04-08 at 17 12 15"
src="57bd3cf1-7477-4bb8-a8e9-802a00d772cb">
This commit is contained in:
Zanie Blue 2024-04-10 11:22:41 -05:00 committed by GitHub
parent 7cd98d2499
commit 44e39bdca3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
31 changed files with 8170 additions and 3703 deletions

View file

@ -21,6 +21,7 @@ use resolve_many::ResolveManyArgs;
use crate::build::{build, BuildArgs};
use crate::clear_compile::ClearCompileArgs;
use crate::compile::CompileArgs;
use crate::fetch_python::FetchPythonArgs;
use crate::render_benchmarks::RenderBenchmarksArgs;
use crate::resolve_cli::ResolveCliArgs;
use crate::wheel_metadata::WheelMetadataArgs;
@ -44,6 +45,7 @@ static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
mod build;
mod clear_compile;
mod compile;
mod fetch_python;
mod render_benchmarks;
mod resolve_cli;
mod resolve_many;
@ -72,6 +74,8 @@ enum Cli {
Compile(CompileArgs),
/// Remove all `.pyc` in the tree.
ClearCompile(ClearCompileArgs),
/// Fetch Python versions for testing
FetchPython(FetchPythonArgs),
}
#[instrument] // Anchor span to check for overhead
@ -92,6 +96,7 @@ async fn run() -> Result<()> {
Cli::RenderBenchmarks(args) => render_benchmarks::render_benchmarks(&args)?,
Cli::Compile(args) => compile::compile(args).await?,
Cli::ClearCompile(args) => clear_compile::clear_compile(&args)?,
Cli::FetchPython(args) => fetch_python::fetch_python(args).await?,
}
Ok(())
}