Refactor fetch-download-metadata.py script (#4853)

## Summary

Similiar to https://github.com/astral-sh/rye/pull/680, I have made a
major refactor to the `fetch-download-metadata.py` script.

Some notable changes:

- Use PEP 723 inline scripts
- Fully type annotated the script
- Implemented async HTTP fetching
- Introduced a `Finder` base class and move finder logic under
`CPythonFinder` subclass, which will make it easier to add a
`PyPyFinder` later.
- Instead of fetching `xxx.sha256` for each file, the script now fetches
a single `SHA256SUMS` file containing checksums for all files in the
release.

As a result, the script now takes around 10 seconds instead of 10+
minutes.

## Plan for Future PRs

- [ ] Implement the `PyPyFinder`
- [ ] Add an GitHub Action to run `fetch-download-metadata.py` daily and
create PR automatically

## Test Plan

```sh
cargo run -- run --isolated -- ./crates/uv-python/fetch-download-metadata.py
```
This commit is contained in:
Jo 2024-07-23 11:06:25 +08:00 committed by GitHub
parent f371195536
commit 0a6efe4d26
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 359 additions and 244 deletions

View file

@ -1,4 +1,9 @@
#!/usr/bin/env python3.12
# /// script
# requires-python = ">=3.12"
# dependencies = [
# "chevron-blue < 1",
# ]
# ///
"""
Generate static Rust code from Python version download metadata.
@ -6,7 +11,7 @@ Generates the `downloads.inc` file from the `downloads.inc.mustache` template.
Usage:
uv run --isolated --with chevron-blue -- crates/uv-python/template-download-metadata.py
uv run --isolated -- crates/uv-python/template-download-metadata.py
"""
import sys
@ -16,6 +21,8 @@ import json
import subprocess
from pathlib import Path
import chevron_blue
CRATE_ROOT = Path(__file__).parent
WORKSPACE_ROOT = CRATE_ROOT.parent.parent
VERSION_METADATA = CRATE_ROOT / "download-metadata.json"
@ -23,16 +30,6 @@ TEMPLATE = CRATE_ROOT / "src" / "downloads.inc.mustache"
TARGET = TEMPLATE.with_suffix("")
try:
import chevron_blue
except ImportError:
print(
"missing requirement `chevron-blue`",
file=sys.stderr,
)
exit(1)
def prepare_name(name: str) -> str:
match name:
case "cpython":
@ -73,8 +70,8 @@ def main():
debug = logging.getLogger().getEffectiveLevel() <= logging.DEBUG
data = {}
data["generated_with"] = Path(__file__).relative_to(WORKSPACE_ROOT)
data["generated_from"] = TEMPLATE.relative_to(WORKSPACE_ROOT)
data["generated_with"] = Path(__file__).relative_to(WORKSPACE_ROOT).as_posix()
data["generated_from"] = TEMPLATE.relative_to(WORKSPACE_ROOT).as_posix()
data["versions"] = [
{"key": key, "value": prepare_value(value)}
for key, value in json.loads(VERSION_METADATA.read_text()).items()