## Summary Follow up to https://github.com/astral-sh/uv/pull/15563 Closes https://github.com/astral-sh/uv/issues/13485 This is a first-pass at adding support for conditional support for Git LFS between git sources, initial feedback welcome. e.g. ``` [tool.uv.sources] test-lfs-repo = { git = "https://github.com/zanieb/test-lfs-repo.git", lfs = true } ``` For context previously a user had to set `UV_GIT_LFS` to have uv fetch lfs objects on git sources. This env var was all or nothing, meaning you must always have it set to get consistent behavior and it applied to all git sources. If you fetched lfs objects at a revision and then turned off lfs (or vice versa), the git db, corresponding checkout lfs artifacts would not be updated properly. Similarly, when git source distributions were built, there would be no distinction between sources with lfs and without lfs. Hence, it could corrupt the git, sdist, and archive caches. In order to support some sources being LFS enabled and other not, this PR adds a stateful layer roughly similar to how `subdirectory` works but for `lfs` since the git database, the checkouts and the corresponding caching layers needed to be LFS aware (requested vs installed). The caches also had to isolated and treated entirely separate when handling LFS sources. Summary * Adds `lfs = true` or `lfs = false` to git sources in pyproject.toml * Added `lfs=true` query param / fragments to most relevant url structs (not parsed as user input) * In the case of uv add / uv tool, `--lfs` is supported instead * `UV_GIT_LFS` environment variable support is still functional for non-project entrypoints (e.g. uv pip) * `direct-url.json` now has an custom `git_lfs` entry under VcsInfo (note, this is not in the spec currently -- see caveats). * git database and checkouts have an different cache key as the sources should be treated effectively different for the same rev. * sdists cache also differ in the cache key of a built distribution if it was built using LFS enabled revisions to distinguish between non-LFS same revisions. This ensures the strong assumption for archive-v0 that an unpacked revision "doesn't change sources" stays valid. Caveats * `pylock.toml` import support has not been added via git_lfs=true, going through the spec it wasn't clear to me it's something we'd support outside of the env var (for now). * direct-url struct was modified by adding a non-standard `git_lfs` field under VcsInfo which may be undersirable although the PEP 610 does say `Additional fields that would be necessary to support such VCS SHOULD be prefixed with the VCS command name` which could be interpret this change as ok. * There will be a slight lockfile and cache churn for users that use `UV_GIT_LFS` as all git lockfile entries will get a `lfs=true` fragment. The cache version does not need an update, but LFS sources will get their own namespace under git-v0 and sdist-v9/git hence a cache-miss will occur once but this can be sufficient to label this as breaking for workflows always setting `UV_GIT_LFS`. ## Test Plan Some initial tests were added. More tests likely to follow as we reach consensus on a final approach. For IT test, we may want to move to use a repo under astral namespace in order to test lfs functionality. Manual testing was done for common pathological cases like killing LFS fetch mid-way, uninstalling LFS after installing an sdist with it and reinstalling, fetching LFS artifacts in different commits, etc. PSA: Please ignore the docker build failures as its related to depot OIDC issues. --------- Co-authored-by: Zanie Blue <contact@zanie.dev> Co-authored-by: konstin <konstin@mailbox.org> |
||
|---|---|---|
| .cargo | ||
| .config | ||
| .github | ||
| assets | ||
| changelogs | ||
| crates | ||
| docs | ||
| ecosystem | ||
| python | ||
| scripts | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .ignore | ||
| .pre-commit-config.yaml | ||
| .prettierignore | ||
| .python-versions | ||
| _typos.toml | ||
| BENCHMARKS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CHANGELOG.md | ||
| clippy.toml | ||
| CONTRIBUTING.md | ||
| dist-workspace.toml | ||
| Dockerfile | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| mkdocs.insiders.yml | ||
| mkdocs.public.yml | ||
| mkdocs.template.yml | ||
| PIP_COMPATIBILITY.md | ||
| pyproject.toml | ||
| README.md | ||
| ruff.toml | ||
| rust-toolchain.toml | ||
| rustfmt.toml | ||
| SECURITY.md | ||
| STYLE.md | ||
| uv.schema.json | ||
uv
An extremely fast Python package and project manager, written in Rust.
Installing Trio's dependencies with a warm cache.
Highlights
- 🚀 A single tool to replace
pip,pip-tools,pipx,poetry,pyenv,twine,virtualenv, and more. - ⚡️ 10-100x faster than
pip. - 🗂️ Provides comprehensive project management, with a universal lockfile.
- ❇️ Runs scripts, with support for inline dependency metadata.
- 🐍 Installs and manages Python versions.
- 🛠️ Runs and installs tools published as Python packages.
- 🔩 Includes a pip-compatible interface for a performance boost with a familiar CLI.
- 🏢 Supports Cargo-style workspaces for scalable projects.
- 💾 Disk-space efficient, with a global cache for dependency deduplication.
- ⏬ Installable without Rust or Python via
curlorpip. - 🖥️ Supports macOS, Linux, and Windows.
uv is backed by Astral, the creators of Ruff.
Installation
Install uv with our standalone installers:
# On macOS and Linux.
curl -LsSf https://astral.sh/uv/install.sh | sh
# On Windows.
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
Or, from PyPI:
# With pip.
pip install uv
# Or pipx.
pipx install uv
If installed via the standalone installer, uv can update itself to the latest version:
uv self update
See the installation documentation for details and alternative installation methods.
Documentation
uv's documentation is available at docs.astral.sh/uv.
Additionally, the command line reference documentation can be viewed with uv help.
Features
Projects
uv manages project dependencies and environments, with support for lockfiles, workspaces, and more,
similar to rye or poetry:
$ uv init example
Initialized project `example` at `/home/user/example`
$ cd example
$ uv add ruff
Creating virtual environment at: .venv
Resolved 2 packages in 170ms
Built example @ file:///home/user/example
Prepared 2 packages in 627ms
Installed 2 packages in 1ms
+ example==0.1.0 (from file:///home/user/example)
+ ruff==0.5.0
$ uv run ruff check
All checks passed!
$ uv lock
Resolved 2 packages in 0.33ms
$ uv sync
Resolved 2 packages in 0.70ms
Audited 1 package in 0.02ms
See the project documentation to get started.
uv also supports building and publishing projects, even if they're not managed with uv. See the publish guide to learn more.
Scripts
uv manages dependencies and environments for single-file scripts.
Create a new script and add inline metadata declaring its dependencies:
$ echo 'import requests; print(requests.get("https://astral.sh"))' > example.py
$ uv add --script example.py requests
Updated `example.py`
Then, run the script in an isolated virtual environment:
$ uv run example.py
Reading inline script metadata from: example.py
Installed 5 packages in 12ms
<Response [200]>
See the scripts documentation to get started.
Tools
uv executes and installs command-line tools provided by Python packages, similar to pipx.
Run a tool in an ephemeral environment using uvx (an alias for uv tool run):
$ uvx pycowsay 'hello world!'
Resolved 1 package in 167ms
Installed 1 package in 9ms
+ pycowsay==0.0.0.2
"""
------------
< hello world! >
------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Install a tool with uv tool install:
$ uv tool install ruff
Resolved 1 package in 6ms
Installed 1 package in 2ms
+ ruff==0.5.0
Installed 1 executable: ruff
$ ruff --version
ruff 0.5.0
See the tools documentation to get started.
Python versions
uv installs Python and allows quickly switching between versions.
Install multiple Python versions:
$ uv python install 3.10 3.11 3.12
Searching for Python versions matching: Python 3.10
Searching for Python versions matching: Python 3.11
Searching for Python versions matching: Python 3.12
Installed 3 versions in 3.42s
+ cpython-3.10.14-macos-aarch64-none
+ cpython-3.11.9-macos-aarch64-none
+ cpython-3.12.4-macos-aarch64-none
Download Python versions as needed:
$ uv venv --python 3.12.0
Using Python 3.12.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
$ uv run --python pypy@3.8 -- python --version
Python 3.8.16 (a9dbdca6fc3286b0addd2240f11d97d8e8de187a, Dec 29 2022, 11:45:30)
[PyPy 7.3.11 with GCC Apple LLVM 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>>
Use a specific Python version in the current directory:
$ uv python pin 3.11
Pinned `.python-version` to `3.11`
See the Python installation documentation to get started.
The pip interface
uv provides a drop-in replacement for common pip, pip-tools, and virtualenv commands.
uv extends their interfaces with advanced features, such as dependency version overrides, platform-independent resolutions, reproducible resolutions, alternative resolution strategies, and more.
Migrate to uv without changing your existing workflows — and experience a 10-100x speedup — with the
uv pip interface.
Compile requirements into a platform-independent requirements file:
$ uv pip compile docs/requirements.in \
--universal \
--output-file docs/requirements.txt
Resolved 43 packages in 12ms
Create a virtual environment:
$ uv venv
Using Python 3.12.3
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Install the locked requirements:
$ uv pip sync docs/requirements.txt
Resolved 43 packages in 11ms
Installed 43 packages in 208ms
+ babel==2.15.0
+ black==24.4.2
+ certifi==2024.7.4
...
See the pip interface documentation to get started.
Platform support
See uv's platform support document.
Versioning policy
See uv's versioning policy document.
Contributing
We are passionate about supporting contributors of all levels of experience and would love to see you get involved in the project. See the contributing guide to get started.
FAQ
How do you pronounce uv?
It's pronounced as "you - vee" (/juː viː/)
How should I stylize uv?
Just "uv", please. See the style guide for details.
Acknowledgements
uv's dependency resolver uses PubGrub under the hood. We're grateful to the PubGrub maintainers, especially Jacob Finkelman, for their support.
uv's Git implementation is based on Cargo.
Some of uv's optimizations are inspired by the great work we've seen in pnpm, Orogene, and Bun. We've also learned a lot from Nathaniel J. Smith's Posy and adapted its trampoline for Windows support.
License
uv is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in uv by you, as defined in the Apache-2.0 license, shall be dually licensed as above, without any additional terms or conditions.