4b83d8e949
In the resolver, our current model for solving URL dependencies requires that we visit the URL dependency _before_ the registry-based dependency. This PR encodes a strict requirement that all URL dependencies be declared upfront, either as requirements or constraints. I wrote more about how it works and why it's necessary in documentation [here](https://github.com/astral-sh/puffin/pull/319/files#diff-2b1c4f36af0c62a2b7bebeae9473ae083588f2a6b18a3ec52393a24266adecbbR20). I think we could relax this constraint over time, but it requires a more sophisticated model -- and for now, I just want something that's (1) correct, (2) easy for us to reason about, and (3) easy for users to reason about. As additional motivation... allowing arbitrary URL dependencies anywhere in the tree creates some really confusing situations in which I'm not even sure what the right answers are. For example, assume you declare a direct dependency on `Werkzeug==2.0.0`. You then depend on a version of Flask that depends on a version of `Werkzeug` from some arbitrary URL. You build the source distribution at that arbitrary URL, and it turns out it _does_ build to a declared version of 2.0.0. What should happen? (And if it resolves to a version that _isn't_ 2.0.0, what should happen _then_?) I suspect different tools handle this differently, but it must lead to a lot of "silent" failures. In my testing of Poetry, it seems like Poetry just ignores the URL dependency, which seems wrong, but is also a behavior we could implement in the future. Closes https://github.com/astral-sh/puffin/issues/303. Closes https://github.com/astral-sh/puffin/issues/284. |
||
|---|---|---|
| .cargo | ||
| .github/workflows | ||
| crates | ||
| scripts | ||
| vendor/pubgrub | ||
| workers/pypi-metadata | ||
| .dockerignore | ||
| .gitignore | ||
| builder.dockerfile | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CONTRIBUTING.md | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| README.md | ||
| ruff.toml | ||
| rust-toolchain.toml | ||
puffin
An experimental Python packaging tool.
Motivation
Puffin is an extremely fast (experimental) Python package resolver and installer, intended to
replace pip and pip-tools (pip-compile and pip-sync).
Puffin itself is not a complete "package manager", but rather a tool for locking dependencies
(similar to pip-compile) and installing them (similar to pip-sync). Puffin can be used to
generate a set of locked dependencies from a requirements.txt file, and then install those
locked dependencies into a virtual environment.
Puffin represents an intermediary goal in our pursuit of building a "Cargo for Python": a Python
package manager that is extremely fast, reliable, and easy to use -- capable of replacing not only
pip, but also pipx, pip-tools, virtualenv, tox, setuptools, and even pyenv, by way of
managing the Python installation itself.
Puffin's limited scope allows us to solve many of the low-level problems that are required to
build such a package manager (like package installation) while shipping an immediately useful tool
with a minimal barrier to adoption. Try it today in lieu of pip and pip-tools.
Features
- Extremely fast dependency resolution and installation: install dependencies in sub-second time.
- Disk-space efficient: Puffin uses a global cache to deduplicate dependencies, and uses Copy-on-Write on supported filesystems to reduce disk usage.
Limitations
Puffin does not yet support:
- Source distributions
- VCS dependencies
- URL dependencies
- Windows
- ...
Like pip-compile, Puffin generates a platform-specific requirements.txt file (unlike, e.g.,
poetry, which generates a platform-agnostic poetry.lock file). As such, Puffin's
requirements.txt files are not portable across platforms and Python versions.
Usage
To resolve a requirements.in file:
cargo run -p puffin-cli -- pip-compile requirements.in
To install from a resolved requirements.txt file:
cargo run -p puffin-cli -- pip-sync requirements.txt
For more, see cargo run -p puffin-cli -- --help:
Usage: puffin-cli <COMMAND>
Commands:
compile Compile a `requirements.in` file to a `requirements.txt` file
sync Sync dependencies from a `requirements.txt` file
clean Clear the cache
freeze Enumerate the installed packages in the current environment
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
License
Puffin is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Puffin by you, as defined in the Apache-2.0 license, shall be dually licensed as above, without any additional terms or conditions.
