Commit graph

16 commits

Author SHA1 Message Date
konsti
91d0fdbbdf
Add script to compare with pip(-tools) (#335)
Add a script to compare with pip-tools and pydantic input we can compare
with it. Below is the output for `pydantic.in`, created from pydantic's
pyproject.toml, which i added for that purpose:

```console
$ scripts/compare_with_pip.sh scripts/benchmarks/requirements/pydantic.in
      Finished dev [unoptimized + debuginfo] target(s) in 0.08s
       Running `target/debug/puffin pip-compile scripts/benchmarks/requirements/pydantic.in`
  Resolved 85 packages in 1.61s

  real    0m1,733s
  user    0m1,714s
  sys     0m0,048s

  real    0m10,843s
  user    0m4,811s
  sys     0m0,399s
  --- /tmp/tmp.Y3FzvQ2xxo/pip-compile.txt 2023-11-06 15:47:29.221834123 +0100
  +++ /tmp/tmp.Y3FzvQ2xxo/puffin.txt      2023-11-06 15:47:18.377408860 +0100
  @@ -31,7 +31,7 @@
   mdurl==0.1.2
   memray==1.10.0
   mergedeep==1.3.4
  -mike @ git+https://github.com/jimporter/mike.git
  +mike @ git+https://github.com/jimporter/mike.git@076a4af3270a448f6aeb880c9c6c2fc0d80f603f
   mkdocs==1.5.3
   mkdocs-autorefs==0.5.0
   mkdocs-embed-external-markdown==3.0.1
  @@ -52,7 +52,7 @@
   py-cpuinfo==9.0.0
   pydantic==2.4.2
   pydantic-core==2.10.1
  -pydantic-extra-types @ git+https://github.com/pydantic/pydantic-extra-types.git@main
  +pydantic-extra-types @ git+https://github.com/pydantic/pydantic-extra-types.git@a973b7942112df731e2618336e55e3343a2e1c32
   pydantic-settings==2.0.3
   pyflakes==3.1.0
   pygments==2.16.1
  @@ -61,7 +61,7 @@
   pytest==7.4.3
   pytest-benchmark==4.0.0
   pytest-examples==0.0.10
  -pytest-memray==1.5.0 ; platform_system != "Windows"
  +pytest-memray==1.5.0
   pytest-mock==3.12.0
   pytest-pretty==1.2.0
   python-dateutil==2.8.2
```
2023-11-07 16:32:12 +01:00
Charlie Marsh
b0286a8939
Add user feedback when building source distributions in the resolver (#347)
It looks like Cargo, notice the bold green lines at the top (which
appear during the resolution, to indicate Git fetches and source
distribution builds):

<img width="868" alt="Screen Shot 2023-11-06 at 11 28 47 PM"
src="9647a480-7be7-41e9-b1d3-69faefd054ae">

<img width="868" alt="Screen Shot 2023-11-06 at 11 28 51 PM"
src="6bc491aa-5b51-4b37-9ee1-257f1bc1c049">

Closes https://github.com/astral-sh/puffin/issues/287 although we can do
a lot more here.
2023-11-07 14:17:31 +00:00
konsti
c9e0f4986f
Add requirements from PDM issue (#326) 2023-11-06 11:07:31 +00:00
konstin
1529def563 Implement mixed PEP 517 and setup.py build
There are packages such as DTLSSocket 0.1.16 that say
```toml
[build-system]
requires = ["Cython<3", "setuptools", "wheel"]
```
In this case we need to install requires PEP 517 style but then call setup.py in the
legacy way

Part of making home-assistant work
2023-10-30 19:11:52 +01:00
konsti
d47dc64974
Ignore self requirements (#233)
gps3 0.33.3 depends on itself, which we can ignore. I've also added the
home assistant requirements since it occurred when testing with this.
2023-10-30 17:13:52 +01:00
Charlie Marsh
1c5cdcd70a
Prioritize packages in visited order (#222) 2023-10-30 00:48:36 +00:00
Charlie Marsh
2ba85bf80e
Add PubGrub's priority queue (#221)
Pulls in https://github.com/pubgrub-rs/pubgrub/pull/104.
2023-10-29 21:16:02 +00:00
konsti
5ad58474ca
Add script to check the top 8k pypi packages (#198)
To check to top 1k (current state):

```bash
scripts/resolve/get_pypi_top_8k.sh
cargo run --bin puffin-dev -- resolve-many scripts/resolve/pypi_top_8k_flat.txt --limit 1000
```

Results:
```
Errors: pywin32, geoip2, maxminddb, pypika, dirac
Success: 995, Error: 5
```
pywin32 has no solution for the build environment, 3 have no
`[build-system]` entry in pyproject.toml, `dirac` is missing cmake
2023-10-26 12:03:59 +00:00
konsti
862c1654a0
Select most recent wheel, most recent sdist (#190)
Select a compatible wheel for a version, even we already found a source
distribution previously.

If no wheel is found, select the most recent source distribution, not
the oldest compatible one.

This fixes the resolution of `mst.in`, which i added
2023-10-26 08:15:26 +00:00
Charlie Marsh
21bb9c29cc
Add an additional requirements fixup (#174)
Also checking in a variety of different requirements inputs.
2023-10-23 19:50:39 -04:00
Charlie Marsh
bd01fb490e
Remove packages when syncing (#135)
`pip-sync` will now uninstall any packages that aren't necessary.

Closes https://github.com/astral-sh/puffin/issues/128.
2023-10-19 00:14:20 -04:00
Charlie Marsh
e15b99b911
Rename commands to pip-sync and pip-compile (#123)
To free up the rest of the interface.
2023-10-18 21:15:20 +00:00
konsti
8cc4fe0d44
Install source distribution requirements with puffin itself instead of pip (#122)
This is also a lot faster. Unfortunately it copies a lot of code from
the sync cli since the `Printer` is private.

The first commit are some refactorings i made when i thought about how i
could reuse the existing code.
2023-10-18 19:11:17 +00:00
Charlie Marsh
471a1d657d
Migrate resolver proof-of-concept to PubGrub (#97)
## Summary

This PR enables the proof-of-concept resolver to backtrack by way of
using the `pubgrub-rs` crate.

Rather than using PubGrub as a _framework_ (implementing the
`DependencyProvider` trait, letting PubGrub call us), I've instead
copied over PubGrub's primary solver hook (which is only ~100 lines or
so) and modified it for our purposes (e.g., made it async).

There's a lot to improve here, but it's a start that will let us
understand PubGrub's appropriateness for this problem space. A few
observations:

- In simple cases, the resolver is slower than our current (naive)
resolver. I think it's just that the pipelining isn't as efficient as in
the naive case, where we can just stream package and version fetches
concurrently without any bottlenecks.
- A lot of the code here relates to bridging PubGrub with our own
abstractions -- so we need a `PubGrubPackage`, a `PubGrubVersion`, etc.
2023-10-15 22:05:44 -04:00
Charlie Marsh
f03c605bde
Add a script to benchmark uninstalls (#78) 2023-10-09 16:59:15 -04:00
Charlie Marsh
75cb7a0178
Add benchmark scripts (#69)
Moving these out of the README and into proper scripts.
2023-10-08 23:37:38 +00:00