Commit graph

117 commits

Author SHA1 Message Date
Viktor Szépe
c191a83c5e
Fix typos configuration (#1742)
## Summary

Narrow down excludes for `typos` and fix 2 additional misspellings.

BTW pre-commit can be run in CI:
2176061490
2024-02-20 09:53:07 -05:00
Bas Zalmstra
daf2800ddf
feat: allow passing in a custom reqwest Client (#1745)
## Summary

I am looking to instantiate a `RegistryClient`. However, when using the
`RegistryClientBuilder` a new reqwest client is always constructed. I
would like to pass in a custom `reqwest::Client` to be able to share the
http resources with other parts of my application.

## Test Plan

The uv codebase does not use my addition to the builder and all tests
still succeed. And in my code I can pass a custom Client.
2024-02-20 09:50:18 -05:00
Di-Is
36edaeecf2
Control pip timeout duration via environment variable (#1694)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Add the environment variable `UV_REQUEST_TIMEOUT` to allow control over
pip timeouts.

Closes #1549 

## Test Plan

I built uv in the repository top Dockerfile, set the timeout to 3
seconds, and ran `uv pip install torch`.
I measured the execution time with the time command and confirmed that
the process finished at a value close to the timeout we set.

```bash
root@037c69228cdc:~# time UV_REQUEST_TIMEOUT=3 /uv pip install torch
Resolved 22 packages in 25ms
error: Failed to download distributions
  Caused by: Failed to fetch wheel: nvidia-cusolver-cu12==11.4.5.107
  Caused by: Failed to extract source distribution
  Caused by: request or response body error: operation timed out
  Caused by: operation timed out

real    0m3.064s
user    0m0.225s
sys     0m0.240s
```
2024-02-19 22:37:56 -06:00
Charlie Marsh
034f62b24f
Respect --index-url provided via requirements.txt (#1719)
## Summary

When we read `--index-url` from a `requirements.txt`, we attempt to
respect the `--index-url` provided by the CLI if it exists.
Unfortunately, `--index-url` from the CLI has a default value... so we
_never_ respect the `--index-url` in the requirements file.

This PR modifies the CLI to use `None`, and moves the default into logic
in the `IndexLocations `struct.

Closes https://github.com/astral-sh/uv/issues/1692.
2024-02-20 00:02:26 +00:00
Charlie Marsh
1d9daa6de1
Preserve trailing slash for --find-links URLs (#1720)
## Summary

We should allow a `--find-links` URL to be provided as _either_ (e.g.)
`https://wheelhouse.acsone.eu/manylinux1` or
`https://wheelhouse.acsone.eu/manylinux1/`. By using the response URL,
we can "always do the right thing" (it will always have a trailing
slash, or always return a `.html` suffix) rather than attempting to
sniff out the URL kind in advance.

Closes https://github.com/astral-sh/uv/issues/1683.

## Test Plan

- `cargo run pip install requests --force-reinstall --no-index
--find-links https://wheelhouse.acsone.eu/manylinux1 -n`
- `cargo run pip install requests --force-reinstall --no-index
--find-links https://wheelhouse.acsone.eu/manylinux1/ -n`
2024-02-19 21:26:12 +00:00
Olivier Le Floch
220bc46643
is_http_range_requests_unsupported should return true on Method Not Allowed (#1713)
## Summary

Azure Artifacts does not allow HEAD requests when attempting to download
packages. This expands error handling in
`is_http_range_requests_unsupported` to identify HTTP 405 (Method Not
Allowed) error codes, and return `true` (i.e. Range requests will not be
supported). This partially addresses #1458 – after this change, Azure
Artifacts downloads still fail, but due to 401 Not Authorized instead of
405 Method Not Allowed.

## Test Plan

I ran something akin to

```
RUST_LOG=trace cargo run -- pip install --index-url=https://REDACTED:REDACTED@pkgs.dev.azure.com/REDACTED/_packaging/REDACTED/pypi/simple/ --upgrade --verbose private-package
```

without this code, and got a 405 failure:

```
error: Failed to download: private-package==1.2.3
  Caused by: HTTP status client error (405 Method Not Allowed) for url (https://pkgs.dev.azure.com/REDACTED/_packaging/REDACTED/pypi/download/private-package/1.2.3/private_package-1.2.3-py3-none-any.whl#sha256=REDACTED)
  ```

with this code, I get a 401 failure:

```
error: Failed to download: private-package==1.2.3
Caused by: HTTP status client error (401 Unauthorized) for url
(https://pkgs.dev.azure.com/REDACTED/_packaging/REDACTED/pypi/download/private-package/1.2.3/private_package-1.2.3-py3-none-any.whl#sha256=REDACTED)
```

## Caveats

I'm not seeing a non HEAD request being reported as being fired, so I'm not sure I'm doing this correctly!
2024-02-19 15:40:25 -05:00
Andrew Gallant
cd1f619d21
re-introduce cache healing when we see an invalid cache entry (#1707)
This PR introduces more robust cache healing when `uv` fails to
deserialize an existing cache entry.

("Cache healing" in this context means that if `uv` fails to
deserialize a cache entry, then it will automatically invalidate that
entry and re-generate the data. Typically by sending an HTTP request.)

Previous to some optimizations I made around deserialization, we were
already doing this. After those optimizations, deserializing a cache
policy and the payload were split into two steps. While deserializing
a cache policy retained its cache healing behavior, deserializing the
payload did not. This became an issue when #1556 landed, which changed
one of our `rkyv` data types. This in turn made our internal types
incompatible with existing cache entries. One could work-around this
by clearing `uv`'s cache with `uv clean`, but we should just do it
automatically on a cache entry by entry basis.

This does technically introduce a new cost by pessimistically cloning
the HTTP request so that we can re-send it if necessary (see the commit
messages for the knot pushing me toward this approach). So I re-ran my
favorite ad-hoc benchmark:

```
$ hyperfine -w10 --runs 50 "uv-main pip compile --cache-dir ~/astral/tmp/cache-main ~/astral/tmp/reqs/home-assistant-reduced.in -o /dev/null" "uv-test pip compile --cache-dir ~/astral/tmp/cache-test ~/astral/tmp/reqs/home-assistant-reduced.in -o /dev/null" ; A bart
Benchmark 1: uv-main pip compile --cache-dir ~/astral/tmp/cache-main ~/astral/tmp/reqs/home-assistant-reduced.in -o /dev/null
  Time (mean ± σ):     114.4 ms ±   3.2 ms    [User: 149.4 ms, System: 221.5 ms]
  Range (min … max):   106.7 ms … 122.0 ms    50 runs

Benchmark 2: uv-test pip compile --cache-dir ~/astral/tmp/cache-test ~/astral/tmp/reqs/home-assistant-reduced.in -o /dev/null
  Time (mean ± σ):     114.0 ms ±   3.0 ms    [User: 146.0 ms, System: 223.3 ms]
  Range (min … max):   105.3 ms … 121.4 ms    50 runs

Summary
  uv-test pip compile --cache-dir ~/astral/tmp/cache-test ~/astral/tmp/reqs/home-assistant-reduced.in -o /dev/null ran
    1.00 ± 0.04 times faster than uv-main pip compile --cache-dir ~/astral/tmp/cache-main ~/astral/tmp/reqs/home-assistant-reduced.in -o /dev/null
```

Which is about what I expected.

We should endeavor to have a better testing strategy for these kinds of
bugs, but I think it might be a little tricky to do. I created
https://github.com/astral-sh/uv/issues/1699 to track that.

Fixes #1571
2024-02-19 12:33:35 -05:00
Aarni Koskela
bc14ed1613
Fix typos & add pre-commit configuration (#1487)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-02-17 18:16:50 +01:00
Charlie Marsh
facc60f3a8
Add graceful fallback for Artifactory indexes (#1574)
## Summary

There are more details in https://github.com/astral-sh/uv/issues/1370,
but it looks like Artifactory servers have incorrect behavior when it
comes to HTTP range requests, in that they return `Accept-Ranges:
bytes`, but then incorrectly return 200 requests when you actually ask
for a given range.

This PR ensures that we fallback gracefully in this case. It's built on
https://github.com/prefix-dev/async_http_range_reader/pull/5. Assuming
that gets merged upstream, we can then remove the Git dependency.

Closes https://github.com/astral-sh/uv/issues/1370.

## Test Plan

`cargo run pip install requests -i
https://killjoyuvbug.jfrog.io/artifactory/api/pypi/pypi/simple
--verbose`
2024-02-17 14:37:06 +00:00
Charlie Marsh
c1eb6130e1
Support MD5 hashes (#1556)
## Summary

We can add other hashes if necessary, but I don't know that they're
really used in practice.

Closes https://github.com/astral-sh/uv/issues/1547.
2024-02-17 00:25:16 +00:00
David Szotten
8050370717
Fix trailing commas on Requires-Python in HTML indexes (#1507)
illustration and suggested fix for #1464
2024-02-16 22:05:28 +00:00
Charlie Marsh
4f216f3a74
Apply percent-decoding to filepaths in HTML find-links (#1544)
## Summary

Closes https://github.com/astral-sh/uv/issues/1542.
2024-02-16 16:47:04 -05:00
Zanie Blue
e6c4c77ba1
Use string display instead of debug for url parse trace (#1498)
e.g. 

`uv_client::html::parse
url=https://download.pytorch.org/whl/torch_stable.html`

instead of

`uv_client::html::parse url=Url { scheme: "https", cannot_be_a_base:
false, username: "", password: None, host:
Some(Domain("download.pytorch.org")), port: None, path:
"/whl/torch_stable.html", query: None, fragment: None }`
2024-02-16 15:13:16 +00:00
Charlie Marsh
c474370064
Allow empty fragments in HTML parser (#1443)
## Summary

It looks like `devpi` might add an empty fragment (`#`) at the end of
the URL. We expect it to contain the hash; this just makes
empty-fragment map to "no hash".

Closes https://github.com/astral-sh/uv/issues/1441.
2024-02-16 06:42:21 +00:00
Charlie Marsh
0d005a2a71
Decode HTML escapes when extracting SHA (#1440)
## Summary

If a distribution contains a `+`, it'll be HTML-escaped; so when we try
to identify the `#`, we'll split in the wrong location.

Closes https://github.com/astral-sh/uv/issues/1338.
2024-02-16 06:15:51 +00:00
Zanie Blue
0bfce353fb
Fix broken URLs parsed from relative paths in registries (#1413)
Closes https://github.com/astral-sh/uv/issues/1388

Fixes incorrect handling of relative paths returned by indexes without
an explicit `<base>`.

`Url.join` will drop the last segment in an url e.g. `http://foo/bar` ->
`http://foo/baz` if there is not a trailing slash but what we want is
`http://foo/bar/baz`. We don't add the trailing `/` in
`base_url_join_relative` because flat indexes are `http://foo/bar.html`
and we _want_ `bar.html` to be replaced.
2024-02-15 22:37:09 -06:00
Zanie Blue
2586f655bb
Rename to uv (#1302)
First, replace all usages in files in-place. I used my editor for this.
If someone wants to add a one-liner that'd be fun.

Then, update directory and file names:

```
# Run twice for nested directories
find . -type d -print0 | xargs -0 rename s/puffin/uv/g
find . -type d -print0 | xargs -0 rename s/puffin/uv/g

# Update files
find . -type f -print0 | xargs -0 rename s/puffin/uv/g
```

Then add all the files again

```
# Add all the files again
git add crates
git add python/uv

# This one needs a force-add
git add -f crates/uv-trampoline
```
2024-02-15 11:19:46 -06:00