uv/crates/uv-resolver/src
Ibraheem Ahmed 783df8f657
Consolidate concurrency limits (#3493)
## Summary

This PR consolidates the concurrency limits used throughout `uv` and
exposes two limits, `UV_CONCURRENT_DOWNLOADS` and
`UV_CONCURRENT_BUILDS`, as environment variables.

Currently, `uv` has a number of concurrent streams that it buffers using
relatively arbitrary limits for backpressure. However, many of these
limits are conflated. We run a relatively small number of tasks overall
and should start most things as soon as possible. What we really want to
limit are three separate operations:
- File I/O. This is managed by tokio's blocking pool and we should not
really have to worry about it.
- Network I/O.
- Python build processes.

Because the current limits span a broad range of tasks, it's possible
that a limit meant for network I/O is occupied by tasks performing
builds, reading from the file system, or even waiting on a `OnceMap`. We
also don't limit build processes that end up being required to perform a
download. While this may not pose a performance problem because our
limits are relatively high, it does mean that the limits do not do what
we want, making it tricky to expose them to users
(https://github.com/astral-sh/uv/issues/1205,
https://github.com/astral-sh/uv/issues/3311).

After this change, the limits on network I/O and build processes are
centralized and managed by semaphores. All other tasks are unbuffered
(note that these tasks are still bounded, so backpressure should not be
a problem).
2024-05-10 12:43:08 -04:00
..
pubgrub uv-resolver: push resolver state to its own type (#3492) 2024-05-09 14:16:43 -04:00
resolver Consolidate concurrency limits (#3493) 2024-05-10 12:43:08 -04:00
snapshots uv-resolver: make hashes optional (#3505) 2024-05-10 10:32:30 -04:00
bare.rs Add support for parsing unnamed URL requirements (#2567) 2024-03-21 03:28:58 +00:00
candidate_selector.rs uv-resolver: make MarkerEnvironment optional 2024-05-09 09:24:37 -04:00
dependency_mode.rs Rename to uv (#1302) 2024-02-15 11:19:46 -06:00
dependency_provider.rs uv-resolver: push resolver state to its own type (#3492) 2024-05-09 14:16:43 -04:00
editables.rs Add basic tool.uv.sources support (#3263) 2024-05-03 21:10:50 +00:00
error.rs Use generic pubgrub incompatibility reason (#3335) 2024-05-08 08:40:15 +00:00
exclude_newer.rs Improve JSON Schema and add export script (#3461) 2024-05-08 16:15:16 +00:00
exclusions.rs Split configuration options out of uv-types (#2924) 2024-04-09 11:35:53 -05:00
flat_index.rs Support unnamed requirements in --require-hashes (#2993) 2024-04-11 11:26:50 -04:00
lib.rs uv-resolver: add initial version of universal lock file format (#3314) 2024-04-29 14:03:17 -04:00
lock.rs uv-resolver: make hashes optional (#3505) 2024-05-10 10:32:30 -04:00
manifest.rs uv-resolver: make MarkerEnvironment optional 2024-05-09 09:24:37 -04:00
options.rs Implement --index-strategy unsafe-best-match (#3138) 2024-04-27 01:24:54 +00:00
pins.rs uv-resolver: push resolver state to its own type (#3492) 2024-05-09 14:16:43 -04:00
preferences.rs uv-resolver: make MarkerEnvironment optional 2024-05-09 09:24:37 -04:00
prerelease_mode.rs uv-resolver: make MarkerEnvironment optional 2024-05-09 09:24:37 -04:00
python_requirement.rs pep508: un-export fields for MarkerEnvironment 2024-05-09 10:06:02 -04:00
redirect.rs Remove redirects from the resolver (#2792) 2024-04-03 02:43:57 +00:00
resolution.rs Add a dedicated struct for source annotations (#3478) 2024-05-09 04:40:35 +00:00
resolution_mode.rs uv-resolver: make MarkerEnvironment optional 2024-05-09 09:24:37 -04:00
version_map.rs Move ExcludeNewer into its own type (#3041) 2024-04-15 20:24:08 +00:00
yanks.rs uv-resolver: make MarkerEnvironment optional 2024-05-09 09:24:37 -04:00