konsti f10f902570 Yield after channel send and move cpu tasks to thread (#1163) ... ## Summary Previously, we were blocking operations that could run in parallel. We would send request through our main requests channel, but not yield so that the receiver could only start processing requests much later than necessary. We solve this by switching to the async `tokio::sync::mpsc::channel`, where send is an async functions that yields. Due to the increased parallelism cache deserialization and the conversion from simple api request to version map became bottlenecks, so i moved them to `spawn_blocking`. Together these result in a 30-60% speedup for larger warm cache resolution. Small cases such as black already resolve in 5.7 ms on my machine so there's no speedup to be gained, refresh and no cache were to noisy to get signal from. Note for the future: Revisit the bounded channel if we want to produce requests from `process_request`, too, (this would be good for prefetching) to avoid deadlocks. ## Details We can look at the behavior change through the spans: ``` RUST_LOG=puffin=info TRACING_DURATIONS_FILE=target/traces/jupyter-warm-branch.ndjson cargo run --features tracing-durations-export --bin puffin-dev --profile profiling -- resolve jupyter 2> /dev/null ``` Below, you can see how on main, we have discrete phases: All (cached) simple api requests in parallel, then all (cached) metadata requests in parallel, repeat until done. The solver is mostly waiting until it has it's version map from the simple API query to be able to choose a version. The main thread is blocked by process requests. In the PR branch, the simple api requests succeeds much earlier, allowing the solver to advance and also to schedule more prefetching. Due to that `parse_cache` and `from_metadata` became bottlenecks, so i moved them off the main thread (green color, and their spans can now overlap because they can run on multiple threads in parallel). The main thread isn't blocked on `process_request` anymore, instead it has frequent idle times. The spans are all much shorter, which indicates that on main they could have finished much earlier, but a task didn't yield so they weren't scheduled to finish (though i haven't dug deep enough to understand the exact scheduling between the process request stream and the solver here). **main**  **PR**  ## Benchmarks ``` $ hyperfine --warmup 3 "target/profiling/main-dev resolve jupyter" "target/profiling/branch-dev resolve jupyter" Benchmark 1: target/profiling/main-dev resolve jupyter Time (mean ± σ): 29.1 ms ± 0.7 ms [User: 22.9 ms, System: 11.1 ms] Range (min … max): 27.7 ms … 32.2 ms 103 runs Benchmark 2: target/profiling/branch-dev resolve jupyter Time (mean ± σ): 18.8 ms ± 1.1 ms [User: 37.0 ms, System: 22.7 ms] Range (min … max): 16.5 ms … 21.9 ms 154 runs Summary target/profiling/branch-dev resolve jupyter ran 1.55 ± 0.10 times faster than target/profiling/main-dev resolve jupyter $ hyperfine --warmup 3 "target/profiling/main-dev resolve meine_stadt_transparent" "target/profiling/branch-dev resolve meine_stadt_transparent" Benchmark 1: target/profiling/main-dev resolve meine_stadt_transparent Time (mean ± σ): 37.8 ms ± 0.9 ms [User: 30.7 ms, System: 14.1 ms] Range (min … max): 36.6 ms … 41.5 ms 79 runs Benchmark 2: target/profiling/branch-dev resolve meine_stadt_transparent Time (mean ± σ): 24.7 ms ± 1.5 ms [User: 47.0 ms, System: 39.3 ms] Range (min … max): 21.5 ms … 28.7 ms 113 runs Summary target/profiling/branch-dev resolve meine_stadt_transparent ran 1.53 ± 0.10 times faster than target/profiling/main-dev resolve meine_stadt_transparent $ hyperfine --warmup 3 "target/profiling/main pip compile scripts/requirements/home-assistant.in" "target/profiling/branch pip compile scripts/requirements/home-assistant.in" Benchmark 1: target/profiling/main pip compile scripts/requirements/home-assistant.in Time (mean ± σ): 229.0 ms ± 2.8 ms [User: 197.3 ms, System: 63.7 ms] Range (min … max): 225.8 ms … 234.0 ms 13 runs Benchmark 2: target/profiling/branch pip compile scripts/requirements/home-assistant.in Time (mean ± σ): 91.4 ms ± 5.3 ms [User: 289.2 ms, System: 176.9 ms] Range (min … max): 81.0 ms … 104.7 ms 32 runs Summary target/profiling/branch pip compile scripts/requirements/home-assistant.in ran 2.50 ± 0.15 times faster than target/profiling/main pip compile scripts/requirements/home-assistant.in ```		2024-02-02 18:18:24 +01:00
..
src	Yield after channel send and move cpu tasks to thread (#1163)	2024-02-02 18:18:24 +01:00
tests	Add a builder for resolution options (#1192)	2024-01-30 19:50:16 +00:00
Cargo.toml	Yield after channel send and move cpu tasks to thread (#1163)	2024-02-02 18:18:24 +01:00