We observed recent deadlocks (e.g.
2522386625)
causing CI to run until the default github actions timeout. To prevent
this, we add a 60s timeout in nextest. 60s should be enough both on CI
and on developers' machines.
## Summary
This is my guess as to the source of the resolver flake, based on
information and extensive debugging from @zanieb. In short, if we rely
on `self.index.packages` as a source of truth during error reporting, we
open ourselves up to a source of non-determinism, because we fetch
package metadata asynchronously in the background while we solve -- so
packages _could_ be included in or excluded from the index depending on
the order in which those requests are returned.
So, instead, we now track the set of packages that _were_ visited by the
solver. Visiting a package _requires_ that we wait for its metadata to
be available. By limiting analysis to those packages that were visited
during solving, we are faithfully representing the state of the solver
at the time of failure.
Closes#863
Follow-up to #996 with retries of the full `pip_install_scenario` module
since there are a few tests that flake and tracking down the
non-determinism is proving to be quite complicated.
Adds a profile that can be used to disable retries, e.g. `--profile
no-retry` when you expect the scenario to change.
As before, hopefully this is a short-term measure.
We should definitely sort this out per #863 but until then I want to
stop getting false reports of failed CI please and this seems better
than turning off the test entirely.
Alternatively #993
