mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-11-17 17:46:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

gh-108834: regrtest reruns failed tests in subprocesses (#108839)

Browse source 
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.

Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.

Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.

Changes:

* Fix and enhance statistics in regrtest summary. Add "(filtered)"
  when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
  using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
  filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
  the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
This commit is contained in:
Victor Stinner 2023-09-03 23:37:15 +02:00 • committed by GitHub
parent c2ec174d24
commit 31c2945f14
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
12 changed files with 819 additions and 478 deletions
Download patch file Download diff file Expand all files Collapse all files

6
Misc/NEWS.d/next/Tests/2023-09-03-02-01-55.gh-issue-108834.iAwXzj.rst Normal file
View file

@ -0,0 +1,6 @@
When regrtest reruns failed tests in verbose mode (``./python -m test
--rerun``), tests are now rerun in fresh worker processes rather than being
executed in the main process. If a test does crash or is killed by a timeout,
the main process can detect and handle the killed worker process. Tests are
rerun in parallel if the ``-jN`` option is used to run tests in parallel.
Patch by Victor Stinner.

2
Misc/NEWS.d/next/Tests/2023-09-03-06-17-12.gh-issue-108834.fjV-CJ.rst Normal file
View file

@ -0,0 +1,2 @@
Rename regrtest ``--verbose2`` option (``-w``) to ``--rerun``. Keep
``--verbose2`` as a deprecated alias. Patch by Victor Stinner.

3
Misc/NEWS.d/next/Tests/2023-09-03-20-15-49.gh-issue-108834.Osvmhf.rst Normal file
View file

@ -0,0 +1,3 @@
Add ``--fail-rerun option`` option to regrtest: if a test failed when then
passed when rerun in verbose mode, exit the process with exit code 2
(error), instead of exit code 0 (success). Patch by Victor Stinner.