If the `shutdown()` call happens before the worker thread starts executing
the task, then nothing will be printed to stdout.
(cherry picked from commit 7d1abe9502)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-109917: Fix test instability in test_concurrent_futures (GH-110306)
The test had an instability issue due to the ordering of the dummy
queue operation and the real wakeup pipe operations. Both primitives
are thread safe but not done atomically as a single update and may
interleave arbitrarily. With the old order of operations this can lead
to an incorrect state where the dummy queue is full but the wakeup
pipe is empty. By swapping the order in clear() I think this can no
longer happen in any possible operation interleaving (famous last
words).
(cherry picked from commit a376a72bd9)
Co-authored-by: elfstrom <elfstrom@users.noreply.github.com>
gh-109047: concurrent.futures catches PythonFinalizationError (#109810)
concurrent.futures: The *executor manager thread* now catches
exceptions when adding an item to the *call queue*. During Python
finalization, creating a new thread can now raise RuntimeError. Catch
the exception and call terminate_broken() in this case.
Add test_python_finalization_error() to test_concurrent_futures.
concurrent.futures._ExecutorManagerThread changes:
* terminate_broken() no longer calls shutdown_workers() since the
call queue is no longer working anymore (read and write ends of
the queue pipe are closed).
* terminate_broken() now terminates child processes, not only
wait until they complete.
* _ExecutorManagerThread.terminate_broken() now holds shutdown_lock
to prevent race conditons with ProcessPoolExecutor.submit().
multiprocessing.Queue changes:
* Add _terminate_broken() method.
* _start_thread() sets _thread to None on exception to prevent
leaking "dangling threads" even if the thread was not started
yet.
(cherry picked from commit 6351842121)
gh-109594: Fix concurrent.futures test_timeout() (GH-110018)
Fix test_timeout() of test_concurrent_futures.test_wait. Remove the
future which may or may not complete depending if it takes longer
than the timeout ot not. Keep the second future which does not
complete before wait(). Make also the test faster: 0.5 second instead
of 6 seconds, so remove @support.requires_resource('walltime')
decorator.
(cherry picked from commit 9be283e5e1)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-109565: Fix concurrent.futures test_future_times_out() (GH-109949)
as_completed() uses a timeout of 100 ms instead of 10 ms. Windows
monotonic clock resolution is around 15.6 ms.
(cherry picked from commit b1aebf1e65)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-109832: concurrent.futures test_deadlock restores sys.stderr (GH-109887)
test_error_at_task_unpickle() and
test_error_during_result_unpickle_in_result_handler() now restore
sys.stderr which is overriden by _raise_error_ignore_stderr().
(cherry picked from commit 2897142d2e)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-109702: Increase concurrent_futures deadlock timeout (GH-109703)
Replace SHORT_TIMEOUT with LONG_TIMEOUT in test_deadlock of
test_concurrent_futures.
(cherry picked from commit 1eb1b45183)
Co-authored-by: Victor Stinner <vstinner@python.org>
* gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.
Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.
Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.
Changes:
* Fix and enhance statistics in regrtest summary. Add "(filtered)"
when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
(cherry picked from commit 31c2945f14)
* gh-108834: regrtest --fail-rerun exits with code 5 (#108896)
When the --fail-rerun option is used and a test fails and then pass,
regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test").
(cherry picked from commit 1170d5a292)
* gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480)
(cherry picked from commit 1e0d62793a)
* Manually sync Lib/test/libregrtest/ from main
---------
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-108416: Mark slow test methods with @requires_resource('cpu') (GH-108421)
Only mark tests which spend significant system or user time,
by itself or in subprocesses.
(cherry picked from commit f3ba0a74cd)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-108388: Convert test_concurrent_futures to package (#108401)
Convert test_concurrent_futures to a package of sub-tests.
(cherry picked from commit aa6f787faa)
