mirror of
https://github.com/python/cpython.git
synced 2025-08-04 17:08:35 +00:00
gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222)
When shutdown is called with wait=False, the executor thread keeps running even after the ProcessPoolExecutor's state is reset. The executor then tries to replenish the worker processes pool resulting in an error and a potential hang when it comes across a worker that has died. Fixed the issue by having _adjust_process_count() return without doing anything if the ProcessPoolExecutor's state has been reset. Added unit tests to validate two scenarios: max_workers < num_tasks (exception) max_workers > num_tasks (exception + hang)
This commit is contained in:
parent
ee7345d507
commit
598aa7cc98
3 changed files with 70 additions and 0 deletions
|
@ -755,6 +755,11 @@ class ProcessPoolExecutor(_base.Executor):
|
|||
self._executor_manager_thread_wakeup
|
||||
|
||||
def _adjust_process_count(self):
|
||||
# gh-132969: avoid error when state is reset and executor is still running,
|
||||
# which will happen when shutdown(wait=False) is called.
|
||||
if self._processes is None:
|
||||
return
|
||||
|
||||
# if there's an idle process, we don't need to spawn a new one.
|
||||
if self._idle_worker_semaphore.acquire(blocking=False):
|
||||
return
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue