mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-07-12 13:55:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 14

gh-89240: Enable multiprocessing on Windows to use large process pools (GH-107873)

Browse source 
We add _winapi.BatchedWaitForMultipleObjects to wait for larger numbers of handles.
This is an internal module, hence undocumented, and should be used with caution.
Check the docstring for info before using BatchedWaitForMultipleObjects.
This commit is contained in:
Steve Dower 2024-02-13 00:28:35 +00:00 • committed by GitHub
parent 2f0778675a
commit ea25f32d5f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 1195 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

14
Lib/multiprocessing/connection.py
View file

@ -1011,8 +1011,20 @@ if sys.platform == 'win32':
# returning the first signalled might create starvation issues.)
L = list(handles)
ready = []
# Windows limits WaitForMultipleObjects at 64 handles, and we use a
# few for synchronisation, so we switch to batched waits at 60.
if len(L) > 60:
try:
res = _winapi.BatchedWaitForMultipleObjects(L, False, timeout)
except TimeoutError:
return []
ready.extend(L[i] for i in res)
if res:
L = [h for i, h in enumerate(L) if i > res[0] & i not in res]
timeout = 0
while L:
res = _winapi.WaitForMultipleObjects(L, False, timeout)
short_L = L[:60] if len(L) > 60 else L
res = _winapi.WaitForMultipleObjects(short_L, False, timeout)
if res == WAIT_TIMEOUT:
break
elif WAIT_OBJECT_0 <= res < WAIT_OBJECT_0 + len(L):