gh-124694: Add concurrent.futures.InterpreterPoolExecutor (gh-124548)

This is an implementation of InterpreterPoolExecutor that builds on ThreadPoolExecutor.

(Note that this is not tied to PEP 734, which is strictly about adding a new stdlib module.)

Possible future improvements:

* support passing a script for the initializer or to submit()
* support passing (most) arbitrary functions without pickling
* support passing closures
* optionally exec functions against __main__ instead of the their original module
This commit is contained in:
Eric Snow 2024-10-16 16:50:46 -06:00 committed by GitHub
parent a38fef4439
commit a5a7f5e16d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 828 additions and 40 deletions

View file

@ -1305,6 +1305,12 @@ Executing code in thread or process pools
pool, cpu_bound)
print('custom process pool', result)
# 4. Run in a custom interpreter pool:
with concurrent.futures.InterpreterPoolExecutor() as pool:
result = await loop.run_in_executor(
pool, cpu_bound)
print('custom interpreter pool', result)
if __name__ == '__main__':
asyncio.run(main())
@ -1329,7 +1335,8 @@ Executing code in thread or process pools
Set *executor* as the default executor used by :meth:`run_in_executor`.
*executor* must be an instance of
:class:`~concurrent.futures.ThreadPoolExecutor`.
:class:`~concurrent.futures.ThreadPoolExecutor`, which includes
:class:`~concurrent.futures.InterpreterPoolExecutor`.
.. versionchanged:: 3.11
*executor* must be an instance of