mirror of
https://github.com/python/cpython.git
synced 2025-08-31 22:18:28 +00:00
Update logging cookbook to show multiple worker processes using the concurrent.futures module. (GH-14905) (GH-14906)
(cherry picked from commit d309352c6f
)
This commit is contained in:
parent
22fd679dc3
commit
5b398520a8
1 changed files with 35 additions and 0 deletions
|
@ -948,6 +948,41 @@ This variant shows how you can e.g. apply configuration for particular loggers
|
||||||
machinery in the main process (even though the logging events are generated in
|
machinery in the main process (even though the logging events are generated in
|
||||||
the worker processes) to direct the messages to the appropriate destinations.
|
the worker processes) to direct the messages to the appropriate destinations.
|
||||||
|
|
||||||
|
Using concurrent.futures.ProcessPoolExecutor
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
If you want to use :class:`concurrent.futures.ProcessPoolExecutor` to start
|
||||||
|
your worker processes, you need to create the queue slightly differently.
|
||||||
|
Instead of
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
queue = multiprocessing.Queue(-1)
|
||||||
|
|
||||||
|
you should use
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
queue = multiprocessing.Manager().Queue(-1) # also works with the examples above
|
||||||
|
|
||||||
|
and you can then replace the worker creation from this::
|
||||||
|
|
||||||
|
workers = []
|
||||||
|
for i in range(10):
|
||||||
|
worker = multiprocessing.Process(target=worker_process,
|
||||||
|
args=(queue, worker_configurer))
|
||||||
|
workers.append(worker)
|
||||||
|
worker.start()
|
||||||
|
for w in workers:
|
||||||
|
w.join()
|
||||||
|
|
||||||
|
to this (remembering to first import :mod:`concurrent.futures`)::
|
||||||
|
|
||||||
|
with concurrent.futures.ProcessPoolExecutor(max_workers=10) as executor:
|
||||||
|
for i in range(10):
|
||||||
|
executor.submit(worker_process, queue, worker_configurer)
|
||||||
|
|
||||||
|
|
||||||
Using file rotation
|
Using file rotation
|
||||||
-------------------
|
-------------------
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue