mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-09-26 18:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 10

Various small fixups to the multiprocessing docs, mostly fixing and

Browse source 
enabling doctests that Sphinx can run, and fixing and disabling tests that
Sphinx can't run.  I hand checked every test not now marked as a doctest,
and all except the two that have open bug reports against them now work,
at least on Linux/trunk. (I did not look at the last example at all since
there was already an open bug).  I did not read the whole document with
an editor's eye, but I did fix a few things I noticed while working on
the tests.
This commit is contained in:
R. David Murray 2009-04-28 16:08:18 +00:00
parent 94cc00ce4d
commit 636b23a96f
1 changed files with 64 additions and 44 deletions
Download patch file Download diff file Expand all files Collapse all files

106
Doc/library/multiprocessing.rst
View file

@ -42,12 +42,18 @@ Windows.
>>> p.map(f, [1,2,3]) >>> p.map(f, [1,2,3])
Process PoolWorker-1: Process PoolWorker-1:
Process PoolWorker-2: Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last): Traceback (most recent call last):
Traceback (most recent call last): Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f' AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f' AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f' AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
The :class:`Process` class The :class:`Process` class
~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -418,7 +424,9 @@ The :mod:`multiprocessing` package mostly replicates the API of the
:attr:`exit_code` methods should only be called by the process that created :attr:`exit_code` methods should only be called by the process that created
the process object. the process object.
Example usage of some of the methods of :class:`Process`:: Example usage of some of the methods of :class:`Process`:
.. doctest::
>>> import multiprocessing, time, signal >>> import multiprocessing, time, signal
>>> p = multiprocessing.Process(target=time.sleep, args=(1000,)) >>> p = multiprocessing.Process(target=time.sleep, args=(1000,))
@ -428,6 +436,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the
>>> print p, p.is_alive() >>> print p, p.is_alive()
<Process(Process-1, started)> True <Process(Process-1, started)> True
>>> p.terminate() >>> p.terminate()
>>> time.sleep(0.1)
>>> print p, p.is_alive() >>> print p, p.is_alive()
<Process(Process-1, stopped[SIGTERM])> False <Process(Process-1, stopped[SIGTERM])> False
>>> p.exitcode == -signal.SIGTERM >>> p.exitcode == -signal.SIGTERM
@ -669,7 +678,7 @@ Miscellaneous
freeze_support() freeze_support()
Process(target=f).start() Process(target=f).start()
If the ``freeze_support()`` line is missed out then trying to run the frozen If the ``freeze_support()`` line is omitted then trying to run the frozen
executable will raise :exc:`RuntimeError`. executable will raise :exc:`RuntimeError`.
If the module is being run normally by the Python interpreter then If the module is being run normally by the Python interpreter then
@ -766,8 +775,8 @@ Connection objects usually created using :func:`Pipe` -- see also
*buffer* must be an object satisfying the writable buffer interface. If *buffer* must be an object satisfying the writable buffer interface. If
*offset* is given then the message will be written into the buffer from *offset* is given then the message will be written into the buffer from
*that position. Offset must be a non-negative integer less than the that position. Offset must be a non-negative integer less than the
*length of *buffer* (in bytes). length of *buffer* (in bytes).
If the buffer is too short then a :exc:`BufferTooShort` exception is If the buffer is too short then a :exc:`BufferTooShort` exception is
raised and the complete message is available as ``e.args[0]`` where ``e`` raised and the complete message is available as ``e.args[0]`` where ``e``
@ -776,6 +785,8 @@ Connection objects usually created using :func:`Pipe` -- see also
For example: For example:
.. doctest::
>>> from multiprocessing import Pipe >>> from multiprocessing import Pipe
>>> a, b = Pipe() >>> a, b = Pipe()
>>> a.send([1, 'hello', None]) >>> a.send([1, 'hello', None])
@ -868,8 +879,9 @@ object -- see :ref:`multiprocessing-managers`.
specifies a timeout in seconds. If *block* is ``False`` then *timeout* is specifies a timeout in seconds. If *block* is ``False`` then *timeout* is
ignored. ignored.
Note that on OS/X ``sem_timedwait`` is unsupported, so timeout arguments .. note::
for these will be ignored. On OS/X ``sem_timedwait`` is unsupported, so timeout arguments for the
aforementioned :meth:`acquire` methods will be ignored on OS/X.
.. note:: .. note::
@ -1066,7 +1078,7 @@ process::
lock = Lock() lock = Lock()
n = Value('i', 7) n = Value('i', 7)
x = Value(ctypes.c_double, 1.0/3.0, lock=False) x = Value(c_double, 1.0/3.0, lock=False)
s = Array('c', 'hello world', lock=lock) s = Array('c', 'hello world', lock=lock)
A = Array(Point, [(1.875,-6.25), (-5.75,2.0), (2.375,9.5)], lock=lock) A = Array(Point, [(1.875,-6.25), (-5.75,2.0), (2.375,9.5)], lock=lock)
@ -1148,21 +1160,21 @@ their parent process exits. The manager classes are defined in the
Returns a :class:`Server` object which represents the actual server under Returns a :class:`Server` object which represents the actual server under
the control of the Manager. The :class:`Server` object supports the the control of the Manager. The :class:`Server` object supports the
:meth:`serve_forever` method: :meth:`serve_forever` method::
>>> from multiprocessing.managers import BaseManager >>> from multiprocessing.managers import BaseManager
>>> m = BaseManager(address=('', 50000), authkey='abc')) >>> manager = BaseManager(address=('', 50000), authkey='abc')
>>> server = m.get_server() >>> server = manager.get_server()
>>> s.serve_forever() >>> server.serve_forever()
:class:`Server` additionally have an :attr:`address` attribute. :class:`Server` additionally has an :attr:`address` attribute.
.. method:: connect() .. method:: connect()
Connect a local manager object to a remote manager process: Connect a local manager object to a remote manager process::
>>> from multiprocessing.managers import BaseManager >>> from multiprocessing.managers import BaseManager
>>> m = BaseManager(address='127.0.0.1', authkey='abc') >>> m = BaseManager(address=('127.0.0.1', 5000), authkey='abc')
>>> m.connect() >>> m.connect()
.. method:: shutdown() .. method:: shutdown()
@ -1290,7 +1302,9 @@ A namespace object has no public methods, but does have writable attributes.
Its representation shows the values of its attributes. Its representation shows the values of its attributes.
However, when using a proxy for a namespace object, an attribute beginning with However, when using a proxy for a namespace object, an attribute beginning with
``'_'`` will be an attribute of the proxy and not an attribute of the referent:: ``'_'`` will be an attribute of the proxy and not an attribute of the referent:
.. doctest::
>>> manager = multiprocessing.Manager() >>> manager = multiprocessing.Manager()
>>> Global = manager.Namespace() >>> Global = manager.Namespace()
@ -1342,17 +1356,15 @@ remote clients can access::
>>> import Queue >>> import Queue
>>> queue = Queue.Queue() >>> queue = Queue.Queue()
>>> class QueueManager(BaseManager): pass >>> class QueueManager(BaseManager): pass
...
>>> QueueManager.register('get_queue', callable=lambda:queue) >>> QueueManager.register('get_queue', callable=lambda:queue)
>>> m = QueueManager(address=('', 50000), authkey='abracadabra') >>> m = QueueManager(address=('', 50000), authkey='abracadabra')
>>> s = m.get_server() >>> s = m.get_server()
>>> s.serveForever() >>> s.serve_forever()
One client can access the server as follows:: One client can access the server as follows::
>>> from multiprocessing.managers import BaseManager >>> from multiprocessing.managers import BaseManager
>>> class QueueManager(BaseManager): pass >>> class QueueManager(BaseManager): pass
...
>>> QueueManager.register('get_queue') >>> QueueManager.register('get_queue')
>>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra') >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra')
>>> m.connect() >>> m.connect()
@ -1363,10 +1375,10 @@ Another client can also use it::
>>> from multiprocessing.managers import BaseManager >>> from multiprocessing.managers import BaseManager
>>> class QueueManager(BaseManager): pass >>> class QueueManager(BaseManager): pass
... >>> QueueManager.register('get_queue')
>>> QueueManager.register('getQueue') >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra')
>>> m = QueueManager.from_address(address=('foo.bar.org', 50000), authkey='abracadabra') >>> m.connect()
>>> queue = m.getQueue() >>> queue = m.get_queue()
>>> queue.get() >>> queue.get()
'hello' 'hello'
@ -1402,7 +1414,9 @@ proxy. Multiple proxy objects may have the same referent.
A proxy object has methods which invoke corresponding methods of its referent A proxy object has methods which invoke corresponding methods of its referent
(although not every method of the referent will necessarily be available through (although not every method of the referent will necessarily be available through
the proxy). A proxy can usually be used in most of the same ways that its the proxy). A proxy can usually be used in most of the same ways that its
referent can:: referent can:
.. doctest::
>>> from multiprocessing import Manager >>> from multiprocessing import Manager
>>> manager = Manager() >>> manager = Manager()
@ -1410,7 +1424,7 @@ referent can::
>>> print l >>> print l
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81] [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> print repr(l) >>> print repr(l)
<ListProxy object, typeid 'list' at 0xb799974c> <ListProxy object, typeid 'list' at 0x...>
>>> l[4] >>> l[4]
16 16
>>> l[2:5] >>> l[2:5]
@ -1423,7 +1437,9 @@ the proxy.
An important feature of proxy objects is that they are picklable so they can be An important feature of proxy objects is that they are picklable so they can be
passed between processes. Note, however, that if a proxy is sent to the passed between processes. Note, however, that if a proxy is sent to the
corresponding manager's process then unpickling it will produce the referent corresponding manager's process then unpickling it will produce the referent
itself. This means, for example, that one shared object can contain a second:: itself. This means, for example, that one shared object can contain a second:
.. doctest::
>>> a = manager.list() >>> a = manager.list()
>>> b = manager.list() >>> b = manager.list()
@ -1437,12 +1453,14 @@ itself. This means, for example, that one shared object can contain a second::
.. note:: .. note::
The proxy types in :mod:`multiprocessing` do nothing to support comparisons The proxy types in :mod:`multiprocessing` do nothing to support comparisons
by value. So, for instance, :: by value. So, for instance, we have:
manager.list([1,2,3]) == [1,2,3] .. doctest::
will return ``False``. One should just use a copy of the referent instead >>> manager.list([1,2,3]) == [1,2,3]
when making comparisons. False
One should just use a copy of the referent instead when making comparisons.
.. class:: BaseProxy .. class:: BaseProxy
@ -1474,7 +1492,9 @@ itself. This means, for example, that one shared object can contain a second::
Note in particular that an exception will be raised if *methodname* has Note in particular that an exception will be raised if *methodname* has
not been *exposed* not been *exposed*
An example of the usage of :meth:`_callmethod`:: An example of the usage of :meth:`_callmethod`:
.. doctest::
>>> l = manager.list(range(10)) >>> l = manager.list(range(10))
>>> l._callmethod('__len__') >>> l._callmethod('__len__')
@ -1899,12 +1919,12 @@ Below is an example session with logging turned on::
>>> logger.warning('doomed') >>> logger.warning('doomed')
[WARNING/MainProcess] doomed [WARNING/MainProcess] doomed
>>> m = multiprocessing.Manager() >>> m = multiprocessing.Manager()
[INFO/SyncManager-1] child process calling self.run() [INFO/SyncManager-...] child process calling self.run()
[INFO/SyncManager-1] created temp directory /.../pymp-Wh47O_ [INFO/SyncManager-...] created temp directory /.../pymp-...
[INFO/SyncManager-1] manager serving at '/.../listener-lWsERs' [INFO/SyncManager-...] manager serving at '/.../listener-...'
>>> del m >>> del m
[INFO/MainProcess] sending shutdown message to manager [INFO/MainProcess] sending shutdown message to manager
[INFO/SyncManager-1] manager exiting with exitcode 0 [INFO/SyncManager-...] manager exiting with exitcode 0
In addition to having these two logging functions, the multiprocessing also In addition to having these two logging functions, the multiprocessing also
exposes two additional logging level attributes. These are :const:`SUBWARNING` exposes two additional logging level attributes. These are :const:`SUBWARNING`
@ -1931,18 +1951,18 @@ with :const:`SUBDEBUG` enabled::
>>> logger.warning('doomed') >>> logger.warning('doomed')
[WARNING/MainProcess] doomed [WARNING/MainProcess] doomed
>>> m = multiprocessing.Manager() >>> m = multiprocessing.Manager()
[INFO/SyncManager-1] child process calling self.run() [INFO/SyncManager-...] child process calling self.run()
[INFO/SyncManager-1] created temp directory /.../pymp-djGBXN [INFO/SyncManager-...] created temp directory /.../pymp-...
[INFO/SyncManager-1] manager serving at '/.../pymp-djGBXN/listener-knBYGe' [INFO/SyncManager-...] manager serving at '/.../pymp-djGBXN/listener-...'
>>> del m >>> del m
[SUBDEBUG/MainProcess] finalizer calling ... [SUBDEBUG/MainProcess] finalizer calling ...
[INFO/MainProcess] sending shutdown message to manager [INFO/MainProcess] sending shutdown message to manager
[DEBUG/SyncManager-1] manager received shutdown message [DEBUG/SyncManager-...] manager received shutdown message
[SUBDEBUG/SyncManager-1] calling <Finalize object, callback=unlink, ... [SUBDEBUG/SyncManager-...] calling <Finalize object, callback=unlink, ...
[SUBDEBUG/SyncManager-1] finalizer calling <built-in function unlink> ... [SUBDEBUG/SyncManager-...] finalizer calling <built-in function unlink> ...
[SUBDEBUG/SyncManager-1] calling <Finalize object, dead> [SUBDEBUG/SyncManager-...] calling <Finalize object, dead>
[SUBDEBUG/SyncManager-1] finalizer calling <function rmtree at 0x5aa730> ... [SUBDEBUG/SyncManager-...] finalizer calling <function rmtree at 0x5aa730> ...
[INFO/SyncManager-1] manager exiting with exitcode 0 [INFO/SyncManager-...] manager exiting with exitcode 0
The :mod:`multiprocessing.dummy` module The :mod:`multiprocessing.dummy` module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~