mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-04 17:08:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

bpo-41586: Add pipesize parameter to subprocess & F_GETPIPE_SZ and F_SETPIPE_SZ to fcntl. (GH-21921)

Browse source 
* Add F_SETPIPE_SZ and F_GETPIPE_SZ to fcntl module
* Add pipesize parameter for subprocess.Popen class

This will allow the user to control the size of the pipes.
On linux the default is 64K. When a pipe is full it blocks for writing.
When a pipe is empty it blocks for reading. On processes that are
very fast this can lead to a lot of wasted CPU cycles. On a typical
Linux system the max pipe size is 1024K which is much better.
For high performance-oriented libraries such as xopen it is nice to
be able to set the pipe size.

The workaround without this feature is to use my_popen_process.stdout.fileno() in
conjuction with fcntl and 1031 (value of F_SETPIPE_SZ) to acquire this behavior.
This commit is contained in:
Ruben Vorderman 2020-10-20 01:30:02 +02:00 • committed by GitHub
parent bf838227c3
commit 23c0fb8edd
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 102 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

10
Doc/library/subprocess.rst
View file

@ -341,7 +341,7 @@ functions.
startupinfo=None, creationflags=0, restore_signals=True, \
start_new_session=False, pass_fds=(), \*, group=None, \
extra_groups=None, user=None, umask=-1, \
encoding=None, errors=None, text=None)
encoding=None, errors=None, text=None, pipesize=-1)
Execute a child program in a new process. On POSIX, the class uses
:meth:`os.execvp`-like behavior to execute the child program. On Windows,
@ -625,6 +625,14 @@ functions.
* :data:`CREATE_DEFAULT_ERROR_MODE`
* :data:`CREATE_BREAKAWAY_FROM_JOB`
*pipesize* can be used to change the size of the pipe when
:data:`PIPE` is used for *stdin*, *stdout* or *stderr*. The size of the pipe
is only changed on platforms that support this (only Linux at this time of
writing). Other platforms will ignore this parameter.
.. versionadded:: 3.10
The ``pipesize`` parameter was added.
Popen objects are supported as context managers via the :keyword:`with` statement:
on exit, standard file descriptors are closed, and the process is waited for.
::