mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-11-03 19:34:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

Merged revisions 55328-55341 via svnmerge from

Browse source 
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

........
  r55329 | brett.cannon | 2007-05-14 16:36:56 -0700 (Mon, 14 May 2007) | 3 lines

  Implement the removal of tuple parameter unpacking (PEP 3113).
  Thanks, Tony Lownds for the patch.
........
  r55331 | neal.norwitz | 2007-05-14 16:40:30 -0700 (Mon, 14 May 2007) | 1 line

  Update to use Python 3.0
........
  r55332 | brett.cannon | 2007-05-14 16:47:18 -0700 (Mon, 14 May 2007) | 2 lines

  Mention PEP 3113.  And thanks to Tony Lownds for the PEP 3113 patch.
........
  r55333 | neal.norwitz | 2007-05-14 16:57:06 -0700 (Mon, 14 May 2007) | 1 line

  Fix exception printing (no more exceptions module)
........
  r55334 | neal.norwitz | 2007-05-14 17:11:10 -0700 (Mon, 14 May 2007) | 1 line

  Remove popen* functions from os
........
  r55335 | neal.norwitz | 2007-05-14 18:03:38 -0700 (Mon, 14 May 2007) | 1 line

  Get rid of most of popen.  There are still some uses I need to cleanup.
........
  r55336 | neal.norwitz | 2007-05-14 21:11:34 -0700 (Mon, 14 May 2007) | 1 line

  Remove a few more remnants of the compiler package
........
  r55337 | neal.norwitz | 2007-05-14 22:28:27 -0700 (Mon, 14 May 2007) | 1 line

  Get test_[cx]pickle working on 64-bit platforms (avoid overflow int/long)
........
This commit is contained in:
Guido van Rossum 2007-05-15 18:46:22 +00:00
parent 360e4b8fb1
commit 1bc535dc78
64 changed files with 1614 additions and 2842 deletions
Download patch file Download diff file Expand all files Collapse all files

1
Doc/lib/lib.tex
View file

@ -280,7 +280,6 @@ and how to embed it in other applications.
\input{libsubprocess}
\input{libsocket}
\input{libsignal}
\input{libpopen2}
\input{libasyncore}
\input{libasynchat}

2
Doc/lib/libitertools.tex
View file

@ -459,7 +459,7 @@ Mark
# is differencing with a range so that consecutive numbers all appear in
# same group.
>>> data = [ 1, 4,5,6, 10, 15,16,17,18, 22, 25,26,27,28]
>>> for k, g in groupby(enumerate(data), lambda (i,x):i-x):
>>> for k, g in groupby(enumerate(data), lambda t:t[0]-t[1]):
... print map(operator.itemgetter(1), g)
...
[1]

65
Doc/lib/libos.tex
View file

@ -378,68 +378,6 @@ deleted once there are no file descriptors for the file.
Availability: Macintosh, \UNIX, Windows.
\end{funcdesc}
There are a number of different \function{popen*()} functions that
provide slightly different ways to create subprocesses.
\deprecated{2.6}{All of the \function{popen*()} functions are obsolete.
Use the \module{subprocess} module.}
For each of the \function{popen*()} variants, if \var{bufsize} is
specified, it specifies the buffer size for the I/O pipes.
\var{mode}, if provided, should be the string \code{'b'} or
\code{'t'}; on Windows this is needed to determine whether the file
objects should be opened in binary or text mode. The default value
for \var{mode} is \code{'t'}.
Also, for each of these variants, on \UNIX, \var{cmd} may be a sequence, in
which case arguments will be passed directly to the program without shell
intervention (as with \function{os.spawnv()}). If \var{cmd} is a string it will
be passed to the shell (as with \function{os.system()}).
These methods do not make it possible to retrieve the exit status from
the child processes. The only way to control the input and output
streams and also retrieve the return codes is to use the
\refmodule{subprocess} module; these are only available on \UNIX.
For a discussion of possible deadlock conditions related to the use
of these functions, see ``\ulink{Flow Control
Issues}{popen2-flow-control.html}''
(section~\ref{popen2-flow-control}).
\begin{funcdesc}{popen2}{cmd\optional{, mode\optional{, bufsize}}}
Executes \var{cmd} as a sub-process. Returns the file objects
\code{(\var{child_stdin}, \var{child_stdout})}.
\deprecated{2.6}{All of the \function{popen*()} functions are obsolete.
Use the \module{subprocess} module.}
Availability: Macintosh, \UNIX, Windows.
\versionadded{2.0}
\end{funcdesc}
\begin{funcdesc}{popen3}{cmd\optional{, mode\optional{, bufsize}}}
Executes \var{cmd} as a sub-process. Returns the file objects
\code{(\var{child_stdin}, \var{child_stdout}, \var{child_stderr})}.
\deprecated{2.6}{All of the \function{popen*()} functions are obsolete.
Use the \module{subprocess} module.}
Availability: Macintosh, \UNIX, Windows.
\versionadded{2.0}
\end{funcdesc}
\begin{funcdesc}{popen4}{cmd\optional{, mode\optional{, bufsize}}}
Executes \var{cmd} as a sub-process. Returns the file objects
\code{(\var{child_stdin}, \var{child_stdout_and_stderr})}.
\deprecated{2.6}{All of the \function{popen*()} functions are obsolete.
Use the \module{subprocess} module.}
Availability: Macintosh, \UNIX, Windows.
\versionadded{2.0}
\end{funcdesc}
(Note that \code{\var{child_stdin}, \var{child_stdout}, and
\var{child_stderr}} are named from the point of view of the child
process, so \var{child_stdin} is the child's standard input.)
This functionality is also available in the \refmodule{popen2} module
using functions of the same names, but the return values of those
functions have a different order.
\subsection{File Descriptor Operations \label{os-fd-ops}}
@ -1575,9 +1513,6 @@ Availability: Macintosh, \UNIX.
\end{funcdesc}
\begin{funcdescni}{popen}{\unspecified}
\funclineni{popen2}{\unspecified}
\funclineni{popen3}{\unspecified}
\funclineni{popen4}{\unspecified}
Run child processes, returning opened pipes for communications. These
functions are described in section \ref{os-newstreams}.
\end{funcdescni}

190
Doc/lib/libpopen2.tex
View file

@ -1,190 +0,0 @@
\section{\module{popen2} ---
Subprocesses with accessible I/O streams}
\declaremodule{standard}{popen2}
\modulesynopsis{Subprocesses with accessible standard I/O streams.}
\sectionauthor{Drew Csillag}{drew_csillag@geocities.com}
\deprecated{2.6}{This module is obsolete. Use the \module{subprocess} module.}
This module allows you to spawn processes and connect to their
input/output/error pipes and obtain their return codes under
\UNIX{} and Windows.
The \module{subprocess} module provides more powerful facilities for
spawning new processes and retrieving their results. Using the
\module{subprocess} module is preferable to using the \module{popen2}
module.
The primary interface offered by this module is a trio of factory
functions. For each of these, if \var{bufsize} is specified,
it specifies the buffer size for the I/O pipes. \var{mode}, if
provided, should be the string \code{'b'} or \code{'t'}; on Windows
this is needed to determine whether the file objects should be opened
in binary or text mode. The default value for \var{mode} is
\code{'t'}.
On \UNIX, \var{cmd} may be a sequence, in which case arguments will be passed
directly to the program without shell intervention (as with
\function{os.spawnv()}). If \var{cmd} is a string it will be passed to the
shell (as with \function{os.system()}).
The only way to retrieve the return codes for the child processes is
by using the \method{poll()} or \method{wait()} methods on the
\class{Popen3} and \class{Popen4} classes; these are only available on
\UNIX. This information is not available when using the
\function{popen2()}, \function{popen3()}, and \function{popen4()}
functions, or the equivalent functions in the \refmodule{os} module.
(Note that the tuples returned by the \refmodule{os} module's functions
are in a different order from the ones returned by the \module{popen2}
module.)
\begin{funcdesc}{popen2}{cmd\optional{, bufsize\optional{, mode}}}
Executes \var{cmd} as a sub-process. Returns the file objects
\code{(\var{child_stdout}, \var{child_stdin})}.
\end{funcdesc}
\begin{funcdesc}{popen3}{cmd\optional{, bufsize\optional{, mode}}}
Executes \var{cmd} as a sub-process. Returns the file objects
\code{(\var{child_stdout}, \var{child_stdin}, \var{child_stderr})}.
\end{funcdesc}
\begin{funcdesc}{popen4}{cmd\optional{, bufsize\optional{, mode}}}
Executes \var{cmd} as a sub-process. Returns the file objects
\code{(\var{child_stdout_and_stderr}, \var{child_stdin})}.
\versionadded{2.0}
\end{funcdesc}
On \UNIX, a class defining the objects returned by the factory
functions is also available. These are not used for the Windows
implementation, and are not available on that platform.
\begin{classdesc}{Popen3}{cmd\optional{, capturestderr\optional{, bufsize}}}
This class represents a child process. Normally, \class{Popen3}
instances are created using the \function{popen2()} and
\function{popen3()} factory functions described above.
If not using one of the helper functions to create \class{Popen3}
objects, the parameter \var{cmd} is the shell command to execute in a
sub-process. The \var{capturestderr} flag, if true, specifies that
the object should capture standard error output of the child process.
The default is false. If the \var{bufsize} parameter is specified, it
specifies the size of the I/O buffers to/from the child process.
\end{classdesc}
\begin{classdesc}{Popen4}{cmd\optional{, bufsize}}
Similar to \class{Popen3}, but always captures standard error into the
same file object as standard output. These are typically created
using \function{popen4()}.
\versionadded{2.0}
\end{classdesc}
\subsection{Popen3 and Popen4 Objects \label{popen3-objects}}
Instances of the \class{Popen3} and \class{Popen4} classes have the
following methods:
\begin{methoddesc}[Popen3]{poll}{}
Returns \code{-1} if child process hasn't completed yet, or its return
code otherwise.
\end{methoddesc}
\begin{methoddesc}[Popen3]{wait}{}
Waits for and returns the status code of the child process. The
status code encodes both the return code of the process and
information about whether it exited using the \cfunction{exit()}
system call or died due to a signal. Functions to help interpret the
status code are defined in the \refmodule{os} module; see section
\ref{os-process} for the \function{W\var{*}()} family of functions.
\end{methoddesc}
The following attributes are also available:
\begin{memberdesc}[Popen3]{fromchild}
A file object that provides output from the child process. For
\class{Popen4} instances, this will provide both the standard output
and standard error streams.
\end{memberdesc}
\begin{memberdesc}[Popen3]{tochild}
A file object that provides input to the child process.
\end{memberdesc}
\begin{memberdesc}[Popen3]{childerr}
A file object that provides error output from the child process, if
\var{capturestderr} was true for the constructor, otherwise
\code{None}. This will always be \code{None} for \class{Popen4}
instances.
\end{memberdesc}
\begin{memberdesc}[Popen3]{pid}
The process ID of the child process.
\end{memberdesc}
\subsection{Flow Control Issues \label{popen2-flow-control}}
Any time you are working with any form of inter-process communication,
control flow needs to be carefully thought out. This remains the case
with the file objects provided by this module (or the \refmodule{os}
module equivalents).
% Example explanation and suggested work-arounds substantially stolen
% from Martin von Löwis:
% http://mail.python.org/pipermail/python-dev/2000-September/009460.html
When reading output from a child process that writes a lot of data to
standard error while the parent is reading from the child's standard
output, a deadlock can occur. A similar situation can occur with other
combinations of reads and writes. The essential factors are that more
than \constant{_PC_PIPE_BUF} bytes are being written by one process in
a blocking fashion, while the other process is reading from the other
process, also in a blocking fashion.
There are several ways to deal with this situation.
The simplest application change, in many cases, will be to follow this
model in the parent process:
\begin{verbatim}
import popen2
r, w, e = popen2.popen3('python slave.py')
e.readlines()
r.readlines()
r.close()
e.close()
w.close()
\end{verbatim}
with code like this in the child:
\begin{verbatim}
import os
import sys
# note that each of these print statements
# writes a single long string
print >>sys.stderr, 400 * 'this is a test\n'
os.close(sys.stderr.fileno())
print >>sys.stdout, 400 * 'this is another test\n'
\end{verbatim}
In particular, note that \code{sys.stderr} must be closed after
writing all data, or \method{readlines()} won't return. Also note
that \function{os.close()} must be used, as \code{sys.stderr.close()}
won't close \code{stderr} (otherwise assigning to \code{sys.stderr}
will silently close it, so no further errors can be printed).
Applications which need to support a more general approach should
integrate I/O over pipes with their \function{select()} loops, or use
separate threads to read each of the individual files provided by
whichever \function{popen*()} function or \class{Popen*} class was
used.
\begin{seealso}
\seemodule{subprocess}{Module for spawning and managing subprocesses.}
\end{seealso}

67
Doc/lib/libsubprocess.tex
View file

@ -15,8 +15,6 @@ and functions, such as:
\begin{verbatim}
os.system
os.spawn*
os.popen*
popen2.*
commands.*
\end{verbatim}
@ -335,68 +333,3 @@ pipe = os.popen(cmd, mode='w', bufsize)
==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
\end{verbatim}
\begin{verbatim}
(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
==>
p = Popen(cmd, shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdin, child_stdout) = (p.stdin, p.stdout)
\end{verbatim}
\begin{verbatim}
(child_stdin,
child_stdout,
child_stderr) = os.popen3(cmd, mode, bufsize)
==>
p = Popen(cmd, shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
(child_stdin,
child_stdout,
child_stderr) = (p.stdin, p.stdout, p.stderr)
\end{verbatim}
\begin{verbatim}
(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
==>
p = Popen(cmd, shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
\end{verbatim}
\subsubsection{Replacing popen2.*}
\note{If the cmd argument to popen2 functions is a string, the command
is executed through /bin/sh. If it is a list, the command is directly
executed.}
\begin{verbatim}
(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
==>
p = Popen(["somestring"], shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdout, child_stdin) = (p.stdout, p.stdin)
\end{verbatim}
\begin{verbatim}
(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
==>
p = Popen(["mycmd", "myarg"], bufsize=bufsize,
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdout, child_stdin) = (p.stdout, p.stdin)
\end{verbatim}
The popen2.Popen3 and popen2.Popen4 basically works as subprocess.Popen,
except that:
\begin{itemize}
\item subprocess.Popen raises an exception if the execution fails
\item the \var{capturestderr} argument is replaced with the \var{stderr}
argument.
\item stdin=PIPE and stdout=PIPE must be specified.
\item popen2 closes all file descriptors by default, but you have to
specify close_fds=True with subprocess.Popen.
\end{itemize}