#14770: improve the library FAQ.

This commit is contained in:
Ezio Melotti 2012-05-13 19:49:00 +03:00
parent 334be84540
commit 6176db5aac

View file

@ -14,7 +14,7 @@ How do I find a module or application to perform task X?
Check :ref:`the Library Reference <library-index>` to see if there's a relevant Check :ref:`the Library Reference <library-index>` to see if there's a relevant
standard library module. (Eventually you'll learn what's in the standard standard library module. (Eventually you'll learn what's in the standard
library and will able to skip this step.) library and will be able to skip this step.)
For third-party packages, search the `Python Package Index For third-party packages, search the `Python Package Index
<http://pypi.python.org/pypi>`_ or try `Google <http://www.google.com>`_ or <http://pypi.python.org/pypi>`_ or try `Google <http://www.google.com>`_ or
@ -28,7 +28,7 @@ Where is the math.py (socket.py, regex.py, etc.) source file?
If you can't find a source file for a module it may be a built-in or If you can't find a source file for a module it may be a built-in or
dynamically loaded module implemented in C, C++ or other compiled language. dynamically loaded module implemented in C, C++ or other compiled language.
In this case you may not have the source file or it may be something like In this case you may not have the source file or it may be something like
mathmodule.c, somewhere in a C source directory (not on the Python Path). :file:`mathmodule.c`, somewhere in a C source directory (not on the Python Path).
There are (at least) three kinds of modules in Python: There are (at least) three kinds of modules in Python:
@ -60,18 +60,18 @@ as the very first line of your file, using the pathname for where the Python
interpreter is installed on your platform. interpreter is installed on your platform.
If you would like the script to be independent of where the Python interpreter If you would like the script to be independent of where the Python interpreter
lives, you can use the "env" program. Almost all Unix variants support the lives, you can use the :program:`env` program. Almost all Unix variants support
following, assuming the Python interpreter is in a directory on the user's the following, assuming the Python interpreter is in a directory on the user's
$PATH:: :envvar:`PATH`::
#!/usr/bin/env python #!/usr/bin/env python
*Don't* do this for CGI scripts. The $PATH variable for CGI scripts is often *Don't* do this for CGI scripts. The :envvar:`PATH` variable for CGI scripts is
very minimal, so you need to use the actual absolute pathname of the often very minimal, so you need to use the actual absolute pathname of the
interpreter. interpreter.
Occasionally, a user's environment is so full that the /usr/bin/env program Occasionally, a user's environment is so full that the :program:`/usr/bin/env`
fails; or there's no env program at all. In that case, you can try the program fails; or there's no env program at all. In that case, you can try the
following hack (due to Alex Rezinsky):: following hack (due to Alex Rezinsky)::
#! /bin/sh #! /bin/sh
@ -92,11 +92,11 @@ Is there a curses/termcap package for Python?
.. XXX curses *is* built by default, isn't it? .. XXX curses *is* built by default, isn't it?
For Unix variants: The standard Python source distribution comes with a curses For Unix variants: The standard Python source distribution comes with a curses
module in the ``Modules/`` subdirectory, though it's not compiled by default module in the :source:`Modules` subdirectory, though it's not compiled by default.
(note that this is not available in the Windows distribution -- there is no (Note that this is not available in the Windows distribution -- there is no
curses module for Windows). curses module for Windows.)
The curses module supports basic curses features as well as many additional The :mod:`curses` module supports basic curses features as well as many additional
functions from ncurses and SYSV curses such as colour, alternative character set functions from ncurses and SYSV curses such as colour, alternative character set
support, pads, and mouse support. This means the module isn't compatible with support, pads, and mouse support. This means the module isn't compatible with
operating systems that only have BSD curses, but there don't seem to be any operating systems that only have BSD curses, but there don't seem to be any
@ -110,7 +110,7 @@ Is there an equivalent to C's onexit() in Python?
------------------------------------------------- -------------------------------------------------
The :mod:`atexit` module provides a register function that is similar to C's The :mod:`atexit` module provides a register function that is similar to C's
onexit. :c:func:`onexit`.
Why don't my signal handlers work? Why don't my signal handlers work?
@ -140,8 +140,8 @@ the expected output given in the docstring.
The :mod:`unittest` module is a fancier testing framework modelled on Java and The :mod:`unittest` module is a fancier testing framework modelled on Java and
Smalltalk testing frameworks. Smalltalk testing frameworks.
For testing, it helps to write the program so that it may be easily tested by To make testing easier, you should use good modular design in your program.
using good modular design. Your program should have almost all functionality Your program should have almost all functionality
encapsulated in either functions or class methods -- and this sometimes has the encapsulated in either functions or class methods -- and this sometimes has the
surprising and delightful effect of making the program run faster (because local surprising and delightful effect of making the program run faster (because local
variable accesses are faster than global accesses). Furthermore the program variable accesses are faster than global accesses). Furthermore the program
@ -157,7 +157,7 @@ at the bottom of the main module of your program.
Once your program is organized as a tractable collection of functions and class Once your program is organized as a tractable collection of functions and class
behaviours you should write test functions that exercise the behaviours. A test behaviours you should write test functions that exercise the behaviours. A test
suite can be associated with each module which automates a sequence of tests. suite that automates a sequence of tests can be associated with each module.
This sounds like a lot of work, but since Python is so terse and flexible it's This sounds like a lot of work, but since Python is so terse and flexible it's
surprisingly easy. You can make coding much more pleasant and fun by writing surprisingly easy. You can make coding much more pleasant and fun by writing
your test functions in parallel with the "production code", since this makes it your test functions in parallel with the "production code", since this makes it
@ -186,7 +186,7 @@ docstrings is `epydoc <http://epydoc.sf.net/>`_. `Sphinx
How do I get a single keypress at a time? How do I get a single keypress at a time?
----------------------------------------- -----------------------------------------
For Unix variants: There are several solutions. It's straightforward to do this For Unix variants there are several solutions. It's straightforward to do this
using curses, but curses is a fairly large module to learn. Here's a solution using curses, but curses is a fairly large module to learn. Here's a solution
without curses:: without curses::
@ -273,7 +273,7 @@ A simple fix is to add a tiny sleep to the start of the run function::
time.sleep(10) time.sleep(10)
Instead of trying to guess how long a :func:`time.sleep` delay will be enough, Instead of trying to guess a good delay value for :func:`time.sleep`,
it's better to use some kind of semaphore mechanism. One idea is to use the it's better to use some kind of semaphore mechanism. One idea is to use the
:mod:`Queue` module to create a queue object, let each thread append a token to :mod:`Queue` module to create a queue object, let each thread append a token to
the queue when it finishes, and let the main thread read as many tokens from the the queue when it finishes, and let the main thread read as many tokens from the
@ -284,10 +284,10 @@ How do I parcel out work among a bunch of worker threads?
--------------------------------------------------------- ---------------------------------------------------------
Use the :mod:`Queue` module to create a queue containing a list of jobs. The Use the :mod:`Queue` module to create a queue containing a list of jobs. The
:class:`~Queue.Queue` class maintains a list of objects with ``.put(obj)`` to :class:`~Queue.Queue` class maintains a list of objects and has a ``.put(obj)``
add an item to the queue and ``.get()`` to return an item. The class will take method that adds items to the queue and a ``.get()`` method to return them.
care of the locking necessary to ensure that each job is handed out exactly The class will take care of the locking necessary to ensure that each job is
once. handed out exactly once.
Here's a trivial example:: Here's a trivial example::
@ -329,6 +329,8 @@ Here's a trivial example::
When run, this will produce the following output: When run, this will produce the following output:
.. code-block:: none
Running worker Running worker
Running worker Running worker
Running worker Running worker
@ -343,15 +345,15 @@ When run, this will produce the following output:
Worker <Thread(worker 1, started)> running with argument 5 Worker <Thread(worker 1, started)> running with argument 5
... ...
Consult the module's documentation for more details; the ``Queue`` class Consult the module's documentation for more details; the :class:`~Queue.Queue`
provides a featureful interface. class provides a featureful interface.
What kinds of global value mutation are thread-safe? What kinds of global value mutation are thread-safe?
---------------------------------------------------- ----------------------------------------------------
A global interpreter lock (GIL) is used internally to ensure that only one A global interpreter lock (:term:`GIL`) is used internally to ensure that only
thread runs in the Python VM at a time. In general, Python offers to switch one thread runs in the Python VM at a time. In general, Python offers to switch
among threads only between bytecode instructions; how frequently it switches can among threads only between bytecode instructions; how frequently it switches can
be set via :func:`sys.setcheckinterval`. Each bytecode instruction and be set via :func:`sys.setcheckinterval`. Each bytecode instruction and
therefore all the C implementation code reached from each instruction is therefore all the C implementation code reached from each instruction is
@ -396,7 +398,7 @@ Can't we get rid of the Global Interpreter Lock?
.. XXX mention multiprocessing .. XXX mention multiprocessing
.. XXX link to dbeazley's talk about GIL? .. XXX link to dbeazley's talk about GIL?
The Global Interpreter Lock (GIL) is often seen as a hindrance to Python's The Global Interpreter Lock (:term:`GIL`) is often seen as a hindrance to Python's
deployment on high-end multiprocessor server machines, because a multi-threaded deployment on high-end multiprocessor server machines, because a multi-threaded
Python program effectively only uses one CPU, due to the insistence that Python program effectively only uses one CPU, due to the insistence that
(almost) all Python code can only run while the GIL is held. (almost) all Python code can only run while the GIL is held.
@ -459,7 +461,7 @@ To rename a file, use ``os.rename(old_path, new_path)``.
To truncate a file, open it using ``f = open(filename, "r+")``, and use To truncate a file, open it using ``f = open(filename, "r+")``, and use
``f.truncate(offset)``; offset defaults to the current seek position. There's ``f.truncate(offset)``; offset defaults to the current seek position. There's
also ``os.ftruncate(fd, offset)`` for files opened with :func:`os.open`, where also ``os.ftruncate(fd, offset)`` for files opened with :func:`os.open`, where
``fd`` is the file descriptor (a small integer). *fd* is the file descriptor (a small integer).
The :mod:`shutil` module also contains a number of functions to work on files The :mod:`shutil` module also contains a number of functions to work on files
including :func:`~shutil.copyfile`, :func:`~shutil.copytree`, and including :func:`~shutil.copyfile`, :func:`~shutil.copytree`, and
@ -493,7 +495,7 @@ The '>' in the format string forces big-endian data; the letter 'h' reads one
"short integer" (2 bytes), and 'l' reads one "long integer" (4 bytes) from the "short integer" (2 bytes), and 'l' reads one "long integer" (4 bytes) from the
string. string.
For data that is more regular (e.g. a homogeneous list of ints or thefloats), For data that is more regular (e.g. a homogeneous list of ints or floats),
you can also use the :mod:`array` module. you can also use the :mod:`array` module.
@ -503,7 +505,7 @@ I can't seem to use os.read() on a pipe created with os.popen(); why?
:func:`os.read` is a low-level function which takes a file descriptor, a small :func:`os.read` is a low-level function which takes a file descriptor, a small
integer representing the opened file. :func:`os.popen` creates a high-level integer representing the opened file. :func:`os.popen` creates a high-level
file object, the same type returned by the built-in :func:`open` function. file object, the same type returned by the built-in :func:`open` function.
Thus, to read n bytes from a pipe p created with :func:`os.popen`, you need to Thus, to read *n* bytes from a pipe *p* created with :func:`os.popen`, you need to
use ``p.read(n)``. use ``p.read(n)``.
@ -522,9 +524,9 @@ Use the :mod:`popen2` module. For example::
Warning: in general it is unwise to do this because you can easily cause a Warning: in general it is unwise to do this because you can easily cause a
deadlock where your process is blocked waiting for output from the child while deadlock where your process is blocked waiting for output from the child while
the child is blocked waiting for input from you. This can be caused because the the child is blocked waiting for input from you. This can be caused by the
parent expects the child to output more text than it does, or it can be caused parent expecting the child to output more text than it does or by data being
by data being stuck in stdio buffers due to lack of flushing. The Python parent stuck in stdio buffers due to lack of flushing. The Python parent
can of course explicitly flush the data it sends to the child before it reads can of course explicitly flush the data it sends to the child before it reads
any output, but if the child is a naive C program it may have been written to any output, but if the child is a naive C program it may have been written to
never explicitly flush its output, even if it is interactive, since flushing is never explicitly flush its output, even if it is interactive, since flushing is
@ -544,8 +546,8 @@ place to insert such a call would be before calling ``popen2`` again.
In many cases, all you really need is to run some data through a command and get In many cases, all you really need is to run some data through a command and get
the result back. Unless the amount of data is very large, the easiest way to do the result back. Unless the amount of data is very large, the easiest way to do
this is to write it to a temporary file and run the command with that temporary this is to write it to a temporary file and run the command with that temporary
file as input. The standard module :mod:`tempfile` exports a ``mktemp()`` file as input. The standard module :mod:`tempfile` exports a
function to generate unique temporary file names. :: :func:`~tempfile.mktemp` function to generate unique temporary file names. ::
import tempfile import tempfile
import os import os
@ -673,7 +675,8 @@ Yes. Here's a simple example that uses httplib::
sys.stdout.write(httpobj.getfile().read()) sys.stdout.write(httpobj.getfile().read())
Note that in general for percent-encoded POST operations, query strings must be Note that in general for percent-encoded POST operations, query strings must be
quoted using :func:`urllib.quote`. For example to send name="Guy Steele, Jr.":: quoted using :func:`urllib.quote`. For example, to send
``name="Guy Steele, Jr."``::
>>> from urllib import quote >>> from urllib import quote
>>> x = quote("Guy Steele, Jr.") >>> x = quote("Guy Steele, Jr.")
@ -689,19 +692,8 @@ What module should I use to help with generating HTML?
.. XXX add modern template languages .. XXX add modern template languages
There are many different modules available: You can find a collection of useful links on the `Web Programming wiki page
<http://wiki.python.org/moin/WebProgramming>`_.
* HTMLgen is a class library of objects corresponding to all the HTML 3.2 markup
tags. It's used when you are writing in Python and wish to synthesize HTML
pages for generating a web or for CGI forms, etc.
* DocumentTemplate and Zope Page Templates are two different systems that are
part of Zope.
* Quixote's PTL uses Python syntax to assemble strings of text.
Consult the `Web Programming wiki pages
<http://wiki.python.org/moin/WebProgramming>`_ for more links.
How do I send mail from a Python script? How do I send mail from a Python script?
@ -730,7 +722,7 @@ work on any host that supports an SMTP listener. ::
server.quit() server.quit()
A Unix-only alternative uses sendmail. The location of the sendmail program A Unix-only alternative uses sendmail. The location of the sendmail program
varies between systems; sometimes it is ``/usr/lib/sendmail``, sometime varies between systems; sometimes it is ``/usr/lib/sendmail``, sometimes
``/usr/sbin/sendmail``. The sendmail manual page will help you out. Here's ``/usr/sbin/sendmail``. The sendmail manual page will help you out. Here's
some sample code:: some sample code::
@ -797,7 +789,7 @@ A more awkward way of doing things is to use pickle's little sister, marshal.
The :mod:`marshal` module provides very fast ways to store noncircular basic The :mod:`marshal` module provides very fast ways to store noncircular basic
Python types to files and strings, and back again. Although marshal does not do Python types to files and strings, and back again. Although marshal does not do
fancy things like store instances or handle shared references properly, it does fancy things like store instances or handle shared references properly, it does
run extremely fast. For example loading a half megabyte of data may take less run extremely fast. For example, loading a half megabyte of data may take less
than a third of a second. This often beats doing something more complex and than a third of a second. This often beats doing something more complex and
general such as using gdbm with pickle/shelve. general such as using gdbm with pickle/shelve.
@ -807,9 +799,9 @@ Why is cPickle so slow?
.. XXX update this, default protocol is 2/3 .. XXX update this, default protocol is 2/3
The default format used by the pickle module is a slow one that results in By default :mod:`pickle` uses a relatively old and slow format for backward
readable pickles. Making it the default, but it would break backward compatibility. You can however specify other protocol versions that are
compatibility:: faster::
largeString = 'z' * (100 * 1024) largeString = 'z' * (100 * 1024)
myPickle = cPickle.dumps(largeString, protocol=1) myPickle = cPickle.dumps(largeString, protocol=1)