Major overhaul of timeout sockets:

- setblocking(0) and settimeout(0) are now equivalent, and ditto for
  setblocking(1) and settimeout(None).

- Don't raise an exception from internal_select(); let the final call
  report the error (this means you will get an EAGAIN error instead of
  an ETIMEDOUT error -- I don't care).

- Move the select to inside the Py_{BEGIN,END}_ALLOW_THREADS brackets,
  so other theads can run (this was a bug in the original code).

- Redid the retry logic in connect() and connect_ex() to avoid masking
  errors.  This probably doesn't work for Windows yet; I'll fix that
  next.  It may also fail on other platforms, depending on what
  retrying a connect does; I need help with this.

- Get rid of the retry logic in accept().  I don't think it was needed
  at all.  But I may be wrong.
This commit is contained in:
Guido van Rossum 2002-06-13 15:07:44 +00:00
parent dfad1a9039
commit 11ba094957
5 changed files with 89 additions and 247 deletions

View file

@ -72,8 +72,9 @@ argument types and out-of-memory conditions can be raised; errors
related to socket or address semantics raise the error related to socket or address semantics raise the error
\exception{socket.error}. \exception{socket.error}.
Non-blocking mode is supported through the Non-blocking mode is supported through
\method{setblocking()} method. \method{setblocking()}. A generalization of this based on timeouts
is supported through \method{settimeout()}.
The module \module{socket} exports the following constants and functions: The module \module{socket} exports the following constants and functions:
@ -284,8 +285,7 @@ checked --- subsequent operations on the object may fail if the file
descriptor is invalid. This function is rarely needed, but can be descriptor is invalid. This function is rarely needed, but can be
used to get or set socket options on a socket passed to a program as used to get or set socket options on a socket passed to a program as
standard input or output (such as a server started by the \UNIX{} inet standard input or output (such as a server started by the \UNIX{} inet
daemon). The socket is assumed to be in blocking mode without daemon). The socket is assumed to be in blocking mode.
a timeout.
Availability: \UNIX. Availability: \UNIX.
\end{funcdesc} \end{funcdesc}
@ -514,38 +514,39 @@ all sockets are in blocking mode. In non-blocking mode, if a
\method{send()} call can't immediately dispose of the data, a \method{send()} call can't immediately dispose of the data, a
\exception{error} exception is raised; in blocking mode, the calls \exception{error} exception is raised; in blocking mode, the calls
block until they can proceed. block until they can proceed.
\code{s.setblocking(0)} is equivalent to \code{s.settimeout(0)};
\code{s.setblocking(1)} is equivalent to \code{s.settimeout(None)}.
\end{methoddesc} \end{methoddesc}
\begin{methoddesc}[socket]{settimeout}{value} \begin{methoddesc}[socket]{settimeout}{value}
Set a timeout on blocking socket operations. Value can be a Set a timeout on blocking socket operations. The \var{value} argument
nonnegative float expressing seconds, or \code{None}. If a float is can be a nonnegative float expressing seconds, or \code{None}.
If a float is
given, subsequent socket operations will raise an \exception{error} given, subsequent socket operations will raise an \exception{error}
exception if the timeout period \var{value} has elapsed before the exception if the timeout period \var{value} has elapsed before the
operation has completed. Setting a timeout of \code{None} disables operation has completed. Setting a timeout of \code{None} disables
timeouts on socket operations. timeouts on socket operations.
\code{s.settimeout(0.0)} is equivalent to \code{s.blocking(0)};
\code{s.settimeout(None)} is equivalent to \code{s.setblocking(1)}.
\versionadded{2.3} \versionadded{2.3}
\end{methoddesc} \end{methoddesc}
\begin{methoddesc}[socket]{gettimeout}{} \begin{methoddesc}[socket]{gettimeout}{}
Returns the timeout in floating seconds associated with socket Returns the timeout in floating seconds associated with socket
operations, or \code{None} if no timeout is set. operations, or \code{None} if no timeout is set. This reflects
the last call to \method{setblocking()} or \method{settimeout()}.
\versionadded{2.3} \versionadded{2.3}
\end{methoddesc} \end{methoddesc}
Some notes on the interaction between socket blocking and timeouts: A Some notes on socket blocking and timeouts: A socket object can be in
socket object can be in one of three modes: blocking, non-blocking, or one of three modes: blocking, non-blocking, or timout. Sockets are
timout. Sockets are always created in blocking mode. In blocking always created in blocking mode. In blocking mode, operations block
mode, operations block until complete. In non-blocking mode, until complete. In non-blocking mode, operations fail (with an error
operations fail (with an error that is unfortunately system-dependent) that is unfortunately system-dependent) if they cannot be completed
if they cannot be completed immediately. In timeout mode, operations immediately. In timeout mode, operations fail if they cannot be
fail if they cannot be completed within the timeout specified for the completed within the timeout specified for the socket. The
socket. \method{setblocking()} method is simply a shorthand for certain
\method{settimeout()} calls.
Calling \method{settimeout()} cancels non-blocking mode as set by
\method{setblocking()}; calling \method{setblocking()} cancels a
previously set timeout. Setting the timeout to zero acts similarly
but is implemented different than setting the socket in non-blocking
mode (this could be considered a bug and may even be fixed).
Timeout mode internally sets the socket in non-blocking mode. The Timeout mode internally sets the socket in non-blocking mode. The
blocking and timeout modes are shared between file descriptors and blocking and timeout modes are shared between file descriptors and

View file

@ -51,7 +51,10 @@ class ThreadableTest:
self.queue = Queue.Queue(1) self.queue = Queue.Queue(1)
# Do some munging to start the client test. # Do some munging to start the client test.
test_method = getattr(self, '_' + self._TestCase__testMethodName) methodname = self.id()
i = methodname.rfind('.')
methodname = methodname[i+1:]
test_method = getattr(self, '_' + methodname)
self.client_thread = thread.start_new_thread( self.client_thread = thread.start_new_thread(
self.clientRun, (test_method,)) self.clientRun, (test_method,))

View file

@ -59,17 +59,17 @@ class CreationTestCase(unittest.TestCase):
self.assertRaises(ValueError, self.sock.settimeout, -1L) self.assertRaises(ValueError, self.sock.settimeout, -1L)
self.assertRaises(ValueError, self.sock.settimeout, -1.0) self.assertRaises(ValueError, self.sock.settimeout, -1.0)
def testTimeoutThenoBlocking(self): def testTimeoutThenBlocking(self):
"Test settimeout() followed by setblocking()" "Test settimeout() followed by setblocking()"
self.sock.settimeout(10) self.sock.settimeout(10)
self.sock.setblocking(1) self.sock.setblocking(1)
self.assertEqual(self.sock.gettimeout(), None) self.assertEqual(self.sock.gettimeout(), None)
self.sock.setblocking(0) self.sock.setblocking(0)
self.assertEqual(self.sock.gettimeout(), None) self.assertEqual(self.sock.gettimeout(), 0.0)
self.sock.settimeout(10) self.sock.settimeout(10)
self.sock.setblocking(0) self.sock.setblocking(0)
self.assertEqual(self.sock.gettimeout(), None) self.assertEqual(self.sock.gettimeout(), 0.0)
self.sock.setblocking(1) self.sock.setblocking(1)
self.assertEqual(self.sock.gettimeout(), None) self.assertEqual(self.sock.gettimeout(), None)

View file

@ -446,26 +446,6 @@ set_gaierror(int error)
return NULL; return NULL;
} }
/* For timeout errors */
static PyObject *
timeout_err(void)
{
PyObject *v;
#ifdef MS_WINDOWS
v = Py_BuildValue("(is)", WSAETIMEDOUT, "Socket operation timed out");
#else
v = Py_BuildValue("(is)", ETIMEDOUT, "Socket operation timed out");
#endif
if (v != NULL) {
PyErr_SetObject(socket_error, v);
Py_DECREF(v);
}
return NULL;
}
/* Function to perform the setting of socket blocking mode /* Function to perform the setting of socket blocking mode
internally. block = (1 | 0). */ internally. block = (1 | 0). */
static int static int
@ -508,16 +488,18 @@ internal_setblocking(PySocketSockObject *s, int block)
return 1; return 1;
} }
/* For access to the select module to poll the socket for timeout /* Do a select() on the socket, if necessary (sock_timeout > 0).
functionality. writing is 1 for writing, 0 for reading. The argument writing indicates the direction.
Return value: -1 if error, 0 if not ready, >= 1 if ready. This does not raise an exception or return a success indicator;
An exception is set when the return value is <= 0 (!). */ we'll let the actual socket call do that. */
static int static void
internal_select(PySocketSockObject *s, int writing) internal_select(PySocketSockObject *s, int writing)
{ {
fd_set fds; fd_set fds;
struct timeval tv; struct timeval tv;
int count;
if (s->sock_timeout <= 0.0)
return;
/* Construct the arguments to select */ /* Construct the arguments to select */
tv.tv_sec = (int)s->sock_timeout; tv.tv_sec = (int)s->sock_timeout;
@ -527,22 +509,9 @@ internal_select(PySocketSockObject *s, int writing)
/* See if the socket is ready */ /* See if the socket is ready */
if (writing) if (writing)
count = select(s->sock_fd+1, NULL, &fds, NULL, &tv); select(s->sock_fd+1, NULL, &fds, NULL, &tv);
else else
count = select(s->sock_fd+1, &fds, NULL, NULL, &tv); select(s->sock_fd+1, &fds, NULL, NULL, &tv);
/* Check for errors */
if (count < 0) {
s->errorhandler();
return -1;
}
/* Set the error if the timeout has elapsed, i.e, we were not
polled. */
if (count == 0)
timeout_err();
return count;
} }
/* Initialize a new socket object. */ /* Initialize a new socket object. */
@ -558,7 +527,6 @@ init_sockobject(PySocketSockObject *s,
s->sock_family = family; s->sock_family = family;
s->sock_type = type; s->sock_type = type;
s->sock_proto = proto; s->sock_proto = proto;
s->sock_blocking = 1; /* Start in blocking mode */
s->sock_timeout = -1.0; /* Start without timeout */ s->sock_timeout = -1.0; /* Start without timeout */
s->errorhandler = &set_error; s->errorhandler = &set_error;
@ -997,45 +965,11 @@ sock_accept(PySocketSockObject *s)
return NULL; return NULL;
memset(addrbuf, 0, addrlen); memset(addrbuf, 0, addrlen);
errno = 0; /* Reset indicator for use with timeout behavior */
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
internal_select(s, 0);
newfd = accept(s->sock_fd, (struct sockaddr *) addrbuf, &addrlen); newfd = accept(s->sock_fd, (struct sockaddr *) addrbuf, &addrlen);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
if (s->sock_timeout >= 0.0) {
#ifdef MS_WINDOWS
if (newfd == INVALID_SOCKET)
if (!s->sock_blocking)
return s->errorhandler();
/* Check if we have a true failure
for a blocking socket */
if (errno != WSAEWOULDBLOCK)
return s->errorhandler();
#else
if (newfd < 0) {
if (!s->sock_blocking)
return s->errorhandler();
/* Check if we have a true failure
for a blocking socket */
if (errno != EAGAIN && errno != EWOULDBLOCK)
return s->errorhandler();
}
#endif
/* try waiting the timeout period */
if (internal_select(s, 0) <= 0)
return NULL;
Py_BEGIN_ALLOW_THREADS
newfd = accept(s->sock_fd,
(struct sockaddr *)addrbuf,
&addrlen);
Py_END_ALLOW_THREADS
}
/* At this point, we really have an error, whether using timeout
behavior or regular socket behavior */
#ifdef MS_WINDOWS #ifdef MS_WINDOWS
if (newfd == INVALID_SOCKET) if (newfd == INVALID_SOCKET)
#else #else
@ -1074,7 +1008,10 @@ Wait for an incoming connection. Return a new socket representing the\n\
connection, and the address of the client. For IP sockets, the address\n\ connection, and the address of the client. For IP sockets, the address\n\
info is a pair (hostaddr, port)."; info is a pair (hostaddr, port).";
/* s.setblocking(1 | 0) method */ /* s.setblocking(flag) method. Argument:
False -- non-blocking mode; same as settimeout(0)
True -- blocking mode; same as settimeout(None)
*/
static PyObject * static PyObject *
sock_setblocking(PySocketSockObject *s, PyObject *arg) sock_setblocking(PySocketSockObject *s, PyObject *arg)
@ -1085,8 +1022,7 @@ sock_setblocking(PySocketSockObject *s, PyObject *arg)
if (block == -1 && PyErr_Occurred()) if (block == -1 && PyErr_Occurred())
return NULL; return NULL;
s->sock_blocking = block; s->sock_timeout = block ? -1.0 : 0.0;
s->sock_timeout = -1.0; /* Always clear the timeout */
internal_setblocking(s, block); internal_setblocking(s, block);
Py_INCREF(Py_None); Py_INCREF(Py_None);
@ -1097,44 +1033,34 @@ static char setblocking_doc[] =
"setblocking(flag)\n\ "setblocking(flag)\n\
\n\ \n\
Set the socket to blocking (flag is true) or non-blocking (false).\n\ Set the socket to blocking (flag is true) or non-blocking (false).\n\
This uses the FIONBIO ioctl with the O_NDELAY flag."; setblocking(True) is equivalent to settimeout(None);\n\
setblocking(False) is equivalent to settimeout(0.0).";
/* s.settimeout(None | float) method. /* s.settimeout(timeout) method. Argument:
Causes an exception to be raised when the given time has None -- no timeout, blocking mode; same as setblocking(True)
elapsed when performing a blocking socket operation. */ 0.0 -- non-blocking mode; same as setblocking(False)
> 0 -- timeout mode; operations time out after timeout seconds
< 0 -- illegal; raises an exception
*/
static PyObject * static PyObject *
sock_settimeout(PySocketSockObject *s, PyObject *arg) sock_settimeout(PySocketSockObject *s, PyObject *arg)
{ {
double value; double timeout;
if (arg == Py_None) if (arg == Py_None)
value = -1.0; timeout = -1.0;
else { else {
value = PyFloat_AsDouble(arg); timeout = PyFloat_AsDouble(arg);
if (value < 0.0) { if (timeout < 0.0) {
if (!PyErr_Occurred()) if (!PyErr_Occurred())
PyErr_SetString(PyExc_ValueError, PyErr_SetString(PyExc_ValueError,
"Invalid timeout value"); "Timeout value out of range");
return NULL; return NULL;
} }
} }
s->sock_timeout = value; s->sock_timeout = timeout;
internal_setblocking(s, timeout < 0.0);
/* The semantics of setting socket timeouts are:
If you settimeout(!=None):
The actual socket gets put in non-blocking mode and the select
is used to control timeouts.
Else if you settimeout(None) [then value is -1.0]:
The old behavior is used AND automatically, the socket is set
to blocking mode. That means that someone who was doing
non-blocking stuff before, sets a timeout, and then unsets
one, will have to call setblocking(0) again if he wants
non-blocking stuff. This makes sense because timeout stuff is
blocking by nature. */
internal_setblocking(s, value < 0.0);
s->sock_blocking = 1; /* Always negate setblocking() */
Py_INCREF(Py_None); Py_INCREF(Py_None);
return Py_None; return Py_None;
@ -1143,8 +1069,10 @@ sock_settimeout(PySocketSockObject *s, PyObject *arg)
static char settimeout_doc[] = static char settimeout_doc[] =
"settimeout(timeout)\n\ "settimeout(timeout)\n\
\n\ \n\
Set a timeout on blocking socket operations. 'timeout' can be a float,\n\ Set a timeout on socket operations. 'timeout' can be a float,\n\
giving seconds, or None. Setting a timeout of None disables timeout."; giving in seconds, or None. Setting a timeout of None disables\n\
the timeout feature and is equivalent to setblocking(1).\n\
Setting a timeout of zero is the same as setblocking(0).";
/* s.gettimeout() method. /* s.gettimeout() method.
Returns the timeout associated with a socket. */ Returns the timeout associated with a socket. */
@ -1355,50 +1283,20 @@ sock_connect(PySocketSockObject *s, PyObject *addro)
if (!getsockaddrarg(s, addro, &addr, &addrlen)) if (!getsockaddrarg(s, addro, &addr, &addrlen))
return NULL; return NULL;
errno = 0; /* Reset the err indicator for use with timeouts */
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
res = connect(s->sock_fd, addr, addrlen); if (s->sock_timeout > 0.0) {
Py_END_ALLOW_THREADS
if (s->sock_timeout >= 0.0) {
if (res < 0) {
/* Return if we're already connected */
#ifdef MS_WINDOWS
if (errno == WSAEINVAL || errno == WSAEISCONN)
#else
if (errno == EISCONN)
#endif
goto connected;
/* Check if we have an error */
if (!s->sock_blocking)
return s->errorhandler();
/* Check if we have a true failure
for a blocking socket */
#ifdef MS_WINDOWS
if (errno != WSAEWOULDBLOCK)
#else
if (errno != EINPROGRESS && errno != EALREADY &&
errno != EWOULDBLOCK)
#endif
return s->errorhandler();
}
/* Check if we're ready for the connect via select */
if (internal_select(s, 1) <= 0)
return NULL;
/* Complete the connection now */
Py_BEGIN_ALLOW_THREADS
res = connect(s->sock_fd, addr, addrlen); res = connect(s->sock_fd, addr, addrlen);
Py_END_ALLOW_THREADS if (res == EINPROGRESS) {
internal_select(s, 1);
res = connect(s->sock_fd, addr, addrlen);
}
} }
else
res = connect(s->sock_fd, addr, addrlen);
Py_END_ALLOW_THREADS
if (res < 0) if (res < 0)
return s->errorhandler(); return s->errorhandler();
connected:
Py_INCREF(Py_None); Py_INCREF(Py_None);
return Py_None; return Py_None;
} }
@ -1422,47 +1320,18 @@ sock_connect_ex(PySocketSockObject *s, PyObject *addro)
if (!getsockaddrarg(s, addro, &addr, &addrlen)) if (!getsockaddrarg(s, addro, &addr, &addrlen))
return NULL; return NULL;
errno = 0; /* Reset the err indicator for use with timeouts */
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
res = connect(s->sock_fd, addr, addrlen); if (s->sock_timeout > 0.0) {
res = connect(s->sock_fd, addr, addrlen);
if (res == EINPROGRESS) {
internal_select(s, 1);
res = connect(s->sock_fd, addr, addrlen);
}
}
else
res = connect(s->sock_fd, addr, addrlen);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
if (s->sock_timeout >= 0.0) {
if (res < 0) {
/* Return if we're already connected */
#ifdef MS_WINDOWS
if (errno == WSAEINVAL || errno == WSAEISCONN)
#else
if (errno == EISCONN)
#endif
goto conex_finally;
/* Check if we have an error */
if (!s->sock_blocking)
goto conex_finally;
/* Check if we have a true failure
for a blocking socket */
#ifdef MS_WINDOWS
if (errno != WSAEWOULDBLOCK)
#else
if (errno != EINPROGRESS && errno != EALREADY &&
errno != EWOULDBLOCK)
#endif
goto conex_finally;
}
/* Check if we're ready for the connect via select */
if (internal_select(s, 1) <= 0)
return NULL;
/* Complete the connection now */
Py_BEGIN_ALLOW_THREADS
res = connect(s->sock_fd, addr, addrlen);
Py_END_ALLOW_THREADS
}
conex_finally:
if (res != 0) { if (res != 0) {
#ifdef MS_WINDOWS #ifdef MS_WINDOWS
res = WSAGetLastError(); res = WSAGetLastError();
@ -1683,7 +1552,7 @@ sock_recv(PySocketSockObject *s, PyObject *args)
if (len < 0) { if (len < 0) {
PyErr_SetString(PyExc_ValueError, PyErr_SetString(PyExc_ValueError,
"negative buffersize in connect"); "negative buffersize in recv");
return NULL; return NULL;
} }
@ -1691,14 +1560,8 @@ sock_recv(PySocketSockObject *s, PyObject *args)
if (buf == NULL) if (buf == NULL)
return NULL; return NULL;
if (s->sock_timeout >= 0.0) {
if (s->sock_blocking) {
if (internal_select(s, 0) <= 0)
return NULL;
}
}
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
internal_select(s, 0);
n = recv(s->sock_fd, PyString_AS_STRING(buf), len, flags); n = recv(s->sock_fd, PyString_AS_STRING(buf), len, flags);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
@ -1741,15 +1604,9 @@ sock_recvfrom(PySocketSockObject *s, PyObject *args)
if (buf == NULL) if (buf == NULL)
return NULL; return NULL;
if (s->sock_timeout >= 0.0) {
if (s->sock_blocking) {
if (internal_select(s, 0) <= 0)
return NULL;
}
}
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
memset(addrbuf, 0, addrlen); memset(addrbuf, 0, addrlen);
internal_select(s, 0);
n = recvfrom(s->sock_fd, PyString_AS_STRING(buf), len, flags, n = recvfrom(s->sock_fd, PyString_AS_STRING(buf), len, flags,
#ifndef MS_WINDOWS #ifndef MS_WINDOWS
#if defined(PYOS_OS2) && !defined(PYCC_GCC) #if defined(PYOS_OS2) && !defined(PYCC_GCC)
@ -1799,14 +1656,8 @@ sock_send(PySocketSockObject *s, PyObject *args)
if (!PyArg_ParseTuple(args, "s#|i:send", &buf, &len, &flags)) if (!PyArg_ParseTuple(args, "s#|i:send", &buf, &len, &flags))
return NULL; return NULL;
if (s->sock_timeout >= 0.0) {
if (s->sock_blocking) {
if (internal_select(s, 1) <= 0)
return NULL;
}
}
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
internal_select(s, 1);
n = send(s->sock_fd, buf, len, flags); n = send(s->sock_fd, buf, len, flags);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
@ -1834,14 +1685,8 @@ sock_sendall(PySocketSockObject *s, PyObject *args)
if (!PyArg_ParseTuple(args, "s#|i:sendall", &buf, &len, &flags)) if (!PyArg_ParseTuple(args, "s#|i:sendall", &buf, &len, &flags))
return NULL; return NULL;
if (s->sock_timeout >= 0.0) {
if (s->sock_blocking) {
if (internal_select(s, 1) <= 0)
return NULL;
}
}
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
internal_select(s, 1);
do { do {
n = send(s->sock_fd, buf, len, flags); n = send(s->sock_fd, buf, len, flags);
if (n < 0) if (n < 0)
@ -1888,14 +1733,8 @@ sock_sendto(PySocketSockObject *s, PyObject *args)
if (!getsockaddrarg(s, addro, &addr, &addrlen)) if (!getsockaddrarg(s, addro, &addr, &addrlen))
return NULL; return NULL;
if (s->sock_timeout >= 0.0) {
if (s->sock_blocking) {
if (internal_select(s, 1) <= 0)
return NULL;
}
}
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
internal_select(s, 1);
n = sendto(s->sock_fd, buf, len, flags, addr, addrlen); n = sendto(s->sock_fd, buf, len, flags, addr, addrlen);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS

View file

@ -83,9 +83,8 @@ typedef struct {
PyObject *(*errorhandler)(void); /* Error handler; checks PyObject *(*errorhandler)(void); /* Error handler; checks
errno, returns NULL and errno, returns NULL and
sets a Python exception */ sets a Python exception */
int sock_blocking; /* Flag indicated whether the double sock_timeout; /* Operation timeout in seconds;
socket is in blocking mode */ 0.0 means non-blocking */
double sock_timeout; /* Operation timeout value */
} PySocketSockObject; } PySocketSockObject;
/* --- C API ----------------------------------------------------*/ /* --- C API ----------------------------------------------------*/