Commit graph

292 commits

Author SHA1 Message Date
Victor Stinner
fb578f0a05 Merge 3.4 (asyncio) 2015-07-09 23:14:50 +02:00
Victor Stinner
e6ecea53c8 asyncio: sync with github asyncio
* queues: get coroutine from asyncio.coroutines, not from asyncio.tasks
* tets: replace tulip with asyncio in comments
2015-07-09 23:13:50 +02:00
Yury Selivanov
00e3372358 Issue #24325, #24400: Add more unittests for types.coroutine; tweak wrapper implementation. 2015-06-24 11:44:51 -04:00
Yury Selivanov
fcba97242b Issue #24495, #24400: Test asyncio.Task.repr in debug mode 2015-06-24 10:55:12 -04:00
Yury Selivanov
5ac716251f Fix asyncio unittests in debug mode 2015-06-24 10:47:44 -04:00
Yury Selivanov
8f1c99321b Issue #24400: Fix CoroWrapper for 'async def' coroutines 2015-06-24 10:32:22 -04:00
Yury Selivanov
29a602a140 Issue #24400: Fix CoroWrapper for 'async def' coroutines 2015-06-24 10:30:14 -04:00
Yury Selivanov
e42cc38639 Issue 24017: Add a test for CoroWrapper and 'async def' coroutines 2015-05-31 21:44:05 -04:00
Yury Selivanov
439c5fe3ae Issue 24004: Fix DeprecationWarning in a unittest 2015-05-31 11:28:35 -04:00
Yury Selivanov
ad583a8e6d Issue 24004: Add a unittest for @asyncio.coroutine supporting Awaitables 2015-05-30 21:04:37 -04:00
Yury Selivanov
56fc614025 Issue 24315: Make collections.abc.Coroutine derived from Awaitable 2015-05-29 09:01:29 -04:00
Yury Selivanov
c41632a856 asyncio/tests: Drop unused import. 2015-05-13 15:37:48 -04:00
Yury Selivanov
6c792bd65a asyncio: Add a test for asyncio.iscoroutine().
Test that asyncio.iscoroutine() supports 'async def' coroutines and
collections.abc.Coroutine types.
2015-05-13 15:34:12 -04:00
Yury Selivanov
33c6b569b7 Issue 24179: Support 'async for' for asyncio.StreamReader. 2015-05-13 14:23:29 -04:00
Yury Selivanov
29f88c22e6 Issue 24178: support 'async with' for asyncio locks. 2015-05-13 14:10:38 -04:00
Yury Selivanov
1af2bf75a2 asyncio: Support PEP 492. Issue #24017. 2015-05-11 22:27:25 -04:00
Yury Selivanov
59eb9a4da5 asyncio: async() function is deprecated in favour of ensure_future(). 2015-05-11 14:48:38 -04:00
Yury Selivanov
740169cd24 Sync asyncio changes from the main repo. 2015-05-11 14:23:38 -04:00
Guido van Rossum
0a9933ebf3 Asyncio issue 222 / PR 231 (Victor Stinner) -- fix @coroutine functions without __name__. 2015-05-02 18:38:24 -07:00
Guido van Rossum
0bd16bc4cd Fix asyncio issue 235: Queue subclass bug caused by JoinableQueue merge. 2015-04-20 09:24:24 -07:00
Victor Stinner
79fd962652 asyncio: Fix _SelectorTransport.__repr__() if the event loop is closed 2015-03-27 15:20:08 +01:00
Victor Stinner
e7a2f64435 asyncio: Fix warning in test_close_kill_running()
Read process exit status to avoid the "Caught subprocess termination from
unknown pid" message.
2015-02-17 23:36:02 +01:00
Victor Stinner
4088ad9dce Issue #23475, asyncio: Fix test_close_kill_running()
Really kill the child process, don't mock completly the Popen.kill() method.

This change fix memory leaks and reference leaks.
2015-02-17 22:54:11 +01:00
Victor Stinner
4cb814c7e1 asyncio, Tulip issue 220: Merge JoinableQueue with Queue.
Merge JoinableQueue with Queue. To more closely match the standard Queue,
asyncio.Queue has "join" and "task_done". JoinableQueue is deleted.

Docstring for Queue.join shouldn't mention threads.

Restore JoinableQueue as a deprecated alias for Queue. To more closely match
the standard Queue, asyncio.Queue has "join" and "task_done".  JoinableQueue
remains as a deprecated alias for Queue to avoid needlessly breaking too much
code that depended on it.

Patch written by A. Jesse Jiryu Davis <jesse@mongodb.com>.
2015-02-17 22:53:28 +01:00
Serhiy Storchaka
85c3033670 Use os.devnull instead of hardcoded '/dev/null'. 2015-02-15 13:58:23 +02:00
Victor Stinner
8e36812e27 asyncio: BaseSubprocessTransport.close() doesn't try to kill the process if it
already finished
2015-02-10 14:49:32 +01:00
Victor Stinner
2fc2313038 asyncio: Only call _check_resolved_address() in debug mode
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
  it is much faster

Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":

* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop

On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
2015-02-04 14:51:23 +01:00
Victor Stinner
f2e43cbbd4 Issue #23347, asyncio: send_signal(), terminate(), kill() don't check if the
transport was closed. The check broken a Tulip example and this limitation is
arbitrary. Check if _proc is None should be enough.

Enhance also close(): do nothing when called the second time.
2015-01-30 01:20:44 +01:00
Victor Stinner
0698638d79 asyncio: Fix ResourceWarning in test_subprocess.test_proc_exit() 2015-01-30 00:11:42 +01:00
Victor Stinner
47cd10d7a9 asyncio: sync with Tulip
Issue #23347: send_signal(), kill() and terminate() methods of
BaseSubprocessTransport now check if the transport was closed and if the
process exited.

Issue #23347: Refactor creation of subprocess transports. Changes on
BaseSubprocessTransport:

* Add a wait() method to wait until the child process exit
* The constructor now accepts an optional waiter parameter. The _post_init()
  coroutine must not be called explicitly anymore. It makes subprocess
  transports closer to other transports, and it gives more freedom if we want
  later to change completly how subprocess transports are created.
* close() now kills the process instead of kindly terminate it: the child
  process may ignore SIGTERM and continue to run. Call explicitly terminate()
  and wait() if you want to kindly terminate the child process.
* close() now logs a warning in debug mode if the process is still running and
  needs to be killed
* _make_subprocess_transport() is now fully asynchronous again: if the creation
  of the transport failed, wait asynchronously for the process eixt. Before the
  wait was synchronous. This change requires close() to *kill*, and not
  terminate, the child process.
* Remove the _kill_wait() method, replaced with a more agressive close()
  method. It fixes _make_subprocess_transport() on error.
  BaseSubprocessTransport.close() calls the close() method of pipe transports,
  whereas _kill_wait() closed directly pipes of the subprocess.Popen object
  without unregistering file descriptors from the selector (which caused severe
  bugs).

These changes simplifies the code of subprocess.py.
2015-01-30 00:05:19 +01:00
Victor Stinner
978a9afc6a Issue #23243, asyncio: Emit a ResourceWarning when an event loop or a transport
is not explicitly closed. Close also explicitly transports in test_sslproto.
2015-01-29 17:50:58 +01:00
Victor Stinner
2934262fd3 asyncio: sync with Tulip
* Cleanup gather(): use cancelled() method instead of using private Future
  attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
  when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
  transport on error. In debug mode, log errors using call_exception_handler()
2015-01-29 14:15:19 +01:00
Victor Stinner
47bbea7124 asyncio: sync with Tulip
* _SelectorTransport constructor: extra parameter is now optional
* Fix _SelectorDatagramTransport constructor. Only start reading after
  connection_made() has been called.
* Fix _SelectorSslTransport.close(). Don't call protocol.connection_lost() if
  protocol.connection_made() was not called yet: if the SSL handshake failed or
  is still in progress. The close() method can be called if the creation of the
  connection is cancelled, by a timeout for example.
2015-01-29 02:56:05 +01:00
Victor Stinner
fa73779b0a asyncio: Fix _SelectorSocketTransport constructor
Only start reading when connection_made() has been called:
protocol.data_received() must not be called before protocol.connection_made().
2015-01-29 00:36:51 +01:00
Victor Stinner
b507cbaac5 asyncio: Fix SSLProtocol.eof_received()
Wake-up the waiter if it is not done yet.
2015-01-29 00:35:56 +01:00
Victor Stinner
ff018e406c asyncio: sync with Tulip
* Remove unused SSLProtocol._closing attribute
* test_sslproto: skip test if ssl module is missing
* Python issue #23208: Don't use the traceback of the current handle if we
  already know the traceback of the source. The handle may be more revelant,
  but having 3 tracebacks (handle, source, exception) becomes more difficult to
  read. The handle may be preferred later but it requires more work to make
  this choice.
2015-01-28 00:30:40 +01:00
Victor Stinner
e0fd157ba0 Issue #23293, asyncio: Rewrite IocpProactor.connect_pipe() as a coroutine
Use a coroutine with asyncio.sleep() instead of call_later() to ensure that the
schedule call is cancelled.

Add also a unit test cancelling connect_pipe().
2015-01-26 15:04:03 +01:00
Victor Stinner
442b0adccd asyncio: pyflakes, remove unused import
tests: Remove unused function; inline another function
2015-01-21 23:39:16 +01:00
Victor Stinner
cd0f7f9832 asyncio: Enhance BaseProactorEventLoop._loop_self_reading()
* Handle correctly CancelledError: just exit
* On error, log the exception and exit

Don't try to close the event loop, it is probably running and so it cannot be
closed.
2015-01-21 23:38:37 +01:00
Victor Stinner
f716d8b7a5 Issue #22685: Fix test_pause_reading() of asyncio/test_subprocess
Override the connect_read_pipe() method of the loop to mock immediatly
pause_reading() and resume_reading() methods.

The test failed randomly on FreeBSD 9 buildbot and on Windows using trollius.
2015-01-15 22:52:59 +01:00
Victor Stinner
922bc2ca12 Closes #23219: cancelling asyncio.wait_for() now cancels the task 2015-01-15 16:29:10 +01:00
Victor Stinner
ab8848bc2a asyncio: Close transports in tests
* Use test_utils.run_briefly() to execute pending calls to really close
  transports
* sslproto: mock also _SSLPipe.shutdown(), it's need to close the transport
* pipe test: the test doesn't close explicitly the PipeHandle, so ignore
  the warning instead
* test_popen: use the context manager ("with p:") to explicitly close pipes
2015-01-15 14:24:55 +01:00
Victor Stinner
02392c9282 Issue #23243: Close explicitly transports in asyncio tests 2015-01-15 13:18:32 +01:00
Victor Stinner
33cb0396e8 Issue #23243: Close explicitly event loops in asyncio tests 2015-01-15 13:17:34 +01:00
Victor Stinner
41ed958ee6 Issue #23243: Fix asyncio._UnixWritePipeTransport.close()
Do nothing if the transport is already closed. Before it was not possible to
close the transport twice.
2015-01-15 13:16:50 +01:00
Victor Stinner
29ad0111bd asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
  ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
  creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
  closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
  not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
  (BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
  readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
  close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
  state in the __repr__() method of _UnixReadPipeTransport and
  _UnixWritePipeTransport classes.
2015-01-15 00:04:21 +01:00
Victor Stinner
177e9f0855 Issue #23197, asyncio: On SSL handshake failure, check if the waiter is
cancelled before setting its exception.

* Add unit tests for this case.
* Cleanup also sslproto.py
2015-01-14 16:56:20 +01:00
Victor Stinner
f651a60407 Python issue #23173: sync with Tulip
* If an exception is raised during the creation of a subprocess, kill the
  subprocess (close pipes, kill and read the return status). Log an error in
  such case.
* Fix SubprocessStreamProtocol.connection_made() to handle cancelled waiter.
  Add unit test cancelling subprocess methods.
2015-01-14 02:10:33 +01:00
Victor Stinner
231b404cb0 Issue #22560: New SSL implementation based on ssl.MemoryBIO
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.

The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).

Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.

This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!

It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
2015-01-14 00:19:09 +01:00
Victor Stinner
9036e49ba1 Tulip issue 184: Fix test_pipe() on Windows
Pass explicitly the event loop to StreamReaderProtocol.
2015-01-13 16:13:06 +01:00