If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:
- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
YIELD_FROM opcode
- ...etc.
However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.
The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.
This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all.
test_is_alive_after_fork() now joins directly the thread to avoid the
following warning added by bpo-30357:
Warning -- threading_cleanup() failed to cleanup 0 threads
after 2 sec (count: 0, dangling: 21)
Use also a different exit code to catch generic exit code 1.
Warnings emitted when compile a regular expression now always point
to the line in the user code. Previously they could point into inners
of the re module if emitted from inside of groups or conditionals.
test_thread: setUp() now uses support.threading_setup() and
support.threading_cleanup() to wait until threads complete to avoid
random side effects on following tests.
Co-Authored-By: Grzegorz Grzywacz <grzegorz.grzywacz@nazwa.pl>
* bpo-30308: Code coverage for argument in random.shuffle
* bpo-30308: Code coverage for argument in random.shuffle
* bpo-30308: Code coverage for argument in random.shuffle
when there are no more `await` or `yield (from)` before return in coroutine,
cancel was ignored.
example:
async def coro():
asyncio.Task.current_task().cancel()
return 42
...
res = await coro() # should raise CancelledError
Now allowed several subsequential inline modifiers at the start of the
pattern (e.g. '(?i)(?s)...'). In verbose mode whitespaces and comments
now are allowed before and between inline modifiers (e.g.
'(?x) (?i) (?s)...').
Rewrite sigwaitinfo() and sigtimedwait() unit tests for EINTR using
pthread_sigmask() to fix a race condition between the child and the
parent process.
Remove the pipe which was used as a weak workaround against the race
condition.
sigtimedwait() is now tested with a child process sending a signal
instead of testing the timeout feature which is more unstable
(especially regarding to clock resolution depending on the platform).
ExpatParser.parse() of xml.sax.xmlreader now always closes the
source: close the file object or the urllib object if source is a
string (not an open file-like object). The change fixes a
ResourceWarning on parsing error.
Add test_parse_close_source() unit test.
Compiled regular expression objects with the re.LOCALE flag no longer
depend on the locale at compile time. Only the locale at matching
time affects the result of matching.
AsyncoreEchoServer of test_ssl now calls
asyncore.close_all(ignore_all=True) to ensure that
asyncore.socket_map is cleared once the test completes, even if
ConnectionHandler was not correctly unregistered.
Fix the following warning:
Warning -- asyncore.socket_map was modified by test_ssl
Before: {}
After: {6: <test.test_ssl.AsyncoreEchoServer.EchoServer.ConnectionHandler>}
* bpo-30197: Enhance functions swap_attr() and swap_item() in test.support.
They now work when delete replaced attribute or item inside the with
statement. The old value of the attribute or item (or None if it doesn't
exist) now will be assigned to the target of the "as" clause, if there is
one.
* Update docstrings.
* bpo-30175: Skip client cert tests of test_imaplib
The IMAP server cyrus.andrew.cmu.edu doesn't accept our randomly
generated client x509 certificate anymore.
* bpo-30188: Catch EOFError in NetworkedNNTPTests
test_nntplib fails randomly with EOFError in
NetworkedNNTPTests.setUpClass(). Catch EOFError to skip tests in that
case.
QueueListenerTest of test_logging now closes the multiprocessing
Queue and joins its thread to prevent leaking dangling threads to
following tests.
Add also @support.reap_threads to detect earlier if a test leaks
threads (and try to "cleanup" these threads).
On macOS, SuppressCrashReport now redirects /usr/bin/defaults command
stderr into a pipe to not pollute stderr. It fixes a
test_io.test_daemon_threads_shutdown_stderr_deadlock() failure when
the CrashReporter domain doesn't exists. Message logged into stderr:
2017-04-24 16:57:21.432 defaults[41046:2462851]
The domain/default pair of (com.apple.CrashReporter, DialogType) does not exist
At the time when an abstract base class' __init_subclass__ runs,
ABCMeta.__new__ has not yet finished running, so in the presence of
__init_subclass__, inspect.isabstract() can no longer depend only on
TPFLAGS_IS_ABSTRACT.
Disable faulthandler to run test_SEH() of test_ctypes to prevent the
following log with a traceback:
Windows fatal exception: access violation
Add support.disable_faulthandler() context manager.
* bpo-30125: Cleanup faulthandler.c
* Use size_t type for iterators
* Add { ... }
* bpo-30125: Fix faulthandler.disable() on Windows
On Windows, faulthandler.disable() now removes the exception handler
installed by faulthandler.enable().
test_io has two unit tests which trigger a deadlock:
* test_daemon_threads_shutdown_stdout_deadlock()
* test_daemon_threads_shutdown_stderr_deadlock()
These tests call Py_FatalError() if the expected bug is triggered
which calls abort(). Use test.support.SuppressCrashReport to prevent
the creation on a core dump, to fix the warning:
Warning -- files was modified by test_io
Before: []
After: ['python.core']
test_quick_connect() runs a thread up to 50 seconds, whereas the
socket is connected in 0.2 second and then the thread is expected to
end in less than 3 second. On Linux, the thread ends quickly because
select() seems to always return quickly. On FreeBSD, sometimes
select() fails with timeout and so the thread runs much longer than
expected.
Fix the thread timeout to fix a race condition in the test.
