Commit graph

5795 commits

Author SHA1 Message Date
Alexey Izbyshev
c1e46e94de bpo-32777: Fix _Py_set_inheritable async-safety in subprocess (GH-5560)
Fix a rare but potential pre-exec child process deadlock in subprocess on POSIX systems when marking file descriptors inheritable on exec in the child process.  This bug appears to have been introduced in 3.4 with the inheritable file descriptors support.

This also changes Python/fileutils.c `set_inheritable` to use the "slow" two `fcntl` syscall path instead of the "fast" single `ioctl` syscall path when asked to be async signal safe (by way of being asked not to raise exceptions).  `ioctl` is not a POSIX async-signal-safe approved function.

ref: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html
2018-02-05 22:09:34 -08:00
Serhiy Storchaka
07ca9afaa8
bpo-10544: Disallow "yield" in comprehensions and generator expressions. (GH-4564) 2018-02-04 10:53:48 +02:00
Leo Arias
c3d9508ff2 bpo-32746: Fix multiple typos (GH-5144)
Fix typos found by codespell in docs, docstrings, and comments.
2018-02-03 19:36:10 -05:00
Eric Snow
4e9da0d163
bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (#5507) 2018-02-02 21:49:49 -07:00
Barry Warsaw
bbbcf8693b
bpo-32303 - Consistency fixes for namespace loaders (#5481)
* Make sure ``__spec__.loader`` matches ``__loader__`` for namespace packages.
* Make sure ``__spec__.origin` matches ``__file__`` for namespace packages.

https://bugs.python.org/issue32303
https://bugs.python.org/issue32305
2018-02-02 15:15:58 -05:00
oldk
aa0735f597 bpo-32747: Remove trailing spaces in docstrings. (GH-5491) 2018-02-02 10:52:55 +02:00
oldk
461d225b19 bpo-32674: Improve the docstring for __import__ (GH-5339)
Clarify that the level argument is used to determine whether to
perform absolute or relative imports: 0 is absolute, while a positive number
is the number of parent directories to search relative to the current module.
2018-02-01 23:20:00 -05:00
Yury Selivanov
55e0839f26
bpo-32436: Fix compiler warning (#5483) 2018-02-01 22:24:56 -05:00
Dmitry Alimov
01a0cb8916 Fix typo in hamt.c comments (#5478) 2018-02-01 21:59:48 -05:00
Stéphane Wirtel
83ab995871 bpo-32711: Fix warnings for Python/ast_unparse.c (#5426)
* bpo-32711: Fix warnings for Python/ast_unparse.c
2018-02-01 17:59:27 +01:00
Eric Snow
7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00
Mark Shannon
332cd5ee4f bpo-32550. Remove the STORE_ANNOTATION bytecode. (GH-5181) 2018-01-29 16:41:04 -08:00
Benjamin Peterson
c65ef772c3
rename _imp initialization function to follow conventions (#5432)
When the C imp module became _imp in 6f44d66bc4, the initialization function should have been renamed from PyInit_imp to PyInit__imp.
2018-01-29 11:33:57 -08:00
Yury Selivanov
b647d7039d
bpo-32707: Fix warnings in hamt.c (#5430) 2018-01-29 13:31:37 -05:00
Victor Stinner
2914bb32e2
bpo-20891: Py_Initialize() now creates the GIL (#4700)
The GIL is no longer created "on demand" to fix a race condition when
PyGILState_Ensure() is called in a non-Python thread.
2018-01-29 11:57:45 +01:00
Xiang Zhang
997478eb79
bpo-32688: Make why the line is commented out clear (GH-5405) 2018-01-29 11:32:12 +08:00
Yury Selivanov
bc4123b0b3
bpo-32436: Use PyThreadState_GET() in all hot paths (GH-5363) 2018-01-27 13:24:20 -05:00
Yury Selivanov
226e50049d
bpo-32436: Make PyContextVar_Get a little bit faster (#5350)
Since context.c is compiled with Py_BUILD_CORE, using a macro
will result in a slightly more optimal code.
2018-01-26 17:24:52 -05:00
Yury Selivanov
43c47fe096
bpo-32670: Enforce PEP 479. (#5327) 2018-01-26 15:24:24 -05:00
Nathaniel J. Smith
dba976b8a2 bpo-32591: fix abort in _PyErr_WarnUnawaitedCoroutine during shutdown (#5337)
When an unawaited coroutine is collected very late in shutdown --
like, during the final GC at the end of PyImport_Cleanup -- then it
was triggering an interpreter abort, because we'd try to look up the
"warnings" module and not only was it missing (we were prepared for
that), but the entire module system was missing (which we were not
prepared for).

I've tried to fix this at the source, by making the utility function
get_warnings_attr robust against this in general. Note that it already
has the convention that it can return NULL without setting an error,
which is how it signals that the attribute it was asked to fetch is
missing, and that all callers already check for NULL returns.

There's a similar check for being late in shutdown at the top of
warn_explicit, which might be unnecessary after this fix, but I'm not
sure so I'm going to leave it.
2018-01-26 14:28:31 -05:00
Guido van Rossum
95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
Yury Selivanov
6ab62920c8
bpo-32436: Fix a refleak; var GC tracking; a GCC warning (#5326)
The refleak in question wasn't really important, as context vars
are usually created at the toplevel and live as long as the interpreter
lives, so the context var name isn't ever GCed anyways.
2018-01-25 14:18:55 -05:00
Dmitry Alimov
93a6119f08 Fix 3rd level node's binary representation in HAMT algorithm description (#5319) 2018-01-25 12:54:41 -05:00
Serhiy Storchaka
f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Yury Selivanov
b7a80d543e
bpo-32436: Don't use native popcount() (also fixes bpo-32641) (#5292) 2018-01-23 22:17:04 -05:00
Yury Selivanov
0bad4d63c6
bpo-32436: Fix potential NULL dereference (#5286) 2018-01-23 16:26:07 -05:00
Yury Selivanov
83c8675edb
bpo-32436: Remove a redundant assert (#5275) 2018-01-23 02:00:03 -05:00
Yury Selivanov
f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Victor Stinner
9089a26591
bpo-29240: PyUnicode_DecodeLocale() uses UTF-8 on Android (#5272)
PyUnicode_DecodeLocaleAndSize(), PyUnicode_DecodeLocale() and
PyUnicode_EncodeLocale() now use always use the UTF-8 encoding on
Android, instead of the current locale encoding.

On Android API 19, mbstowcs() and wcstombs() are broken and cannot be
used.
2018-01-22 19:07:32 +01:00
Victor Stinner
13ff24582c
bpo-32593: Drop FreeBSD 9 and older support (#5232)
Drop support of FreeBSD 9 and older.
2018-01-22 18:32:50 +01:00
Yury Selivanov
3510334361
bpo-32591: Fix PyExc_WarnFormat call (follow-up commit) (#5263)
The previous version was correct in terms of behaviour, but
checking the return value of PyErr_WarnFormat allows to
avoid calling PyErr_Occurred and silences the coverity alarm.
2018-01-21 20:47:04 -05:00
Nathaniel J. Smith
fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00
Victor Stinner
05d68a8bd8
bpo-9566: Fix size_t=>int downcast warnings (#5230)
* Use wider types (int => Py_ssize_t) to avoid integer overflows.
* Fix gc.get_freeze_count(): use Py_ssize_t type rather than int, since gc_list_size() returns a Py_ssize_t.
2018-01-18 11:15:25 +01:00
hui shang
6027802ca7 Fix typo in thread_nt.h code comment (GH-5211)
The comment for PyThread_allocate_lock says "It has too be implemented ...".
There was an extra "o" in ".. to be implemented.."
2018-01-17 15:21:01 -08:00
INADA Naoki
378edee0a3
bpo-32544: Speed up hasattr() and getattr() (GH-5173)
AttributeError was raised always when attribute is not found.
This commit skip raising AttributeError when `tp_getattro` is `PyObject_GenericGetAttr`.
It makes hasattr() and getattr() about 4x faster when attribute is not found.
2018-01-16 20:52:41 +09:00
Barry Warsaw
5ec0feeeec
Implement the get_resource_reader() API for file system imports (#5168) 2018-01-15 15:07:11 -08:00
Victor Stinner
cb064fc232
bpo-31900: Fix localeconv() encoding for LC_NUMERIC (#4174)
* Add _Py_GetLocaleconvNumeric() function: decode decimal_point and
  thousands_sep fields of localeconv() from the LC_NUMERIC encoding,
  rather than decoding from the LC_CTYPE encoding.
* Modify locale.localeconv() and "n" formatter of str.format() (for
  int, float and complex to use _Py_GetLocaleconvNumeric()
  internally.
2018-01-15 15:58:02 +01:00
Victor Stinner
7ed7aead95
bpo-29240: Fix locale encodings in UTF-8 Mode (#5170)
Modify locale.localeconv(), time.tzname, os.strerror() and other
functions to ignore the UTF-8 Mode: always use the current locale
encoding.

Changes:

* Add _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx(). On decoding or
  encoding error, they return the position of the error and an error
  message which are used to raise Unicode errors in
  PyUnicode_DecodeLocale() and PyUnicode_EncodeLocale().
* Replace _Py_DecodeCurrentLocale() with _Py_DecodeLocaleEx().
* PyUnicode_DecodeLocale() now uses _Py_DecodeLocaleEx() for all
  cases, especially for the strict error handler.
* Add _Py_DecodeUTF8Ex(): return more information on decoding error
  and supports the strict error handler.
* Rename _Py_EncodeUTF8_surrogateescape() to _Py_EncodeUTF8Ex().
* Replace _Py_EncodeCurrentLocale() with _Py_EncodeLocaleEx().
* Ignore the UTF-8 mode to encode/decode localeconv(), strerror()
  and time zone name.
* Remove PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize()
  and PyUnicode_EncodeLocale() now ignore the UTF-8 mode: always use
  the "current" locale.
* Remove _PyUnicode_DecodeCurrentLocale(),
  _PyUnicode_DecodeCurrentLocaleAndSize() and
  _PyUnicode_EncodeCurrentLocale().
2018-01-15 10:45:49 +01:00
Serhiy Storchaka
782d6fe443
bpo-31113: Get rid of recursion in the compiler for normal control flow. (#3015) 2018-01-11 20:20:13 +02:00
Victor Stinner
2cba6b8579
bpo-29240: readline now ignores the UTF-8 Mode (#5145)
Add new fuctions ignoring the UTF-8 mode:

* _Py_DecodeCurrentLocale()
* _Py_EncodeCurrentLocale()
* _PyUnicode_DecodeCurrentLocaleAndSize()
* _PyUnicode_EncodeCurrentLocale()

Modify the readline module to use these functions.

Re-enable test_readline.test_nonascii().
2018-01-10 22:46:15 +01:00
Serhiy Storchaka
d4864c61e3
bpo-24340: Fix estimation of the code stack size. (#5076) 2018-01-09 21:54:52 +02:00
Nick Coghlan
9b99747386
bpo-31975 (PEP 565): Show DeprecationWarning in __main__ (GH-4458)
- primary change is to add a new default filter entry for
  'default::DeprecationWarning:__main__'
- secondary change is an internal one to cope with plain
  strings in the warning module's internal filter list
  (this avoids the need to create a compiled regex object
  early on during interpreter startup)
- assorted documentation updates, including many more
  examples of configuring the warnings settings
- additional tests to ensure that both the pure Python and
  the C accelerated warnings modules have the expected
  default configuration
2018-01-08 12:45:02 +10:00
Nathaniel J. Smith
e46a8af450 bpo-30579: Allow TracebackType creation and tb_next mutation from Python (GH-4793)
Third party projects may wish to hide their own internal machinery in
order to present more comprehensible tracebacks to end users
(e.g. Jinja2 and Trio both do this).

Previously such projects have had to rely on ctypes to do so:

  fe3dadacdf/jinja2/debug.py (L345)
  1e86b1aee8/trio/_core/_multierror.py (L296)

This provides a Python level API for creating and modifying real
Traceback objects, allowing tracebacks to be edited at runtime.

Patch by Nathaniel Smith.
2018-01-07 23:30:18 +10:00
Nathaniel J. Smith
735ae8d139 bpo-29137: Remove fpectl module (#4789)
This module has never been enabled by default, never worked correctly
on x86-64, and caused ABI problems that caused C extension
compatibility. See bpo-29137 for details/discussion.
2018-01-05 23:15:34 -08:00
Benjamin Peterson
65f2a6dcc2
advance copyright years to 2018 (#5094) 2018-01-04 22:34:19 -08:00
Benjamin Peterson
0a37a30037
closes bpo-32460: ensure all non-static globals have initializers (#5061) 2017-12-31 10:04:13 -08:00
Serhiy Storchaka
02b9ef2775
bpo-32439: Clean up the code for compiling comparison expressions. (#5029) 2017-12-30 09:47:42 +02:00
Serhiy Storchaka
13a6c098c2
bpo-32259: Make a TypeError message when unpack non-iterable more specific. (#4903) 2017-12-26 12:30:41 +02:00
Serhiy Storchaka
3dfbaf51f0
bpo-32372: Move __debug__ optimization to the AST level. (#4925) 2017-12-25 12:47:50 +02:00
Benjamin Peterson
62ed6be8da
remove the dynload_next.c file (closes bpo-32386) (#4957) 2017-12-21 21:43:09 -08:00