Commit graph

23510 commits

Author SHA1 Message Date
Victor Stinner
dedaac040f
bpo-40910: Export Py_GetArgcArgv() function (GH-20721) (GH-20723)
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.

* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
  _PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().

(cherry picked from commit e81f6e687d)
2020-06-08 18:44:50 +02:00
Miss Islington (bot)
8df4f3942f
bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser (GH-20697)
Automerge-Triggered-By: @pablogsal
(cherry picked from commit 9f495908c5)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-08 02:22:06 -07:00
Miss Islington (bot)
1e72fb2b78
bpo-40741: Update macOS installer to use SQLite 3.32.2. (GH-20705)
(cherry picked from commit 37eed5a9ee)

Co-authored-by: Ned Deily <nad@python.org>
2020-06-07 19:43:11 -07:00
Miss Islington (bot)
9cf1be46e3
bpo-39791 native hooks for importlib.resources.files (GH-20576)
* Provide native .files support on SourceFileLoader.

* Add native importlib.resources.files() support to zipimporter. Remove fallback support.

* make regen-all

* 📜🤖 Added by blurb_it.

* Move 'files' into the ResourceReader so it can carry the relevant module name context.

* Create 'importlib.readers' module and add FileReader to it.

* Add zip reader and rely on it for a TraversableResources object on zipimporter.

* Remove TraversableAdapter, no longer needed.

* Update blurb.

* Replace backslashes with forward slashes.

* Incorporate changes from importlib_metadata 2.0, finalizing the interface for extension via get_resource_reader.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 843c277656)

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2020-06-07 18:30:08 -07:00
Miss Islington (bot)
6440911736
bpo-40904: Fix segfault in the new parser with f-string containing yield statements with no value (GH-20701)
(cherry picked from commit 972ab03276)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-07 18:08:53 -07:00
Miss Islington (bot)
1e4fa91104
bpo-40724: Support setting buffer slots from type specs (GH-20648) (GH-20683)
This is not part of the limited API but makes the buffer slots available for type specs.
(cherry picked from commit f7c4e23642)
2020-06-07 09:06:40 +02:00
Miss Islington (bot)
90ee51f1cd
bpo-40870: Invalidate usage of some constants with ast.Name (GH-20649)
(cherry picked from commit 68874a8502)

Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2020-06-06 10:04:38 -07:00
Miss Islington (bot)
15fec5627a
bpo-40880: Fix invalid read in newline_in_string in pegen.c (GH-20666)
* bpo-40880: Fix invalid read in newline_in_string in pegen.c

* Update Parser/pegen/pegen.c

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>

* Add NEWS entry

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
(cherry picked from commit 2e6593db00)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-05 17:13:14 -07:00
Miss Islington (bot)
79e6c15aed
bpo-40883: Fix memory leak in fstring_compile_expr in parse_string.c (GH-20667)
(cherry picked from commit a54096e305)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-05 17:10:57 -07:00
Miss Islington (bot)
a4fa9a9515
bpo-39791: Refresh importlib.metadata from importlib_metadata 1.6.1. (GH-20659) (GH-20661)
* Refresh importlib.metadata from importlib_metadata 1.6.1.

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 161541ab45)

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2020-06-05 17:46:24 -04:00
Miss Islington (bot)
3dfe549bfa
bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486)
* bpo-40807: Show warnings once from codeop._maybe_compile

* Move catch_warnings

* news

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit 052d3fc090)

Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
2020-06-04 16:59:44 -07:00
Miss Islington (bot)
196810a98a
bpo-17258: Add requires_hashdigest to multiprocessing tests (GH-20412)
Skip some :mod:`multiprocessing` tests when MD5 hash digest is blocked.

Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit b022e5cffb)

Co-authored-by: Christian Heimes <christian@python.org>
2020-06-04 10:22:36 -07:00
Victor Stinner
6d62dc1ea4
[3.9] bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578) (GH-20618)
* bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)

Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.

(cherry picked from commit 3026cad59b)

* bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)

PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.

(cherry picked from commit cbe1296922)
2020-06-03 20:16:39 +02:00
Victor Stinner
5d2396c8cf
[3.9] bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20613)
* bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579)

Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception.

Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup
these functions.

(cherry picked from commit c353764fd5)

* bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)

my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().

my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.

test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().

Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.

(cherry picked from commit fa7ab6aa0f)
2020-06-03 17:49:25 +02:00
Miss Islington (bot)
74a1fceeb6
bpo-40767: Allow pure Wayland to get default XDG web browser (GH-20382)
Would be nice to backport to python 3.7+. I don't think it's worth the hassle to backport this all the way down to 3.10. But I'll let the maintainers decide.

This is hard to test because the test setup already includes this [environment variable](https://github.com/python/cpython/blob/master/Lib/test/pythoninfo.pyGH-L292)

Let me know if something doesn't match the PR guidelines. This is my first PR in the python source code.
(cherry picked from commit c822efeda9)

Co-authored-by: Jeremy Attali <jeremy.attali@gmail.com>
2020-06-03 06:04:29 -07:00
Huon Wilson
3c7609a23c
[3.9] bpo-40630: Add tracemalloc.reset_peak (GH-20102) (GH-20545)
* bpo-40630: Add tracemalloc.reset_peak (GH-20102, cherrypick 8b62644)

The reset_peak function sets the peak memory size to the current size,
representing a resetting of that metric. This allows for recording the
peak of specific sections of code, ignoring other code that may have
had a higher peak (since the most recent `tracemalloc.start()` or
tracemalloc.clear_traces()` call).

* Adjust docs to point to 3.9
2020-06-01 19:26:22 +02:00
Miss Islington (bot)
24a88b008b
bpo-30008: Fix OpenSSL no-deprecated compilation (GH-20397)
Fix :mod:`ssl`` code to be compatible with OpenSSL 1.1.x builds that use
``no-deprecated`` and ``--api=1.1.0``.

Note: Tests assume full OpenSSL API and fail with limited API.

Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Mark Wright <gienah@gentoo.org>
(cherry picked from commit a871f692b4)

Co-authored-by: Christian Heimes <christian@python.org>
2020-06-01 00:18:39 -07:00
Miss Islington (bot)
0a674638a3
bpo-17005: Move topological sort functionality to its own module (GH-20558)
The topological sort functionality that was introduced initially in the
functools module has been moved to a new graphlib module to
better accommodate the new tools and keep the original scope of the
functools module.
(cherry picked from commit 2f172d8f15)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-31 17:01:37 -07:00
Miss Islington (bot)
96d91071c0
bpo-40759: Deprecate the symbol module (GH-20364)
Automerge-Triggered-By: @pablogsal
(cherry picked from commit cf88871d6a)

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-05-31 15:23:29 -07:00
Miss Islington (bot)
9f3f70fd0b
bpo-39885: Make IDLE context menu cut and copy work again (GH-18951)
Leave selection when right click within.  This exception to clearing selections when right-clicking was omitted from the previous commit, 4ca060d.  I did not realize that this completely disabled the context menu entries, and  I should have merged a minimal fix immediately.  An automated test should follow.
(cherry picked from commit 97e4e0f53d)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-05-29 16:13:21 -07:00
Miss Islington (bot)
ad088ca5c6
bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (GH-20435) (GH-20514)
* Fix failure of _Py_dg_dtoa to remove trailing zeros

* Add regression test and news entry

* Add explanation about why it's safe to strip trailing zeros

* Make code safer, clean up comments, add change note at top of file

* Nitpick: avoid implicit int-to-float conversion in tests
(cherry picked from commit 895c9c1d43)

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
2020-05-29 14:46:54 +01:00
Miss Islington (bot)
a6ae02d7e9
bpo-39040: Fix parsing of email mime headers with whitespace between encoded-words. (gh-17620)
* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed904)

Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
2020-05-29 04:43:06 -07:00
Miss Islington (bot)
eceee544de
[3.9] bpo-40777: Initialize PyDateTime_IsoCalendarDateType.tp_base at run-time (GH-20493) (GH-20495)
Recent changes to _datetimemodule broke compilation on mingw; see the comments in this change for details.

FWIW, @corona10: this issue is why `PyType_FromModuleAndSpec` & friends take the `bases` argument at run time.
(cherry picked from commit 459acc5516)


Co-authored-by: Petr Viktorin <encukou@gmail.com>
2020-05-28 09:41:41 -07:00
Miss Islington (bot)
a5936ad632
bpo-1294959: Try to clarify the meaning of platlibdir (GH-20332)
Try to make the meaning of platlibdir clear.  The previous wording could
be misinterpreted to suggest that it will be used to find all shared
libraries on the system, and not just Python extensions.  Furthermore,
it was unclear whether it affects third-party (site-packages) extensions
or not.  The new wording tries to make its dual purpose clear,
and provide the additional example of extensions in site-packages.
(cherry picked from commit 242d95659b)

Co-authored-by: Michał Górny <mgorny@gentoo.org>
2020-05-28 09:41:31 -07:00
Miss Islington (bot)
bcbe5c59dd
bpo-40217: Ensure Py_VISIT(Py_TYPE(self)) is always called for PyType_FromSpec types (reverts GH-19414) (GH-20264)
Heap types now always visit the type in tp_traverse. See added docs for details.

This reverts commit 0169d3003b.

Automerge-Triggered-By: @encukou
(cherry picked from commit 1cf15af9a6)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-28 08:12:23 -07:00
Christian Heimes
8183e11d87
[3.9] bpo-40791: Use CRYPTO_memcmp() for compare_digest (GH-20456) (GH-20461)
hashlib.compare_digest uses OpenSSL's CRYPTO_memcmp() function
when OpenSSL is available.

Note: The _operator module is a builtin module. I don't want to add
libcrypto dependency to libpython. Therefore I duplicated the wrapper
function and added a copy to _hashopenssl.c..
(cherry picked from commit db5aed931f)

Co-authored-by: Christian Heimes <christian@python.org>
2020-05-28 05:09:38 -07:00
Miss Islington (bot)
9e3c583954
bpo-40795: ctypes calls unraisablehook with an exception (GH-20452)
If ctypes fails to convert the result of a callback or if a ctypes
callback function raises an exception, sys.unraisablehook is now
called with an exception set. Previously, the error was logged into
stderr by PyErr_Print().
(cherry picked from commit 10228bad04)

Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-27 16:00:01 -07:00
Pablo Galindo
9b83829e7d
[3.9] bpo-40614: Respect feature version for f-string debug expressions (GH-20196) (GH-20464)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
(cherry picked from commit c116c94)

Co-authored-by: Shantanu <hauntsaninja@users.noreply.github.com>
2020-05-27 22:01:11 +01:00
Miss Islington (bot)
3a2667d91e
bpo-30064: Fix asyncio loop.sock_* race condition issue (GH-20369)
(cherry picked from commit 210a137396)

Co-authored-by: Fantix King <fantix.king@gmail.com>
2020-05-27 13:39:03 -07:00
Miss Islington (bot)
788d7bfe18
bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914)
ctypes now raises an ArgumentError when a callback
is invoked with more than 1024 arguments.

The ctypes module allocates arguments on the stack in
ctypes_callproc() using alloca(), which is problematic
when large numbers of arguments are passed. Instead
of a stack overflow, this commit raises an ArgumentError
if more than 1024 parameters are passed.
(cherry picked from commit 29a1384c04)

Co-authored-by: Sean Gillespie <sean@swgillespie.me>
2020-05-27 08:46:57 -07:00
Miss Islington (bot)
1c88bf8541
bpo-39244: multiprocessing return default start method first on macOS (GH-18625)
(cherry picked from commit db098bc1f0)

Co-authored-by: idomic <michael.ido@gmail.com>
2020-05-26 08:13:33 -07:00
Miss Islington (bot)
7df9c41c69
bpo-40737: Fix possible reference leak for sqlite3 initialization (GH-20323)
(cherry picked from commit 5eb45d7d4e)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2020-05-26 05:38:51 -07:00
Miss Islington (bot)
5221a10dde
bpo-35714: Reject null characters in struct format strings (GH-16928)
struct.error is now raised if there is a null character in a struct
format string.
(cherry picked from commit 3f59b55316)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2020-05-26 00:04:59 -07:00
Lysandros Nikolaou
791a46ed58
[3.9] bpo-38964: Print correct filename on a SyntaxError in an fstring (GH-20399) (GH-20404)
When a `SyntaxError` in the expression part of a fstring is found,
the filename attribute of the `SyntaxError` is always `<fstring>`.
With this commit, it gets changed to always have the name of the file
the fstring resides in.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>.
(cherry picked from commit f7b1e46156)
2020-05-26 02:24:31 +01:00
Miss Islington (bot)
318a18eb88
bpo-23082: Better error message for PurePath.relative_to() from pathlib (GH-19611)
Co-authored-by: Sadhana Srinivasan <rotuna@Sadhanas-MBP.fritz.box>
(cherry picked from commit 448325369f)

Co-authored-by: Rotuna <sadhanasrinivasan@protonmail.com>
2020-05-25 13:01:20 -07:00
Miss Islington (bot)
82da2c3eb4
bpo-40750: Support -d flag in the new parser (GH-20340)
(cherry picked from commit 800a35c623)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 10:58:03 -07:00
Miss Islington (bot)
1fe1a14703
bpo-40671: Prepare _hashlib for PEP 489 (GH-20180)
(cherry picked from commit 20c22db602)

Co-authored-by: Christian Heimes <christian@python.org>
2020-05-25 05:18:49 -07:00
Miss Islington (bot)
7015823971
bpo-40695: Limit hashlib builtin hash fallback (GH-20259)
:mod:`hashlib` no longer falls back to builtin hash implementations when
OpenSSL provides a hash digest and the algorithm is blocked by security
policy.

Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 4cc2f9348c)

Co-authored-by: Christian Heimes <christian@python.org>
2020-05-25 02:07:35 -07:00
Miss Islington (bot)
1a4e9e6f35
bpo-36290: Fix keytword collision handling in AST node constructors (GH-12382)
(cherry picked from commit c73914a562)

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
2020-05-24 14:32:32 -07:00
Miss Islington (bot)
874506cff9
bpo-40723: Make IDLE autocomplete test run without __main__.__file__ (GH-20311)
This was the only failure running unittest.main(test.test_idle) after imports.
(cherry picked from commit 905b3cd05f)

Co-authored-by: Florian Dahlitz <f2dahlitz@freenet.de>
2020-05-24 04:14:17 -07:00
Miss Islington (bot)
1e5cf949ce
[3.9] bpo-39631: Adds NEWS entry (GH-20227) (GH-20241)
(cherry picked from commit 92327a9913)


Co-authored-by: Steve Dower <steve.dower@python.org>
2020-05-22 15:29:13 -07:00
Miss Islington (bot)
7f77ac463c
bpo-40696: Fix a hang that can arise after gen.throw() (GH-20287)
This updates _PyErr_ChainStackItem() to use _PyErr_SetObject()
instead of _PyErr_ChainExceptions(). This prevents a hang in
certain circumstances because _PyErr_SetObject() performs checks
to prevent cycles in the exception context chain while
_PyErr_ChainExceptions() doesn't.
(cherry picked from commit 7c30d12bd5)

Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
2020-05-22 14:35:22 -07:00
Miss Islington (bot)
a08b7c3bb0
bpo-9216: hashlib usedforsecurity fixes (GH-20258)
func:`hashlib.new` passed ``usedforsecurity`` to OpenSSL EVP constructor
``_hashlib.new()``. test_hashlib and test_smtplib handle strict security
policy better.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
(cherry picked from commit 909b5714e1)

Co-authored-by: Christian Heimes <christian@python.org>
2020-05-22 11:22:30 -07:00
Miss Islington (bot)
f541a371a5
bpo-40698: Improve distutils upload hash digests (GH-20260)
- Fix upload test on systems that blocks MD5
- Add SHA2-256 and Blake2b-256 digests based on new Warehous and twine
  specs.

Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e572c7f6db)

Co-authored-by: Christian Heimes <christian@python.org>
2020-05-20 07:57:08 -07:00
Miss Islington (bot)
e7bf8684e2
bpo-34956: edit and format better NEWS item in 3.9.0b1 changelog (GH-20255)
(cherry picked from commit bac170cd93)

Co-authored-by: Ned Deily <nad@python.org>
2020-05-20 02:47:07 -07:00
Victor Stinner
9512ad74b0
[3.9] bpo-40514: Remove --with-experimental-isolated-subinterpreters in 3.9 (GH-20228)
Remove --with-experimental-isolated-subinterpreters configure option
in Python 3.9: the experiment continues in the master branch, but
it's no longer needed in 3.9.
2020-05-20 00:27:46 +02:00
Miss Islington (bot)
b5bd4358fc
bpo-40683: Add zoneinfo to LIBSUBDIRS (GH-20229)
Without this, only the _zoneinfo module is getting installed, not the
zoneinfo module. I believe this was not noticed earlier because
test.test_zoneinfo was also not being installed.
(cherry picked from commit 2abededbc4)

Co-authored-by: Paul Ganssle <paul@ganssle.io>
2020-05-19 09:13:15 -07:00
Miss Islington (bot)
76d9a9cdc1
bpo-40677: Define IO_REPARSE_TAG_APPEXECLINK explicitly (GH-20206)
This allows building with older versions of the Windows SDK where the value is not defined.
(cherry picked from commit 711f9e180a)

Co-authored-by: Minmin Gong <gongminmin@msn.com>
2020-05-19 05:40:21 -07:00
Miss Islington (bot)
e2991308c9
bpo-32309: Implement asyncio.to_thread() (GH-20143)
Implements `asyncio.to_thread`, a coroutine for asynchronously running IO-bound functions in a separate thread without blocking the event loop. See the discussion starting from [here](https://github.com/python/cpython/pull/18410GH-issuecomment-628930973) in GH-18410 for context.

Automerge-Triggered-By: @aeros
(cherry picked from commit cc2bbc2227)

Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-05-19 03:03:25 -07:00
Łukasz Langa
97fe9cfd9f
Python 3.9.0b1 2020-05-19 01:07:09 +02:00