Commit graph

7550 commits

Author SHA1 Message Date
Miss Islington (bot)
a4ff0eaaf0
gh-96848: Fix -X int_max_str_digits option parsing (GH-96988)
Fix command line parsing: reject "-X int_max_str_digits" option with
no value (invalid) when the PYTHONINTMAXSTRDIGITS environment
variable is set to a valid limit.
(cherry picked from commit 41351662bc)

Co-authored-by: Victor Stinner <vstinner@python.org>
2022-09-26 08:43:26 -07:00
Miss Islington (bot)
6a646dd1ff
GH-96975: Skip incomplete frames in PyEval_GetFrame (GH-97018)
(cherry picked from commit 8fd2c3b75b)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2022-09-22 10:17:53 -07:00
Brandt Bucher
fb87aaafba
Remove a stray DISPATCH() (GH-97004) 2022-09-22 09:18:37 -07:00
Brandt Bucher
0e61d2b557
[3.11] GH-95921: Fix positions for some chained comparisons (GH-96968) (GH-96973)
(cherry picked from commit dfc73b5724)

Automerge-Triggered-By: GH:brandtbucher
2022-09-20 14:56:38 -07:00
Miss Islington (bot)
32d80decbf
GH-96864: Check for error between line and opcode events (GH-96880)
(cherry picked from commit c10e33ac11)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2022-09-20 10:49:01 -07:00
Victor Stinner
6ff54716f1
gh-96387: take_gil() resets drop request before exit (#96869) (#96941)
At Python exit, sometimes a thread holding the GIL can wait forever
for a thread (usually a daemon thread) which requested to drop the
GIL, whereas the thread already exited. To fix the race condition,
the thread which requested the GIL drop now resets its request before
exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before
PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a
race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.

(cherry picked from commit 04f4977f50)
2022-09-20 02:12:09 +02:00
Miss Islington (bot)
c4cf745c72
GH-96754: Check whether the interpreter frame is complete before creating frame object. (GH-96776) (#96787)
(cherry picked from commit 12c5f328d2)

Co-authored-by: Mark Shannon <mark@hotpy.org>

Co-authored-by: Mark Shannon <mark@hotpy.org>
2022-09-13 11:56:27 +01:00
Christian Heimes
4958820032
[3.11] gh-95853: Add script to automate WASM build (GH-95828, GH-95985, GH-96045, GH-96389, GH-96744) (GH-96749)
Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.
2022-09-13 12:06:39 +02:00
Miss Islington (bot)
7033dc8adc
GH-96678: Fix undefined behavior in ceval.c (GH-96708)
(cherry picked from commit 50a70a083d)

Co-authored-by: Mark Shannon <mark@hotpy.org>
2022-09-10 06:58:45 -07:00
Miss Islington (bot)
c563b89261
Fix possible NULL pointer dereference in _PyThread_CurrentFrames (GH-96584)
(cherry picked from commit 88a7f661ca)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-09-10 04:12:46 -07:00
Mark Shannon
5586da65e2
[3.11] GH-96636: Remove all uses of NOTRACE_DISPATCH (GH-96643) (#96688)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit aa3b4cf779)
2022-09-09 09:24:44 -07:00
Mark Shannon
e72f469e85
[3.11] GH-96569: Avoid undefined behavior (#96616)
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2022-09-08 12:00:04 +01:00
Miss Islington (bot)
a389fdb095
GH-96572: fix use after free in trace refs build mode (GH-96618)
(cherry picked from commit 67444902a0)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-09-06 11:28:18 -07:00
Gregory P. Smith
f8b71da9aa
[3.11] gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96500)
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.

This PR comes fresh from a pile of work done in our private PSRT security response team repo.

This backports https://github.com/python/cpython/pull/96499 aka 511ca94520

Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).

<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->

I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#).
2022-09-02 09:48:57 -07:00
Miss Islington (bot)
27f390bc30
gh-96125: Fix sys.thread_info.name on pthread platforms (GH-96126) (#96128)
Automerge-Triggered-By: GH:tiran
(cherry picked from commit 822955c166)

Co-authored-by: Christian Heimes <christian@python.org>

Co-authored-by: Christian Heimes <christian@python.org>
2022-08-21 17:37:15 +01:00
Miss Islington (bot)
1fa9644c23
GH-96071: fix deadlock in PyGILState_Ensure (GH-96124) (#96129)
Alternative of GH-96107
(cherry picked from commit e0d54a4a79)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-08-19 22:11:31 +01:00
Miss Islington (bot)
6fc90c1183
GH-95818: Skip incomplete frames in PyThreadState_GetFrame (GH-95886) (#95890)
(cherry picked from commit 1b46d118e6)

Co-authored-by: Mark Shannon <mark@hotpy.org>

Co-authored-by: Mark Shannon <mark@hotpy.org>
2022-08-12 19:40:49 +01:00
Christian Heimes
e62a0dfab2
[3.11] gh-95174: Add pthread stubs for WASI (GH-95234) (#95503)
Co-authored-by: Brett Cannon <brett@python.org>.
(cherry picked from commit 0fe645d6fd)

Co-authored-by: Christian Heimes <christian@python.org>
2022-08-01 15:37:45 +01:00
Miss Islington (bot)
147a9a8be7
bpo-45445: Revert "bpo-45445: Fail if an invalid X-option is provided in the command line (GH-28823)" (GH-94745)
(cherry picked from commit aa37ffda29)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2022-07-31 08:58:02 -07:00
Christian Heimes
db13c0c1b8
[3.11] gh-95174: Handle missing dup() and constants in WASI (GH-95229) (GH-95272)
Co-authored-by: Christian Heimes <christian@python.org>
2022-07-31 16:39:41 +01:00
Miss Islington (bot)
00566a8124
GH-90081: Run python tracers at full speed (GH-95328) (#95363)
(cherry picked from commit b8b2990fb3)

Co-authored-by: Mark Shannon <mark@hotpy.org>

Co-authored-by: Mark Shannon <mark@hotpy.org>
2022-07-29 09:43:52 +01:00
Serhiy Storchaka
33efd7f7a3
[3.11] gh-94938: Fix errror detection of unexpected keyword arguments (GH-94999) (GH-95353)
When keyword argument name is an instance of a str subclass with
overloaded methods __eq__ and __hash__, the former code could not find
the name of an extraneous keyword argument to report an error, and
_PyArg_UnpackKeywords() returned success without setting the
corresponding cell in the linearized arguments array. But since the number
of expected initialized cells is determined as the total number of passed
arguments, this lead to reading NULL as a keyword parameter value, that
caused SystemError or crash or other undesired behavior.
(cherry picked from commit ebad53a4dc)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-07-28 09:51:45 +03:00
Miss Islington (bot)
86eb500068
[3.11] gh-95185: Check recursion depth in the AST constructor (GH-95186) (GH-95208)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit 0047447294)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2022-07-26 12:19:22 +02:00
Mark Shannon
e5ff5ec3ff
[3.11] GH-94739: Backport GH-94958 to 3.11 (#94965) 2022-07-25 12:11:06 +01:00
Brandt Bucher
064462a719
[3.11] GH-94036: Fix more attribute location quirks (GH-95028) (GH-95156)
(cherry picked from commit 900bfc53cb)
2022-07-22 17:31:06 -07:00
Brandt Bucher
5a48ab01e9
[3.11] GH-95113: Don't use EXTENDED_ARG_QUICK in unquickened code (GH-95121) (GH-95143)
(cherry picked from commit e402b26b7f)
2022-07-22 11:56:10 -07:00
Brandt Bucher
e2fce3a8e7
[3.11] GH-91409: Don't overwrite valid locations with NOP locations (GH-95067) (GH-95068)
(cherry picked from commit 742d4614e1)
2022-07-20 15:01:42 -07:00
Miss Islington (bot)
16cb8ca52e
GH-94851: fix immortal objects refcounting in compiler (gh-95040)
(cherry picked from commit 7476154886)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-07-20 03:42:05 -07:00
Miss Islington (bot)
bb8e20a329
gh-91256: Ensure help text has the program name even before getpath is called (GH-94929)
(cherry picked from commit 49aeff49d7)

Co-authored-by: Steve Dower <steve.dower@python.org>
2022-07-19 12:03:15 -07:00
Brandt Bucher
eda2f90094
[3.11] GH-94822: Don't specialize when metaclasses are involved (GH-94892) (GH-94980)
(cherry picked from commit daf68ba92f)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2022-07-18 11:55:07 -07:00
Miss Islington (bot)
ec6ed6681d
gh-94930: skipitem() in getargs.c should return non-NULL on error (GH-94931)
(cherry picked from commit 067f0da335)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-07-18 08:41:22 -07:00
Miss Islington (bot)
1c63734217
gh-91348: Restore frame argument to sys._getframe audit event (GH-94928)
(cherry picked from commit 044a593cbb)

Co-authored-by: Steve Dower <steve.dower@python.org>
2022-07-17 08:49:57 -07:00
Serhiy Storchaka
107c21c5d5
gh-94864: Fix PyArg_Parse* with deprecated format units "u" and "Z" (GH-94902)
It returned 1 (success) when warnings are turned into exceptions.
2022-07-17 08:23:37 +03:00
John Belmonte
45896f2a02
[3.11] gh-93883: elide traceback indicators when possible (GH-93994) (GH-94740)
Elide traceback column indicators when the entire line of the
frame is implicated.  This reduces traceback length and draws
more attention to the remaining (very relevant) indicators.

Example:
```
Traceback (most recent call last):
  File "query.py", line 99, in <module>
    bar()
  File "query.py", line 66, in bar
    foo()
  File "query.py", line 37, in foo
    magic_arithmetic('foo')
  File "query.py", line 18, in magic_arithmetic
    return add_counts(x) / 25
           ^^^^^^^^^^^^^
  File "query.py", line 24, in add_counts
    return 25 + query_user(user1) + query_user(user2)
                ^^^^^^^^^^^^^^^^^
  File "query.py", line 32, in query_user
    return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)
                               ~~~~~~~~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable
```

Automerge-Triggered-By: GH:pablogsal
2022-07-11 04:27:29 -07:00
Brandt Bucher
e5c8ad3e15
[3.11] GH-94694: Fix column offsets for multi-line method lookups (GH-94721)
(cherry picked from commit 264b3ddfd5)
2022-07-09 22:12:45 -07:00
Kumar Aditya
7a341724e4
[3.11] GH-93252: Fix error handling for failed Python calls (GH-94693) (GH-94708)
Automerge-Triggered-By: GH:tiran
2022-07-09 05:09:15 -07:00
Christian Heimes
36a3372d51
[3.11] gh-94215: Fix error handling for line-tracing events (GH-94681) (GH-94688)
* Re-enable crasher
* Fix error handling for line-tracing events
* blurb add
(cherry picked from commit 23ee4a8067)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2022-07-08 15:25:53 +02:00
Christian Heimes
74c953d396
[3.11] gh-92228: disable the compiler's 'small exit block inlining' optimization for blocks that have a line number (GH-94592) (GH-94643)
Inlining of code that corresponds to source code lines, can make it hard to distinguish later between code which is only reachable from except handlers, and that which is reachable in normal control flow. This caused problems with the debugger's jump feature.

This PR turns off the inlining optimisation for code which has line numbers. We still inline things like the implicit "return None"..
(cherry picked from commit bde06e1b83)

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2022-07-07 03:10:32 -07:00
Łukasz Langa
5f4a16b291
[3.11] gh-94510: Raise on re-entrant calls to sys.setprofile and sys.settrace (GH-94511) (GH-94578)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit 40d81fd63b)
2022-07-05 21:02:43 +02:00
Łukasz Langa
1bfe83a114
[3.11] gh-94485: Set line number of module's RESUME instruction to 0 as specified by PEP 626 (GH-94552) (GH-94562)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Mark Shannon <mark@hotpy.org>

(cherry picked from commit 324d01944d)
2022-07-05 16:01:24 +02:00
Miss Islington (bot)
68f5fa6683
[3.11] GH-94262: Don't create frame objects for frames that aren't yet complete. (GH-94371) (#94482)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2022-07-04 19:43:12 +01:00
Mark Shannon
113b309f18
[3.11] GH-93354: Use exponential backoff to avoid excessive specialization attempts (GH-93355) (GH-93379)
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-06-30 23:03:37 +02:00
Miss Islington (bot)
fdc008138e
gh-91719: Reload opcode on unknown error so that C can optimize the dispatching in ceval.c (GH-94364) (#94453)
(cherry picked from commit ea39b77de9)

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
2022-06-30 09:11:05 -07:00
Miss Islington (bot)
f58c366a73
GH-94329: Don't raise on excessive stack consumption (GH-94421) (GH-94446)
(cherry picked from commit b152bf448b)
2022-06-30 15:53:20 +01:00
Irit Katriel
48a739ec10
gh-94332: make it safe to call assemble_free when assemble_init has not been called (GH-94389) (GH-94442)
(cherry picked from commit be82d26570)
2022-06-30 15:30:12 +01:00
Christian Heimes
9140c413d0
[3.11] GH-93516: Drop broken assert, fixes GH-93769 (GH-94411) 2022-06-29 13:53:56 +01:00
Mark Shannon
3b4f5ed168
[3.11] GH-93516: Backport GH-93769 (GH-94231)
* Store offset of first traceable instruction to avoid having to recompute it all the time when tracing.
2022-06-28 16:30:22 +01:00
Mark Shannon
3ece6e6feb
[3.11] GH-93516: Backport GH-93769: Speedup line number checks when tracing (GH-94127)
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2022-06-22 16:32:02 +01:00
Miss Islington (bot)
2733c64e28
gh-94021: Address unreachable code warning in specialize code (GH-94022)
(cherry picked from commit 77c839c98f)

Co-authored-by: Christian Heimes <christian@python.org>
2022-06-21 00:01:52 -07:00
Miss Islington (bot)
b8fe3bd1d4
gh-91985: Ensure in-tree builds override platstdlib_dir in every path calculation (GH-93641)
(cherry picked from commit 38af903506)

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
2022-06-20 11:37:27 -07:00