Commit graph

8120 commits

Author SHA1 Message Date
Serhiy Storchaka
4be9fa8961
[3.12] gh-90300: Improve the Python CLI help output (GH-115853) (GH-117022)
* document equivalent command-line options for all environment variables
* document equivalent environment variables for all command-line options
* reduce the size of variable and option descriptions to minimum
* remove the ending period in single-sentence descriptions

(cherry picked from commit b85572c47d)

Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-03-19 18:33:34 +00:00
Tian Gao
688623d402
[3.12] gh-116735: Use MISSING for CALL event if argument is absen… (#116873)
[3.12] gh-116735: Use `MISSING` for `CALL` event if argument is absent (GH-116737)
2024-03-19 17:00:54 +00:00
Antoine Pitrou
25243b1461
[3.12] gh-112536: Add TSAN build on Github Actions (GH-116872)
(cherry picked from commit 20578a1f68)

Co-authored-by: Donghee Na <donghee.na@python.org>
2024-03-18 09:52:54 +00:00
Tian Gao
56a3c5f767
[3.12] gh-116626: Emit CALL events for all `INSTRUMENTED_CALL_FUNCTION_EX (GH-116732)
Backport of GH-116627
2024-03-14 16:23:15 +00:00
Serhiy Storchaka
fc4d5fdffe
[3.12] gh-90300: Fix undocumented envvars in the Python CLI help (GH-116765) (GH-116797)
(cherry picked from commit 19ac28bd08)
2024-03-14 14:50:39 +02:00
Serhiy Storchaka
991710af02
[3.12] gh-90300: Document equivalent -X options for envvars in the Python CLI help (GH-116756) (GH-116786)
(cherry picked from commit 8c6db45ce3)
2024-03-14 12:04:14 +02:00
Serhiy Storchaka
870cd901ea
[3.12] gh-90300: Sort the -X options and some envvars in the Python CLI help (GH-116739) (GH-116766)
(cherry picked from commit e54bdeab9c)
2024-03-13 23:02:29 +00:00
Miss Islington (bot)
89b5c5e321
[3.12] gh-90300: Fix cmdline.rst (GH-116721) (GH-116724)
* Fix the description of the "-b" option.
* Add references to environment variables for "-s" and "-X dev" options.
(cherry picked from commit 33662d4e01)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-03-13 13:46:58 +00:00
Miss Islington (bot)
2647afeab6
[3.12] gh-116447: Fix possible UB in arraymodule and getargs (GH-116459) (#116496)
gh-116447: Fix possible UB in `arraymodule` and `getargs` (GH-116459)
(cherry picked from commit fdb2d90a27)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-03-08 11:22:07 +00:00
Miss Islington (bot)
9a42cfa368
[3.12] gh-116326: Handler errors correctly in getwindowsversion in sysmodule (GH-116339) (#116354)
* gh-116326: Handler errors correctly in `getwindowsversion` in `sysmodule` (GH-116339)
(cherry picked from commit c91bdf86ef)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-03-05 18:52:15 +03:00
Miss Islington (bot)
0f70c14939
[3.12] gh-115320: Refactor get_hash_info in sysmodule.c not to swallow errors (GH-115321) (#116323)
gh-115320: Refactor `get_hash_info` in `sysmodule.c` not to swallow errors (GH-115321)
(cherry picked from commit 207030f552)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-03-04 21:13:38 +00:00
Irit Katriel
6359141867
[3.12] gh-116034: fix location info on the error of a failed assertion (#116054) 2024-02-29 12:36:54 +00:00
Łukasz Langa
2bdd1d6f15
[3.12] gh-90300: Reformat the Python CLI help output (GH-93415) (#115847)
(cherry picked from commit 2e92ffd7fa)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-02-23 18:23:16 +01:00
Miss Islington (bot)
bf0e072980
[3.12] gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561) (GH-115603)
(cherry picked from commit 664965a1c1)

Co-authored-by: wookie184 <wookie1840@gmail.com>
2024-02-17 14:49:39 +02:00
Mark Shannon
4d87832d87
[3.12] GH-112215: Backport C recursion changes (GH-115083) 2024-02-13 10:45:59 +01:00
Serhiy Storchaka
d8346d6c06
[3.12] gh-115011: Improve support of __index__() in setters of members with unsigned integer type (GH-115029) (GH-115294)
Setters for members with an unsigned integer type now support
the same range of valid values for objects that has a __index__()
method as for int.

Previously, Py_T_UINT, Py_T_ULONG and Py_T_ULLONG did not support
objects that has a __index__() method larger than LONG_MAX.

Py_T_ULLONG did not support negative ints. Now it supports them and
emits a RuntimeWarning.
(cherry picked from commit d9d6909697)
2024-02-11 11:56:17 +00:00
Miss Islington (bot)
967f27d0a1
[3.12] gh-114828: Fix __class__ in class-scope inlined comprehensions (GH-115139) (#115140)
gh-114828: Fix __class__ in class-scope inlined comprehensions (GH-115139)
(cherry picked from commit fedbf77191)

Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-02-07 17:12:54 +00:00
Serhiy Storchaka
b9937a6f22
[3.12] gh-114388: Fix warnings when assign an unsigned integer member (GH-114391) (GH-115001)
* Fix a RuntimeWarning emitted when assign an integer-like value that
  is not an instance of int to an attribute that corresponds to a C
  struct member of type T_UINT and T_ULONG.
* Fix a double RuntimeWarning emitted when assign a negative integer value
  to an attribute that corresponds to a C struct member of type T_UINT.
(cherry picked from commit 3ddc515255)
2024-02-04 17:21:05 +00:00
Miss Islington (bot)
2a1d2c8325
[3.12] gh-114685: Fix incorrect use of PyBUF_READ in import.c (GH-114686) (GH-114700)
(cherry picked from commit 1ac1b2f953)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-01-29 10:09:51 +00:00
Jérome Perrin
00e8c9ce9e
[3.12] gh-113358: Fix rendering tracebacks with exceptions with a broken __getattr__ (GH-113359) (#114173) 2024-01-21 17:12:17 +00:00
Miss Islington (bot)
afefa4a74c
[3.12] gh-114384: Align sys.set_asyncgen_hooks signature in docs to reflect implementation (GH-114385) (#114386)
(cherry picked from commit 38768e4cdd)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-01-21 13:22:57 +00:00
Peter Lazorchak
ae2a25bf60
[3.12] Check for valid tp_version_tag in specializer (gh-89811) (gh-114216) 2024-01-20 04:45:33 +08:00
Miss Islington (bot)
6b6f91ec88
[3.12] gh-113842: Add missing error check for PyIter_Next() in Python/symtable.c (GH-113843) (GH-113851)
(cherry picked from commit fda901a1ff)

Co-authored-by: Yan Yanchii <yyanchiy@gmail.com>
2024-01-09 11:00:07 +00:00
Serhiy Storchaka
d58a5f453f
[3.12] gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. (GH-113035) (GH-113472)
(cherry picked from commit 48c49739f5)

Co-authored-by: Yilei Yang <yileiyang@google.com>
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-12-25 19:20:07 +00:00
Irit Katriel
9d72a5cae7
[3.12] gh-113297: Fix segfault in compiler for with statement with 19 context managers (#113327) (#113404) 2023-12-23 13:29:11 +00:00
Miss Islington (bot)
c38ebb40d1
[3.12] gh-113343: Fix error check on mmap(2) (GH-113342) (#113374)
gh-113343: Fix error check on mmap(2) (GH-113342)

Fix error check on mmap(2)

It should check MAP_FAILED instead of NULL for error.

On mmap(2) man page:

  RETURN VALUE
       On success, mmap() returns a pointer to the mapped area.
       On error, the value MAP_FAILED (that is, (void *) -1) is
       returned, and errno is set to indicate the error.
(cherry picked from commit 6b70c3dc5a)

Co-authored-by: Namhyung Kim <namhyung@gmail.com>
2023-12-21 19:44:15 +00:00
Serhiy Storchaka
59a22d3e50
[3.12] gh-112716: Fix SystemError when __builtins__ is not a dict (GH-112770) (GH-113103)
It was raised in two cases:
* in the import statement when looking up __import__
* in pickling some builtin type when looking up built-ins iter, getattr, etc.

(cherry picked from commit 1161c14e8c)
2023-12-14 12:54:25 +00:00
Pablo Galindo Salgado
e4d2fb242a
[3.12] gh-112943: Correctly compute end offsets for multiline tokens in the tokenize module (GH-112949) (#112957)
(cherry picked from commit a135a6d2c6)
2023-12-11 12:48:19 +00:00
Victor Stinner
34d57d5871
[3.12] gh-106560: Fix redundant declarations in Python/frozen.c (#112612) (#112651)
gh-106560: Fix redundant declarations in Python/frozen.c (#112612)

Avoid duplicated declarations of "extern" functions in
Python/frozen.c.

Compiler warnings seen by building Python with gcc -Wredundant-decls.

(cherry picked from commit d9e444dbb8)
2023-12-03 11:54:59 +00:00
Pablo Galindo Salgado
11232c1299
[3.12] gh-112367: Only free perf trampoline arenas at shutdown (GH-112368) (#112590)
(cherry picked from commit a73aa48e6b)

Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-12-01 16:05:31 +00:00
Dennis Sweeney
a99305e7fb
[3.12] gh-112356: LOAD_GLOBAL can only include one PUSH_NULL (#112566) 2023-12-01 00:35:49 +00:00
Eric Snow
1e1a30f9f4
[3.12] gh-110310: Add a Per-Interpreter XID Registry for Heap Types (gh-110311) (gh-110714)
We do the following:

* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()

The per-interpreter registry helps preserve isolation between interpreters.  This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.

(cherry-picked from commit 80dc39e1dc)
2023-11-28 02:36:29 +00:00
Eric Snow
0122b4d7c9
[3.12] gh-105716: Support Background Threads in Subinterpreters Consistently (gh-109921) (gh-110707)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.

We add PyInterpreterState.threads.main, with some internal C-API functions.

(cherry-picked from commit 1dd9dee45d)
2023-11-27 19:01:05 -07:00
Eric Snow
82ae5a609d
[3.12] gh-109793: Allow Switching Interpreters During Finalization (gh-109794) (gh-110705)
Essentially, we should check the thread ID rather than the thread state pointer.
2023-11-28 00:58:02 +00:00
Eric Snow
4f71f1680d
[3.12] gh-106931: Intern Statically Allocated Strings Globally (gh-107272) (gh-110713)
We tried this before with a dict and for all interned strings.  That ran into problems due to interpreter isolation.  However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning.  Here we circle back to using a global cache, but only for statically allocated strings.  We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.

Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter.  Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.

(cherry-picked from commit b72947a8d2)
2023-11-27 23:51:12 +00:00
Eric Snow
313554457e
[3.12] gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994) (gh-110701)
This change makes sure sys.path[0] is set properly for subinterpreters.  Before, it wasn't getting set at all.

This change does not address the broader concerns from gh-109853.

(cherry-picked from commit a040a32ea2)
2023-11-27 22:21:12 +00:00
Eric Snow
592a849fdf
[3.12] gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556) (gh-112288)
This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
(cherry picked from commit fd7e08a6f3)
2023-11-27 14:49:48 -07:00
Miss Islington (bot)
46047bb12b
[3.12] gh-112438: Fix support of format units with the "e" prefix in nested tuples in PyArg_Parse (gh-112439) (GH-112460)
(cherry picked from commit 4eea1e8236)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-11-27 17:57:30 +00:00
Miss Islington (bot)
dfdbfc548f
gh-111856: Fix os.fstat on windows with FAT32 and exFAT filesystem (GH-112038)
(cherry picked from commit 29af7369db)

Co-authored-by: AN Long <aisk@users.noreply.github.com>
2023-11-13 16:25:01 +00:00
Miss Islington (bot)
158a7e1acd
[3.12] gh-111933: fix broken link to A.Neumaier article (gh-111937) (gh-111993) 2023-11-12 01:22:44 +00:00
Victor Stinner
4b0c875d91
[3.12] gh-109181: Fix refleak in tb_get_lineno() (#111948)
PyFrame_GetCode() returns a strong reference.
2023-11-10 14:07:45 +01:00
Jelle Zijlstra
1a95ad68b4
[3.12] gh-111654: remove redundant decref in LOAD_FROM_DICT_OR_DEREF (GH-111655) (#111674)
(cherry picked from commit 3a1b09e6d0)

Co-authored-by: AN Long <aisk@users.noreply.github.com>
2023-11-03 07:02:07 -07:00
Pablo Galindo Salgado
4f619e83fe
[3.12] gh-109181: Speed up Traceback object creation by lazily compute the line number (GH-111548) (#111551)
.
(cherry picked from commit abb15420c1)
2023-10-31 18:18:11 +00:00
Pablo Galindo Salgado
c81ebf5b3d
[3.12] bpo-43950: handle wide unicode characters in tracebacks (GH-28150) (#111346) 2023-10-27 06:43:38 +09:00
Nikita Sobolev
b6755d81d4
[3.12] gh-109216: Fix possible memory leak in BUILD_MAP (#109324) 2023-10-15 07:08:40 +05:30
Miss Islington (bot)
dae3db1883
[3.12] gh-110514: Add PY_THROW to sys.setprofile events (GH-110524) (#110541)
gh-110514: Add PY_THROW to `sys.setprofile` events (GH-110524)
(cherry picked from commit dd4bb0529e)

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
2023-10-09 09:11:51 +00:00
Miss Islington (bot)
ef4bd1b57f
[3.12] gh-110237: Check PyList_Append for errors in _PyEval_MatchClass (GH-110238) (#110511)
gh-110237: Check `PyList_Append` for errors in `_PyEval_MatchClass` (GH-110238)
(cherry picked from commit dd9d781da3)

Co-authored-by: denballakh <47365157+denballakh@users.noreply.github.com>
2023-10-08 00:40:38 +00:00
Miss Islington (bot)
2465fe0014
[3.12] GH-110455: Guard assert(tstate->thread_id > 0) with GH-ifndef HAVE_PTHREAD_STUBS (GH-110487) (GH-110491)
GH-110455: Guard `assert(tstate->thread_id > 0)` with `GH-ifndef HAVE_PTHREAD_STUBS` (GH-110487)
(cherry picked from commit 5fd8821cf8)

Co-authored-by: Brett Cannon <brett@python.org>
2023-10-06 23:48:48 +00:00
Victor Stinner
4936fa9541
[3.12] gh-108987: Fix _thread.start_new_thread() race condition (#109135) (#110342)
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)

Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.

thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.

Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().

(cherry picked from commit 517cd82ea7)

* gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808)

(cherry picked from commit 1b8f2366b3)

---------

Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
2023-10-04 11:20:31 +00:00
Victor Stinner
d0641ab676
[3.12] gh-110052: Fix faulthandler for freed tstate (#110069) (#110071)
gh-110052: Fix faulthandler for freed tstate (#110069)

faulthandler now detected freed interp and freed tstate, and no
longer dereference them.

(cherry picked from commit 2e37a38bcb)
2023-10-02 17:44:51 +02:00