[3.12] gh-126238: Fix possible null pointer dereference of freevars in _PyCompile_LookupArg (GH-126239)
* Replace Py_DECREF by Py_XDECREF
(cherry picked from commit 8525c9375f)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
* [3.12] gh-125832: Clarify comment for inlined comprehensions as per PEP-709 (gh-126322)
* Fix comprehensions comment to inlined by pep 709
* Update spacing
Co-authored-by: RUANG (James Roy) <longjinyii@outlook.com>
* Add reference to PEP 709
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: RUANG (James Roy) <longjinyii@outlook.com>
* Add space
---------
Co-authored-by: rimchoi <hyerimc858@gmail.com>
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: RUANG (James Roy) <longjinyii@outlook.com>
[3.12] gh-126105: Fix crash in `ast` module, when `._fields` is deleted (GH-126115)
Previously, if the `ast.AST._fields` attribute was deleted, attempts to create a new `as`t node would crash due to the assumption that `_fields` always had a non-NULL value. Now it has been fixed by adding an extra check to ensure that `_fields` does not have a NULL value (this can happen when you manually remove `_fields` attribute).
(cherry picked from commit b2eaa75b17)
Co-authored-by: sobolevn <mail@sobolevn.me>
This backports several PRs for gh-113993, making interned strings mortal so they can be garbage-collected when no longer needed.
* Allow interned strings to be mortal, and fix related issues (GH-120520)
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
* Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (GH-121364)
* Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs
* Document immortality in some functions that take `const char *`
This is PyUnicode_InternFromString;
PyDict_SetItemString, PyObject_SetAttrString;
PyObject_DelAttrString; PyUnicode_InternFromString;
and the PyModule_Add convenience functions.
Always point out a non-immortalizing alternative.
* Don't immortalize user-provided attr names in _ctypes
* Immortalize names in code objects to avoid crash (GH-121903)
* Intern latin-1 one-byte strings at startup (GH-122303)
There are some 3.12-specific changes, mainly to allow statically allocated strings in deepfreeze. (In 3.13, deepfreeze switched to the general `_Py_ID`/`_Py_STR`.)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* Detect source file encoding.
* Use the "replace" error handler even for UTF-8 (default) encoding.
* Remove the BOM.
* Fix detection of too long lines if they contain NUL.
* Return the head rather than the tail for truncated long lines.
(cherry picked from commit e2f710792b)
gh-123892: Add "_wmi" to sys.stdlib_module_names (GH-123893)
(cherry picked from commit fb1b51a58d)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix _PyArg_UnpackKeywordsWithVararg for the case when argument for
positional-or-keyword parameter is passed by keyword.
There was only one such case in the stdlib -- the TypeVar constructor.
(cherry picked from commit 540fcc62f5)
Fix crash when importing ssl after re-initialization
The current METH_FASTCALL|METH_KEYWORDS functions in a non-builtin module can cause segfaults after restarting the main interpreter, invoking _PyArg_UnpackKeywords() with an insufficiently cleared _PyArg_Parser struct.
This patch fixes the invalidation of the static argument parsers.
gh-122029: Log call events in sys.setprofile when it's a method with c function (GH-122072)
Log call events in sys.setprofile when it is a method with a C function.
(cherry picked from commit e91ef13861)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
* The result has type Py_ssize_t, not intptr_t.
* Type cast between unsigned and signed integer types should be explicit.
* Downcasting should be explicit.
* Fix integer overflow check in sum().
(cherry picked from commit 1801545)
gh-121390: tracemalloc: Fix tracebacks memory leak (GH-121391)
The tracemalloc_tracebacks hash table has traceback keys and NULL
values, but its destructors do not reflect this -- key_destroy_func is
NULL while value_destroy_func is raw_free. Swap these to free the
traceback keys instead.
(cherry picked from commit db39bc42f9)
Co-authored-by: Josh Brobst <jbrobst@proton.me>
gh-120155: Fix optimize_and_assemble_code_unit() error handling
Don't use 'g' before it's being initialized: don't use the 'error'
label if consts_dict_keys_inorder() failed.
Fix the Coverity issue:
Error: UNINIT (CWE-457):
Python-3.12.2/Python/compile.c:7670:5: skipped_decl: Jumping over declaration of ""g"".
Python-3.12.2/Python/compile.c:7714:5: uninit_use_in_call: Using uninitialized value ""g.g_block_list"" when calling ""_PyCfgBuilder_Fini"".
Python-3.12.2/Python/compile.c:7714:5: uninit_use_in_call: Using uninitialized value ""g.g_entryblock"" when calling ""_PyCfgBuilder_Fini"".
7712| Py_XDECREF(consts);
7713| instr_sequence_fini(&optimized_instrs);
7714|-> _PyCfgBuilder_Fini(&g);
7715| return co;
7716| }
gh-111499: Fix PYTHONMALLOCSTATS at Python exit (#120021)
Call _PyObject_DebugMallocStats() earlier in Py_FinalizeEx(), before
the interpreter is deleted.
(cherry picked from commit 5a1205b641)
The implementation basically copies LOAD_GLOBAL. Possibly it could be deduplicated,
but that seems like it may get hairy since the two operations have different operands.
This is important to fix in 3.14 for PEP 649, but it's a bug in earlier versions too,
and we should backport to 3.13 and 3.12 if possible.
(cherry picked from commit 80a4e38994)
Make sure that `gilstate_counter` is not zero in when calling
`PyThreadState_Clear()`. A destructor called from `PyThreadState_Clear()` may
call back into `PyGILState_Ensure()` and `PyGILState_Release()`. If
`gilstate_counter` is zero, it will try to create a new thread state before
the current active thread state is destroyed, leading to an assertion failure
or crash.
(cherry picked from commit bcc1be39cb)
- Cache line object to avoid creating a Unicode object
for all of the tokens in the same line.
- Speed up byte offset to column offset conversion by using the
smallest buffer possible to measure the difference.
(cherry picked from commit d87b015106)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
_PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed.
This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple.
This change also fixes a bug where only the most recent parser was added to the global linked list.
(cherry picked from commit 81865002ae)
gh-118513: Fix sibling comprehensions with a name bound in one and global in the other (GH-118526)
(cherry picked from commit c8deb1e4b4)
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
(cherry picked from commit c1bf4874c1)
* document equivalent command-line options for all environment variables
* document equivalent environment variables for all command-line options
* reduce the size of variable and option descriptions to minimum
* remove the ending period in single-sentence descriptions
(cherry picked from commit b85572c47d)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Fix the description of the "-b" option.
* Add references to environment variables for "-s" and "-X dev" options.
(cherry picked from commit 33662d4e01)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-116447: Fix possible UB in `arraymodule` and `getargs` (GH-116459)
(cherry picked from commit fdb2d90a27)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
