Starting in Python 3.12, we prevented calling fork() and starting new threads
during interpreter finalization (shutdown). This has led to a number of
regressions and flaky tests. We should not prevent starting new threads
(or `fork()`) until all non-daemon threads exit and finalization starts in
earnest.
This changes the checks to use `_PyInterpreterState_GetFinalizing(interp)`,
which is set immediately before terminating non-daemon threads.
(cherry picked from commit 60e105c1c1)
gh-110819: Fix ‘kind’ may be used uninitialized warning in `longobject` (GH-116599)
(cherry picked from commit eb947cdc13)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Fixes GH-109894
* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init
---------
(cherry picked from commit 47d3e2ed93)
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.
Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
(cherry-picked from commit b72947a8d2)
gh-112266: Remove `(if defined)` part from `__dict__` and `__weakref__` docstrings (GH-112268)
(cherry picked from commit f8129146ef)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-110782: Fix crash when TypeVar is constructed with keyword args (GH-110784)
(cherry picked from commit d2a536b170)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
(cherry picked from commit 2b15536fa9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.12] gh-86457: Fix signature for code.replace() (GH-23199)
Also add support of @text_signature in Argument Clinic..
(cherry picked from commit 0e6e32fb84)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Update 2023-08-07-16-30-48.gh-issue-95065.-im4R5.rst
gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
(cherry picked from commit 58ef741867)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-106092: Fix use-after-free crash in frame_dealloc (GH-106875)
It was possible for the trashcan to delay the deallocation of a
PyFrameObject until after its corresponding _PyInterpreterFrame has
already been freed. So frame_dealloc needs to avoid dereferencing the
f_frame pointer unless it first checks that the pointer still points
to the interpreter frame within the frame object.
(cherry picked from commit 557b05c7a5)
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Co-authored-by: Anders Kaseorg <andersk@mit.edu>
The _xxsubinterpreters module was meant to only use public API. Some internal C-API usage snuck in over the last few years (e.g. gh-28969). This fixes that.
(cherry picked from commit e6373c0d8b)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-105699: Fix an Interned Strings Crasher (gh-106930)
A static (process-global) str object must only have its "interned" state cleared when no longer interned in any interpreters. They are the only ones that can be shared by interpreters so we don't have to worry about any other str objects.
We trigger clearing the state with the main interpreter, since no other interpreters may exist at that point and _PyUnicode_ClearInterned() is only called during interpreter finalization.
We do not address here the fact that a string will only be interned in the first interpreter that interns it. In any subsequent interpreters str.state.interned is already set so _PyUnicode_InternInPlace() will skip it. That needs to be addressed separately from fixing the crasher.
(cherry picked from commit 87e7cb09e4)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-105699: Fix a Crasher Related to a Deprecated Global Variable (gh-106923)
There was a slight race in _Py_ClearFileSystemEncoding() (when called from _Py_SetFileSystemEncoding()), between freeing the value and setting the variable to NULL, which occasionally caused crashes when multiple isolated interpreters were used. (Notably, I saw at least 10 different, seemingly unrelated spooky-action-at-a-distance, ways this crashed. Yay, free threading!) We avoid the problem by only setting the global variables with the main interpreter (i.e. runtime init).
(cherry picked from commit 0ba07b2108)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720)
No longer suppress arbitrary errors. Simplify the code.
(cherry picked from commit e1c295e3da)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252)
* Add a special case for s[-m:] == p in _PyBytes_Find
* Add tests for _PyBytes_Find
* Make sure that start <= end in mmap.find
(cherry picked from commit ab86426a34)
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
gh-106403: Restore weakref support for TypeVar and friends (GH-106418)
(cherry picked from commit 945d3cbf2e)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-105227: Add PyType_GetDict() (GH-105747)
This compensates for static builtin types having `tp_dict` set to `NULL`.
(cherry picked from commit a840806d33)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-105340: include hidden fast-locals in locals() (GH-105715)
* gh-105340: include hidden fast-locals in locals()
(cherry picked from commit 104d7b760f)
Co-authored-by: Carl Meyer <carl@oddbird.net>
gh-105486: Change the `repr` of `ParamSpec` list of args in `GenericAlias` (GH-105488)
(cherry picked from commit eb7d6e7ad8)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Fix possible refleak in CodeType.replace() (GH-106243)
A reference to c_code was leaked if PySys_Audit() failed.
(cherry picked from commit 3c70d467c1)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-106033: Get rid of PyDict_GetItem in _PyFunction_FromConstructor (GH-106044)
(cherry picked from commit 08c08d21b0)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
[3.12] gh-106033: Get rid of new occurrences of PyDict_GetItem and PyObject_HasAttr (GH-106034)
These functions are broken by design because they discard any exceptions raised
inside, including MemoryError and KeyboardInterrupt. They should not be
used in new code.
(cherry picked from commit 1d33d53780)
Bail on first error to prevent exceptions from possibly being overwritten.
(cherry picked from commit 555be81026)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
