We do the following:
* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()
The per-interpreter registry helps preserve isolation between interpreters. This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.
(cherry-picked from commit 80dc39e1dc)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.
We add PyInterpreterState.threads.main, with some internal C-API functions.
(cherry-picked from commit 1dd9dee45d)
We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.
Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
(cherry-picked from commit b72947a8d2)
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all.
This change does not address the broader concerns from gh-109853.
(cherry-picked from commit a040a32ea2)
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.
thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.
Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
(cherry picked from commit 517cd82ea7)
* gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808)
(cherry picked from commit 1b8f2366b3)
---------
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
gh-110052: Fix faulthandler for freed tstate (#110069)
faulthandler now detected freed interp and freed tstate, and no
longer dereference them.
(cherry picked from commit 2e37a38bcb)
gh-109889: fix compiler's redundant NOP detection to look past NOPs with no lineno when looking for the next instruction's lineno (GH-109987)
(cherry picked from commit f580edcc6a)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-109521: Fix obscure cases handling in PyImport_GetImporter() (GH-109522)
PyImport_GetImporter() now sets RuntimeError if it fails to get sys.path_hooks
or sys.path_importer_cache or they are not list and dict correspondingly.
Previously it could return NULL without setting error in obscure cases,
crash or raise SystemError if these attributes have wrong type.
(cherry picked from commit 62c7015e89)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
(cherry picked from commit 87a7faf6b6)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-104690: thread_run() checks for tstate dangling pointer (#109056)
thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.
Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
(cherry picked from commit f63d37877a)
* [3.12] gh-109889: comment out assertion indicating a failed optimization of a redundant NOP
* comment out the function to avoid warnings on it being unused
gh-109823: Adjust labels in compiler when removing an empty basic block which is a jump target (GH-109839)
(cherry picked from commit d73c12b88c)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-109219: propagate free vars through type param scopes (GH-109377)
(cherry picked from commit 909adb5092)
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-109195: fix source location for super load before LOAD_SUPER_ATTR (GH-109289)
(cherry picked from commit ceeb4173ae)
Co-authored-by: Carl Meyer <carl@oddbird.net>
* gh-109118: Fix runtime crash when NameError happens in PEP 695 function (#109123)
(cherry picked from commit 17f994174d)
* [3.12] gh-109118: Fix runtime crash when NameError happens in PEP 695 function (GH-109123).
(cherry picked from commit 17f994174d)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* GH-108390: Prevent non-local events being set with `sys.monitoring.set_local_events()` (GH-108420)
* Restore generated objects
* Restore size of monitoring arrays in code object for 3.12 ABI compatibility.
* Update ABI file
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
(cherry picked from commit 2b15536fa9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Remove assert that should've been DEOPT_IF
The assert(method != NULL) in CALL_NO_KW_LIST_APPEND is wrong --
this condition should lead to a deoptimization, and indeed there
is a DEOPT_IF two lines later that will trigger if method == NULL.
This would crash in a devious repro scenario (first seen live
in boto3 tests) when compiled with assertions enabled.
In a production version there is no crash, so impact is limited.
(The crash also appears in main; I will prepare a separate PR.)
* Add back a different assert(self != NULL)
* 📜🤖 Added by blurb_it.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
gh-107901: Fix missing line number on BACKWARD_JUMP at the end of a for loop (GH-108242)
(cherry picked from commit a1cc74c4ee)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929)
(cherry picked from commit 80bdebdd85)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
(cherry picked from commit 633ea217a8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Define PY_SSIZE_T_CLEAN.
---------
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
