[3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805) (GH-137827)
Some checks are pending
Tests / Windows MSI (push) Blocked by required conditions
Tests / Hypothesis tests on Ubuntu (push) Blocked by required conditions
Tests / (push) Blocked by required conditions
Tests / Change detection (push) Waiting to run
Tests / Docs (push) Blocked by required conditions
Tests / Check if the ABI has changed (push) Blocked by required conditions
Tests / Check if Autoconf files are up to date (push) Blocked by required conditions
Tests / Check if generated files are up to date (push) Blocked by required conditions
Tests / Android (x86_64) (push) Blocked by required conditions
Tests / Ubuntu SSL tests with OpenSSL (push) Blocked by required conditions
Tests / Android (aarch64) (push) Blocked by required conditions
Tests / WASI (push) Blocked by required conditions
Tests / Address sanitizer (push) Blocked by required conditions
Tests / Sanitizers (push) Blocked by required conditions
Tests / CIFuzz (push) Blocked by required conditions
Tests / All required checks pass (push) Blocked by required conditions
Lint / lint (push) Waiting to run

* [3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805)

Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.

(cherry picked from commit 8cc5aa47ee)

Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>

* state "3.13.7 and earlier"
* backport: do not add the deprecated marker
* fix Py_IsFinalizing doc ref

---------

Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
This commit is contained in:
Gregory P. Smith 2025-08-17 09:08:14 -07:00 committed by GitHub
parent f2b7954ce0
commit 9face218e7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 247 additions and 28 deletions

View file

@ -3322,6 +3322,34 @@ test_atexit(PyObject *self, PyObject *Py_UNUSED(args))
Py_RETURN_NONE;
}
// Used by `finalize_thread_hang`.
#ifdef _POSIX_THREADS
static void finalize_thread_hang_cleanup_callback(void *Py_UNUSED(arg)) {
// Should not reach here.
Py_FatalError("pthread thread termination was triggered unexpectedly");
}
#endif
// Tests that finalization does not trigger pthread cleanup.
//
// Must be called with a single nullary callable function that should block
// (with GIL released) until finalization is in progress.
static PyObject *
finalize_thread_hang(PyObject *self, PyObject *callback)
{
// WASI builds some pthread stuff but doesn't have these APIs today?
#if defined(_POSIX_THREADS) && !defined(__wasi__)
pthread_cleanup_push(finalize_thread_hang_cleanup_callback, NULL);
#endif
PyObject_CallNoArgs(callback);
// Should not reach here.
Py_FatalError("thread unexpectedly did not hang");
#if defined(_POSIX_THREADS) && !defined(__wasi__)
pthread_cleanup_pop(0);
#endif
Py_RETURN_NONE;
}
static void
tracemalloc_track_race_thread(void *data)
@ -3590,6 +3618,7 @@ static PyMethodDef TestMethods[] = {
{"test_atexit", test_atexit, METH_NOARGS},
{"tracemalloc_track_race", tracemalloc_track_race, METH_NOARGS},
{"toggle_reftrace_printer", toggle_reftrace_printer, METH_O},
{"finalize_thread_hang", finalize_thread_hang, METH_O, NULL},
{NULL, NULL} /* sentinel */
};