[3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805) (GH-137827)

* [3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805)

Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.

(cherry picked from commit 8cc5aa47ee)

Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>

* state "3.13.7 and earlier"
* backport: do not add the deprecated marker
* fix Py_IsFinalizing doc ref

---------

Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>

Include/internal/pycore_pythread.h

@@ -152,6 +152,19 @@ PyAPI_FUNC(int) PyThread_join_thread(PyThread_handle_t);
  * a non-zero value on failure.
  */
 PyAPI_FUNC(int) PyThread_detach_thread(PyThread_handle_t);
+/*
+ * Hangs the thread indefinitely without exiting it.
+ *
+ * gh-87135: There is no safe way to exit a thread other than returning
+ * normally from its start function.  This is used during finalization in lieu
+ * of actually exiting the thread.  Since the program is expected to terminate
+ * soon anyway, it does not matter if the thread stack stays around until then.
+ *
+ * This is unfortunate for embedders who may not be terminating their process
+ * when they're done with the interpreter, but our C API design does not allow
+ * for safely exiting threads attempting to re-enter Python post finalization.
+ */
+void _Py_NO_RETURN PyThread_hang_thread(void);
 
 #ifdef __cplusplus
 }

Include/pythread.h

@@ -17,7 +17,26 @@ typedef enum PyLockStatus {
 
 PyAPI_FUNC(void) PyThread_init_thread(void);
 PyAPI_FUNC(unsigned long) PyThread_start_new_thread(void (*)(void *), void *);
+/* Terminates the current thread. Considered unsafe.
+ *
+ * WARNING: This function is only safe to call if all functions in the full call
+ * stack are written to safely allow it.  Additionally, the behavior is
+ * platform-dependent.  This function should be avoided, and is no longer called
+ * by Python itself.  It is retained only for compatibility with existing C
+ * extension code.
+ *
+ * With pthreads, calls `pthread_exit` causes some libcs (glibc?) to attempt to
+ * unwind the stack and call C++ destructors; if a `noexcept` function is
+ * reached, they may terminate the process. Others (macOS) do unwinding.
+ *
+ * On Windows, calls `_endthreadex` which kills the thread without calling C++
+ * destructors.
+ *
+ * In either case there is a risk of invalid references remaining to data on the
+ * thread stack.  This is deprecated in 3.14 onwards.  Retained for API compat.
+ */
 PyAPI_FUNC(void) _Py_NO_RETURN PyThread_exit_thread(void);
+
 PyAPI_FUNC(unsigned long) PyThread_get_thread_ident(void);
 
 #if (defined(__APPLE__) || defined(__linux__) || defined(_WIN32) \