mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-12-04 00:30:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)

Browse source 
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules.  The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL.  It turns out it's tricky to use the same thread state for different threads.  Who could have guessed?

We solve the problem by eliminating the one object we were still sharing between interpreters.  We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.

We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
This commit is contained in:
Eric Snow 2023-07-28 14:39:08 -06:00 • committed by GitHub
parent 55ed85e49c
commit 8ba4df91ae
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 129 additions and 191 deletions
Download patch file Download diff file Expand all files Collapse all files

11
Include/internal/pycore_import.h
View file

@ -5,6 +5,7 @@
extern "C" {
#endif
#include "pycore_hashtable.h" // _Py_hashtable_t
#include "pycore_time.h" // _PyTime_t
extern int _PyImport_IsInitialized(PyInterpreterState *);
@ -36,19 +37,15 @@ struct _import_runtime_state {
See PyInterpreterState.modules_by_index for more info. */
Py_ssize_t last_module_index;
struct {
/* A thread state tied to the main interpreter,
used exclusively for when the extensions dict is access/modified
from an arbitrary thread. */
PyThreadState main_tstate;
/* A lock to guard the dict. */
/* A lock to guard the cache. */
PyThread_type_lock mutex;
/* A dict mapping (filename, name) to PyModuleDef for modules.
/* The actual cache of (filename, name, PyModuleDef) for modules.
Only legacy (single-phase init) extension modules are added
and only if they support multiple initialization (m_size >- 0)
or are imported in the main interpreter.
This is initialized lazily in _PyImport_FixupExtensionObject().
Modules are added there and looked up in _imp.find_extension(). */
PyObject *dict;
_Py_hashtable_t *hashtable;
} extensions;
/* Package context -- the full module name for package imports */
const char * pkgcontext;