_PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed.
This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple.
This change also fixes a bug where only the most recent parser was added to the global linked list.
(cherry picked from commit 81865002ae)
gh-118513: Fix sibling comprehensions with a name bound in one and global in the other (GH-118526)
(cherry picked from commit c8deb1e4b4)
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
(cherry picked from commit c1bf4874c1)
* document equivalent command-line options for all environment variables
* document equivalent environment variables for all command-line options
* reduce the size of variable and option descriptions to minimum
* remove the ending period in single-sentence descriptions
(cherry picked from commit b85572c47d)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Fix the description of the "-b" option.
* Add references to environment variables for "-s" and "-X dev" options.
(cherry picked from commit 33662d4e01)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-116447: Fix possible UB in `arraymodule` and `getargs` (GH-116459)
(cherry picked from commit fdb2d90a27)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-115320: Refactor `get_hash_info` in `sysmodule.c` not to swallow errors (GH-115321)
(cherry picked from commit 207030f552)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Setters for members with an unsigned integer type now support
the same range of valid values for objects that has a __index__()
method as for int.
Previously, Py_T_UINT, Py_T_ULONG and Py_T_ULLONG did not support
objects that has a __index__() method larger than LONG_MAX.
Py_T_ULLONG did not support negative ints. Now it supports them and
emits a RuntimeWarning.
(cherry picked from commit d9d6909697)
* Fix a RuntimeWarning emitted when assign an integer-like value that
is not an instance of int to an attribute that corresponds to a C
struct member of type T_UINT and T_ULONG.
* Fix a double RuntimeWarning emitted when assign a negative integer value
to an attribute that corresponds to a C struct member of type T_UINT.
(cherry picked from commit 3ddc515255)
(cherry picked from commit 48c49739f5)
Co-authored-by: Yilei Yang <yileiyang@google.com>
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
gh-113343: Fix error check on mmap(2) (GH-113342)
Fix error check on mmap(2)
It should check MAP_FAILED instead of NULL for error.
On mmap(2) man page:
RETURN VALUE
On success, mmap() returns a pointer to the mapped area.
On error, the value MAP_FAILED (that is, (void *) -1) is
returned, and errno is set to indicate the error.
(cherry picked from commit 6b70c3dc5a)
Co-authored-by: Namhyung Kim <namhyung@gmail.com>
It was raised in two cases:
* in the import statement when looking up __import__
* in pickling some builtin type when looking up built-ins iter, getattr, etc.
(cherry picked from commit 1161c14e8c)
gh-106560: Fix redundant declarations in Python/frozen.c (#112612)
Avoid duplicated declarations of "extern" functions in
Python/frozen.c.
Compiler warnings seen by building Python with gcc -Wredundant-decls.
(cherry picked from commit d9e444dbb8)
We do the following:
* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()
The per-interpreter registry helps preserve isolation between interpreters. This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.
(cherry-picked from commit 80dc39e1dc)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.
We add PyInterpreterState.threads.main, with some internal C-API functions.
(cherry-picked from commit 1dd9dee45d)
We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.
Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
(cherry-picked from commit b72947a8d2)
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all.
This change does not address the broader concerns from gh-109853.
(cherry-picked from commit a040a32ea2)
