The clearing of the key, hash, and value need to use atomic operations
to avoid a data race with concurrent read operations.
(cherry picked from commit c00ac57824)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The free-threading build interns and immortalizes most constants
generated by the bytecode compiler. However, users can construct their
own code objects with arbitrary constants. We should not intern or
immortalize these objects if they are not of a type that we know how to
handle.
This change fixes a reference leak failure in the recently added
`test_code.test_unusual_constants` test. It also addresses a potential
crash that could occur when attempting to destroy an immortalized
object during interpreter shutdown.
(cherry picked from commit 12db45211d)
gh-130932: Fix incorrect exception handling in _PyModule_IsPossiblyShadowing (GH-130934)
I chose to not raise an exception here because I think it would be
confusing for module attribute access to start raising something other
than AttributeError if e.g. the cwd goes away
Without the change in moduleobject.c
```
./python.exe -m unittest test.test_import.ImportTests.test_script_shadowing_stdlib_cwd_failure
...
Assertion failed: (PyErr_Occurred()), function _PyObject_SetAttributeErrorContext, file object.c, line 1253.
```
(cherry picked from commit 0a9ae5ed48)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.
The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
(cherry picked from commit 2905690a91)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-130547: Fix race between dict_dealloc and split_keys_entry_added (gh-130778)
(cherry picked from commit 80e6d3ec49)
Co-authored-by: Donghee Na <donghee.na@python.org>
This avoids a case where the interpreter's queue of memory to be freed
could grow rapidly if there are many short lived threads.
(cherry picked from commit 2f6e0e9f70)
Postpone <stdbool.h> inclusion after Python.h (#130641)
Remove inclusions prior to Python.h.
<stdbool.h> will cause <features.h> to be included before Python.h can
define some macros to enable some additional features, causing multiple
types not to be defined down the line.
(cherry picked from commit 830f04b505)
Co-authored-by: Hugo Beauzée-Luyssen <hugo@beauzee.fr>
Fix a few thread-safety bugs to enable test_opcache when run with TSAN:
* Use relaxed atomics when clearing `ht->_spec_cache.getitem`
(gh-115999)
* Add temporary suppression for type slot modifications (gh-127266)
* Use atomic load when reading `*dictptr`
(cherry picked from commit f151d27159)
The use of PySys_GetObject() and _PySys_GetAttr(), which return a borrowed
reference, has been replaced by using one of the following functions, which
return a strong reference and distinguish a missing attribute from an error:
_PySys_GetOptionalAttr(), _PySys_GetOptionalAttrString(),
_PySys_GetRequiredAttr(), and _PySys_GetRequiredAttrString().
(cherry picked from commit 0ef4ffeefd)
* gh-128714: Fix function object races in `__annotate__`, `__annotations__` and `__type_params__` in free-threading build (#129016)
(cherry picked from commit 55f17b77c3)
---------
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
[3.13] gh-128396: Fix a crash when inline comprehension has the same local variable as the outside scope (GH-130235)
(cherry picked from commit ccf17323c2)
We had the definition of what makes a character "printable" documented in three places, giving two different definitions.
The definition in the comment on `_PyUnicode_IsPrintable` was inverted; correct that.
With that correction, the two definitions turn out to be equivalent -- but to confirm that, you have to go look up, or happen to know, that those are the only five "Other" categories and only three "Separator" categories in the Unicode character database. That makes it hard for the reader to tell whether they really are the same, or if there's some subtle difference in the intended semantics.
Fix that by cutting the C API docs' and the C comment's copies of the subtle details, in favor of referring to the Python-level docs. That ensures it's explicit that these are all meant to agree, and also lets us concentrate improvements to the wording in one place.
Speaking of which, borrow some ideas from the C comment, along with other tweaks, to hopefully add a bit more clarity to that one newly-centralized copy in the docs.
Also add a thorough test that the implementation agrees with this definition.
Author: Greg Price <gnprice@gmail.com>
Co-authored-by: Greg Price <gnprice@gmail.com>
(cherry picked from commit 3402e133ef)
We should use a relaxed atomic load in the free threading build in
`PyType_Modified()` because that's called without the type lock held.
It's not necessary to use atomics in `type_modified_unlocked()`.
We should also use `FT_ATOMIC_STORE_UINT_RELAXED()` instead of the
`UINT32` variant because `tp_version_tag` is declared as `unsigned int`.
(cherry picked from commit 57f45ee2d8)
The call to `PySequence_List()` could temporarily unlock and relock the
set, allowing the items to be cleared and return the incorrect
notation `{}` for a empty set (it should be `set()`).
(cherry picked from commit a7427f2db9)
Co-authored-by: T. Wouters <thomas@python.org>
Found while running `test_load_attr_module` from `test_opcache` under TSan.
(cherry picked from commit 34379d0a59)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The MemoryError freelist was not thread-safe in the free threaded build.
Use a mutex to protect accesses to the freelist. Unlike other freelists,
the MemoryError freelist is not performance sensitive.
(cherry picked from commit 51b4edb1a4)
gh-126862: Use `Py_ssize_t` instead of `int` when processing the number of super-classes (GH-127523)
(cherry picked from commit 2fcdc8488c)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-127903: Fix a crash on debug builds when calling `Objects/unicodeobject::_copy_characters`` (GH-127876)
(cherry picked from commit 46cb6340d7)
Co-authored-by: Alexander Shadchin <shadchin@yandex-team.com>
Clean up redundant ifdef in list getitem (GH-128257)
It's already inside a `Py_GIL_DISABLED` block so the `#else` clause is always unused.
(cherry picked from commit 42f7a00ae8)
Co-authored-by: da-woods <dw-git@d-woods.co.uk>
This fixes a UBSan failure (unaligned zero-size memcpy) in `dictobject.c`.
(cherry picked from commit 9af96f4406)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Objects may be temporarily "resurrected" in destructors when calling
finalizers or watcher callbacks. We previously undid the resurrection
by decrementing the reference count using `Py_SET_REFCNT`. This was not
thread-safe because other threads might be accessing the object
(modifying its reference count) if it was exposed by the finalizer,
watcher callback, or temporarily accessed by a racy dictionary or list
access.
This adds internal-only thread-safe functions for temporary object
resurrection during destructors.
(cherry picked from commit f4f530804b)
In the free threading build, if a non-owning thread resizes a list,
it must use QSBR to free the old list array because there may be a
concurrent access (without a lock) from the owning thread.
To match the pattern in dictobject.c, we just mark the list as "shared"
before resizing if it's from a non-owning thread and not already marked
as shared.
(cherry picked from commit c7dec02de2)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Note, that transformed expression is not an equivalent for original one (1/exp(-x) != exp(x) in general for floating-point numbers). Though, the difference seems to be ~1ULP for good libm implementations.
It's more interesting why division was used from beginning. Closest algorithm I've found (no error checks, of course;)) - it's Algorithm 190 from ACM: https://dl.acm.org/doi/10.1145/366663.366679. It uses subtraction in the exponent.
(cherry picked from commit f7bb658124)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Some fields in PyCodeObject are lazily initialized. Use atomics and
critical sections to make their initializations and accesses thread-safe.
(cherry picked from commit 3926842117)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)
Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.
This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
(cherry picked from commit f2cb399470)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.13] gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)
They used to be shared, before 3.12. Returning to sharing them resolves a failure on Py_TRACE_REFS builds.
---------
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-116938: Fix `dict.update` docstring and remove erraneous full stop from `dict` documentation (GH-125421)
(cherry picked from commit 5527c4051c)
Co-authored-by: Prometheus3375 <35541026+Prometheus3375@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Dictionary watchers on an object's attributes dictionary
(`object.__dict__`) were not triggered when the managed dictionary used
the object's inline values.
(cherry picked from commit 5989eb7446)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-125590: Allow FrameLocalsProxy to delete and pop keys from extra locals (GH-125616)
(cherry picked from commit 5b7a872b26)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
gh-125221: Fix free-threading data race in `object.__reduce_ex__` (GH-125267)
(cherry picked from commit b12e99261e)
Co-authored-by: Sam Gross <colesbury@gmail.com>
