Commit graph

685 commits

Author SHA1 Message Date
Sam Gross
9966daee60
[3.13] gh-117657: Enable test_opcache under TSAN (GH-129831) (GH-130597)
Fix a few thread-safety bugs to enable test_opcache when run with TSAN:

 * Use relaxed atomics when clearing `ht->_spec_cache.getitem`
   (gh-115999)
 * Add temporary suppression for type slot modifications (gh-127266)
 * Use atomic load when reading `*dictptr`

(cherry picked from commit f151d27159)
2025-02-26 13:59:59 -05:00
Miss Islington (bot)
052632279d
[3.13] gh-128100: Use atomic dictionary load in _PyObject_GenericGetAttrWithDict (GH-128297) (GH-129979)
(cherry picked from commit 47d2cb8eb7)

Co-authored-by: Bogdan Romanyuk <65823030+wrongnull@users.noreply.github.com>
2025-02-10 20:48:24 +00:00
Pablo Galindo Salgado
eb692d945e
[3.13] gh-126076: Account for relocated objects in tracemalloc (GH-126077) (#127823)
(cherry picked from commit 30aeb00d36)
2024-12-11 14:15:37 +01:00
Sam Gross
69bb1c6c41
[3.13] gh-127582: Make object resurrection thread-safe for free threading. (GH-127612) (GH-127659)
Objects may be temporarily "resurrected" in destructors when calling
finalizers or watcher callbacks. We previously undid the resurrection
by decrementing the reference count using `Py_SET_REFCNT`. This was not
thread-safe because other threads might be accessing the object
(modifying its reference count) if it was exposed by the finalizer,
watcher callback, or temporarily accessed by a racy dictionary or list
access.

This adds internal-only thread-safe functions for temporary object
resurrection during destructors.
(cherry picked from commit f4f530804b)
2024-12-05 18:49:33 -05:00
Miss Islington (bot)
02cd3ce0f2
[3.13] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865) (gh-125709) (GH-125204)
* gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)

Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.

This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
(cherry picked from commit f2cb399470)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>

* [3.13] gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)

They used to be shared, before 3.12.  Returning to sharing them resolves a failure on Py_TRACE_REFS builds.

---------

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-11-12 13:45:12 +01:00
Miss Islington (bot)
b3badabcd9
[3.13] gh-125221: Fix free-threading data race in object.__reduce_ex__ (GH-125267) (#125305)
gh-125221: Fix free-threading data race in `object.__reduce_ex__` (GH-125267)
(cherry picked from commit b12e99261e)

Co-authored-by: Sam Gross <colesbury@gmail.com>
2024-10-11 08:26:23 +00:00
Miss Islington (bot)
bd29ce8509
[3.13] gh-123448: Move _PyNoDefault_Type to the static types array (GH-123449) (#123450)
(cherry picked from commit c9930f5022)

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2024-08-28 18:57:28 -07:00
Miss Islington (bot)
5212624c42
[3.13] gh-116622: Fix testPyObjectPrintOSError on Android (GH-122487) (#122490)
gh-116622: Fix testPyObjectPrintOSError on Android (GH-122487)

Adds extra handling for way BSD/Android return errors from calls to fwrite.
(cherry picked from commit 82db572813)

Co-authored-by: Malcolm Smith <smith@chaquo.com>
2024-07-31 00:49:14 +00:00
Miss Islington (bot)
0794220a69
[3.13] gh-121794: Don't set ob_tid to zero in fast-path dealloc (GH-121799) (#121821)
We should maintain the invariant that a zero `ob_tid` implies the
refcount fields are merged.

* Move the assignment in `_Py_MergeZeroLocalRefcount` to immediately
  before the refcount merge.
* Update `_PyTrash_thread_destroy_chain` to set `ob_ref_shared` to
  `_Py_REF_MERGED` when setting `ob_tid` to zero.

Also check this invariant with assertions in the GC in debug builds.
That uncovered a bug when running out of memory during GC.
(cherry picked from commit d23be3947c)

Co-authored-by: Sam Gross <colesbury@gmail.com>
2024-07-15 22:15:23 +00:00
Petr Viktorin
9769b7ae06
[3.13] gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520) (GH-120945)
* Add an InternalDocs file describing how interning should work and how to use it.

* Add internal functions to *explicitly* request what kind of interning is done:
  - `_PyUnicode_InternMortal`
  - `_PyUnicode_InternImmortal`
  - `_PyUnicode_InternStatic`

* Switch uses of `PyUnicode_InternInPlace` to those.

* Disallow using `_Py_SetImmortal` on strings directly.
  You should use `_PyUnicode_InternImmortal` instead:
  - Strings should be interned before immortalization, otherwise you're possibly
    interning a immortalizing copy.
  - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
    `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
    backports, as they are now part of public API and version-specific ABI.

* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
  - `_Py_ID`
  - `_Py_STR` (including the empty string)
  - one-character latin-1 singletons

  Now, when you intern a singleton, that exact singleton will be interned.

* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

* Intern `_Py_STR` singletons at startup.

* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.

* Beef up the tests. Cover internal details (marked with `@cpython_only`).

* Add lots of assertions

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-06-24 20:24:19 +02:00
Sam Gross
3cb6c4cd60
[3.13] gh-117511: Make PyMutex public in the non-limited API (GH-117731) (#120800)
(cherry picked from commit 3af7263037)
2024-06-20 16:00:25 +00:00
Miss Islington (bot)
327a36a4f9
[3.13] gh-119999: Fix potential race condition in _Py_ExplicitMergeRefcount (GH-120000) (#120073)
We need to write to `ob_ref_local` and `ob_tid` before `ob_ref_shared`.
Once we mark `ob_ref_shared` as merged, some other thread may free the
object because the caller also passes in `-1` as `extra` to give up its
only reference.
(cherry picked from commit 4055577221)

Co-authored-by: Sam Gross <colesbury@gmail.com>
2024-06-04 19:57:18 +00:00
Miss Islington (bot)
e5fb3a2385
[3.13] gh-117398: Use Per-Interpreter State for the _datetime Static Types (gh-120009)
We make use of the same mechanism that we use for the static builtin types.  This required a few tweaks.

This change is the final piece needed to make _datetime support multiple interpreters.  I've updated the module slot accordingly.

(cherry picked from commit 105f22ea46, AKA gh-119929)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-06-03 23:37:28 +00:00
Sam Gross
ae705319fc
[3.13] gh-117657: Fix race involving immortalizing objects (GH-119927) (#120005)
The free-threaded build currently immortalizes objects that use deferred
reference counting (see gh-117783). This typically happens once the
first non-main thread is created, but the behavior can be suppressed for
tests, in subinterpreters, or during a compile() call.

This fixes a race condition involving the tracking of whether the
behavior is suppressed.

(cherry picked from commit 47fb4327b5)
2024-06-03 22:21:32 +00:00
Miss Islington (bot)
7b9ca26812
[3.13] gh-117657: Fix data races when writing / reading ob_gc_bits (GH-118292) (#118796)
Use relaxed atomics when reading / writing to the field. There are still a
few places in the GC where we do not use atomics. Those should be safe as
the world is stopped.
(cherry picked from commit cb6f75a32c)

Co-authored-by: mpage <mpage@meta.com>
2024-05-08 21:31:37 +00:00
Dino Viehland
636b8d94c9
gh-112075: Fix race in constructing dict for instance (#118499) 2024-05-06 23:31:09 +00:00
Dino Viehland
5a1618a2c8
gh-118362: Fix thread safety around lookups from the type cache in the face of concurrent mutators (#118454)
Add _PyType_LookupRef and use incref before setting attribute on type
Makes setting an attribute on a class and signaling type modified atomic
Avoid adding re-entrancy exposing the type cache in an inconsistent state by decrefing after type is updated
2024-05-06 10:50:35 -07:00
Tian Gao
b034f14a4b
gh-74929: Implement PEP 667 (GH-115153) 2024-05-04 12:12:10 +01:00
Pablo Galindo Salgado
6bcbee09df
gh-93502: Add new C-API functions to trace object creation and destruction (#115945) 2024-05-02 19:30:00 +02:00
Guido van Rossum
7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Sam Gross
7ccacb220d
gh-117783: Immortalize objects that use deferred reference counting (#118112)
Deferred reference counting is not fully implemented yet. As a temporary
measure, we immortalize objects that would use deferred reference
counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first
non-main thread is started. Additionally, some tests, including refleak
tests, suppress this behavior.
2024-04-29 14:36:02 -04:00
Dino Viehland
8b541c017e
gh-112075: Make instance attributes stored in inline "dict" thread safe (#114742)
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
Dino Viehland
b45af00bad
[gh-117657] _Py_MergeZeroLocalRefcount isn't loading ob_ref_shared with strong enough semantics (#118111)
Use acquire for load of ob_ref_shared
2024-04-19 14:21:01 -07:00
Irit Katriel
c179c0e6cb
gh-117680: make _PyInstructionSequence a PyObject and use it in tests (#117629) 2024-04-17 16:42:04 +01:00
Mark Shannon
147cd0581e
GH-117760: Streamline the trashcan mechanism (GH-117763) 2024-04-17 11:08:05 +01:00
Sam Gross
4ad8f090cc
gh-117376: Partial implementation of deferred reference counting (#117696)
This marks objects as using deferred refrence counting using the
`ob_gc_bits` field in the free-threaded build and collects those objects
during GC.
2024-04-12 17:36:20 +00:00
Serhiy Storchaka
2d3d9b4461
gh-117764: Add docstrings and signatures for the types of None, Ellipsis and NotImplemented (GH-117813) 2024-04-12 15:45:23 +03:00
Sam Gross
1a6594f661
gh-117439: Make refleak checking thread-safe without the GIL (#117469)
This keeps track of the per-thread total reference count operations in
PyThreadState in the free-threaded builds. The count is merged into the
interpreter's total when the thread exits.
2024-04-08 12:11:36 -04:00
Mark Shannon
c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
Eric Snow
617158e078
gh-76785: Drop PyInterpreterID_Type (gh-117101)
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation.  Relatively recently I refactored that implementation to no longer rely on InterpreterID objects.  Thus now I'm removing it.
2024-03-21 17:15:02 +00:00
Victor Stinner
8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Mark Shannon
15309329b6
GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Sam Gross
347acded84
gh-115491: Keep some fields valid across allocations in obmalloc (free-threading) (#115745) 2024-02-21 10:00:08 -08:00
Donghee Na
f15795c9a0
gh-111968: Rename freelist related struct names to Eric's suggestion (gh-115329) 2024-02-14 00:32:51 +00:00
Donghee Na
d4d5bae147
gh-111968: Refactor _PyXXX_Fini to integrate with _PyObject_ClearFreeLists (gh-114899) 2024-02-10 00:57:04 +00:00
Sam Gross
a3af3cb4f4
gh-110481: Implement inter-thread queue for biased reference counting (#114824)
Biased reference counting maintains two refcount fields in each object:
`ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two
fields. In some cases, when refcounting operations are split across threads,
the ob_ref_shared field can be negative (although the total refcount must be
at least zero). In this case, the thread that decremented the refcount
requests that the owning thread give up ownership and merge the refcount
fields.
2024-02-09 17:08:32 -05:00
Sam Gross
31633f4473
gh-115184: Fix refleak tracking issues in free-threaded build (#115188)
Fixes a few issues related to refleak tracking in the free-threaded build:

- Count blocks in abandoned segments
- Call `_mi_page_free_collect` earlier during heap traversal in order to get an accurate count of blocks in use.
- Add missing refcount tracking in `_Py_DecRefSharedDebug` and `_Py_ExplicitMergeRefcount`.
- Pause threads in  `get_num_global_allocated_blocks` to ensure that traversing the mimalloc heaps is safe.
2024-02-09 09:23:12 -05:00
Mark Shannon
8a3c499ffe
GH-108362: Revert "GH-108362: Incremental GC implementation (GH-108038)" (#115132)
Revert "GH-108362: Incremental GC implementation (GH-108038)"

This reverts commit 36518e69d7.
2024-02-07 12:38:34 +00:00
Mark Shannon
36518e69d7
GH-108362: Incremental GC implementation (GH-108038) 2024-02-05 18:28:51 +00:00
Sam Gross
587d480203
gh-112529: Remove PyGC_Head from object pre-header in free-threaded build (#114564)
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build

This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.

 * The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
   in the free-threaded build.
 * The GDB libpython.py file now determines the offset of the managed
   dict field based on whether the running process is a free-threaded
   build. Those are identified by the `ob_ref_local` field in PyObject.
 * Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
   size of `PyGC_Head` in the size of static `PyTypeObject`.
2024-02-01 12:29:19 -08:00
Sam Gross
d0f0308a37
gh-113750: Fix object resurrection in free-threaded builds (gh-113751)
gh-113750: Fix object resurrection on free-threaded builds

This avoids the undesired re-initializing of fields like `ob_gc_bits`,
`ob_mutex`, and `ob_tid` when an object is resurrected due to its
finalizer being called.

This change has no effect on the default (with GIL) build.
2024-01-06 12:12:26 +09:00
Christopher Chavez
a545a86ec6
gh-111178: Make slot functions in typeobject.c have compatible types (GH-112752) 2023-12-20 15:13:44 +01:00
andrewluotechnologies
9c3458e058
gh-112125: Fix None.__ne__(None) returning NotImplemented instead of False (#112504) 2023-12-07 13:56:01 +01:00
Hugo van Kemenade
3b3ec0d77f
gh-111863: Rename Py_NOGIL to Py_GIL_DISABLED (#111864)
Rename Py_NOGIL to Py_GIL_DISABLED
2023-11-20 15:52:00 +02:00
Serhiy Storchaka
18203a6bc9
gh-111789: Use PyDict_GetItemRef() in Objects/ (GH-111827) 2023-11-14 11:25:39 +02:00
Sam Gross
fa84e5fe0a
gh-110481: fix 'unused function' warning for is_shared_refcnt_dead. (gh-111974)
Fix 'unused function' warning for `is_shared_refcnt_dead`.

The `is_shared_refcnt_dead` function is only used if `Py_REF_DEBUG` is set.
2023-11-11 08:54:35 +09:00
Sam Gross
31c90d5838
gh-111569: Implement Python critical section API (gh-111571)
Critical sections are helpers to replace the global interpreter lock
with finer grained locking.  They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves.  Critical
sections are implicitly ended whenever the GIL would be released.  They
are resumed when the GIL would be acquired.  Nested critical sections
behave as if the sections were interleaved.
2023-11-08 15:39:29 -07:00
Serhiy Storchaka
f55cb44359
gh-106672: C API: Report indiscriminately ignored errors (GH-106674)
Functions which indiscriminately ignore all errors now report them as
unraisable errors.
2023-11-07 15:58:04 +02:00
Victor Stinner
20cfab903d
gh-111506: Implement Py_SET_REFCNT() as opaque function in limited C API (#111508)
In the limited C API version 3.13, Py_SET_REFCNT() function is now
implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
2023-11-03 18:18:57 +01:00
Sam Gross
6dfb8fe023
gh-110481: Implement biased reference counting (gh-110764) 2023-10-30 16:06:09 +00:00