Commit graph

9344 commits

Author SHA1 Message Date
Victor Stinner
36c5e3bcc2
gh-128679: Redesign tracemalloc locking (#128888)
* Use TABLES_LOCK() to protect 'tracemalloc_config.tracing'.
* Hold TABLES_LOCK() longer while accessing tables.
* tracemalloc_realloc() and tracemalloc_free() no longer
  remove the trace on reentrant call.
* _PyTraceMalloc_Stop() unregisters _PyTraceMalloc_TraceRef().
* _PyTraceMalloc_GetTraces() sets the reentrant flag.
* tracemalloc_clear_traces_unlocked() sets the reentrant flag.
2025-01-15 20:22:44 +00:00
Neil Schemenauer
080f444a58
gh-128807: Add marking phase for free-threaded cyclic GC (gh-128808) 2025-01-15 11:27:28 -08:00
mpage
b5ee0258bf
gh-115999: Specialize LOAD_ATTR for instance and class receivers in free-threaded builds (#128164)
Finish specialization for LOAD_ATTR in the free-threaded build by adding support for class and instance receivers.
2025-01-14 11:56:11 -08:00
Neil Schemenauer
1c13c56a34
gh-128384: Add locking to warnings.py. (gh-128386)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-01-14 11:43:42 -08:00
Mark Shannon
f49a1df6f3
GH-128682: Convert explicit loops closing arrays into DECREF_INPUTS. (GH-128822)
* Mark Py_DECREF and Py_XDECREF as escaping

* Remove explicit loops for clearing array inputs
2025-01-14 15:08:56 +00:00
Peter Bierma
bf64a582f0
gh-128400: Only show the current thread in Py_FatalError on the free-threaded build (#128758) 2025-01-13 20:06:54 +05:30
Mark Shannon
517dc65ffc
GH-128682: Stronger checking of PyStackRef_CLOSE and DEAD. (GH-128683) 2025-01-13 12:37:48 +00:00
Mark Shannon
ddd959987c
GH-128685: Specialize (rather than quicken) LOAD_CONST into LOAD_CONST_[IM]MORTAL (GH-128708) 2025-01-13 10:30:28 +00:00
Hood Chatham
d0ecbdd838
gh-128627: Emscripten: Use wasm-gc based call adaptor if available (#128628)
Replaces the trampoline mechanism in Emscripten with an implementation that uses a
recently added feature of wasm-gc instead of JS type reflection, when that feature is
available.
2025-01-13 07:09:39 +08:00
Peter Bierma
f6c61bf2d7
gh-128717: Stop-the-world when setting the recursion limit (#128741) 2025-01-12 18:34:30 +05:30
9cel
3a570c6d58
Make the Python CLI error message style more consistent (GH-128129) 2025-01-11 11:17:35 +02:00
Bénédikt Tran
28ffdc5702
gh-111178: fix UBSan failures in Python/bltinmodule.c (GH-128235)
* fix UBSan failures for `filterobject`
* fix UBSan failures for `mapobject`
* fix UBSan failures for `zipobject`
2025-01-10 11:51:24 +01:00
Petr Viktorin
1439b81928
gh-128629: Add Py_PACK_VERSION and Py_PACK_FULL_VERSION (GH-128630) 2025-01-09 11:10:28 +01:00
Brandt Bucher
004f9fd1f2
Remove unnecessary LIST_TO_TUPLE conversions (GH-126558) 2025-01-08 09:00:11 -08:00
Bénédikt Tran
1da0901894
gh-111178: fix UBSan failures in Python/context.c (GH-128242)
* fix UBSan failures for `PyContext`
* fix UBSan failures for `PyContextVar`
* fix UBSan failures for `PyContextToken`
* fix UBSan failures for `_PyContextTokenMissing`
2025-01-08 14:52:27 +01:00
Bénédikt Tran
c22302ecea
gh-111178: fix UBSan failures in Python/hamt.c (GH-128247)
* fix UBSan failures for `PyHamtObject`
* fix UBSan failures for `PyHamtNode_Array`
* fix UBSan failures for `PyHamtNode_Collision`
* fix UBSan failures for `PyHamtNode_Bitmap`
2025-01-08 14:50:40 +01:00
Brandt Bucher
65ae3d5a73
GH-127809: Fix the JIT's understanding of ** (GH-127844) 2025-01-07 17:25:48 -08:00
T. Wouters
8f93dd8a8f
gh-115999: Add free-threaded specialization for COMPARE_OP (#126410)
Add free-threaded specialization for COMPARE_OP, and tests for COMPARE_OP specialization in general.

Co-authored-by: Donghee Na <donghee.na92@gmail.com>
2025-01-07 06:41:01 -08:00
Russell Keith-Magee
e837a1f71e
gh-128146: Exclude os/log.h import on older macOS versions. (#128165)
Reworks the handling of Apple system log handling to account for older macOS 
versions that don't provide os-log.
2025-01-07 13:12:58 +08:00
Mark Shannon
2434fd2d50
GH-128533: Add NOT_TAKEN instruction after bytecode optimization. (GH-128554) 2025-01-06 22:01:07 +00:00
Mark Shannon
f826beca0c
GH-128375: Better instrument for FOR_ITER (GH-128445) 2025-01-06 17:54:47 +00:00
Victor Stinner
f89e5e20cb
gh-127350: Add Py_fopen() and Py_fclose() functions (#127821) 2025-01-06 12:43:09 +00:00
Anders Kaseorg
a626f9a67b
Remove asserts that confuse enum _framestate with enum _frameowner (GH-124148)
The `owner` field of `_PyInterpreterFrame` is supposed to be a member of
`enum _frameowner`, but `FRAME_CLEARED` is a member of `enum _framestate`.

At present, it happens that `FRAME_CLEARED` is not numerically equal to any
member of `enum _frameowner`, but that could change in the future. The code
that incorrectly assigned `owner = FRAME_CLEARED` was deleted in commit
a53cc3f494 (GH-116687). Remove the incorrect
checks for `owner != FRAME_CLEARED` as well.
2025-01-02 16:55:33 +00:00
Kumar Aditya
e389d6c650
gh-128277: make globals variables thread safe in socket module (#128286) 2024-12-31 19:10:06 +05:30
Ken Jin
7ef4907412
gh-128262: Allow specialization of calls to classes with __slots__ (GH-128263) 2024-12-31 12:24:17 +08:00
Yan Yanchii
fe4dd07a84
gh-119786: Mention InternalDocs/interpreter.md instead of non-existing adaptive.md (#128329)
`Python/specialize.c`: Mention `InternalDocs/interpreter.md` instead of non-existing `adaptive.md`


Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2024-12-30 18:38:09 +00:00
Jelle Zijlstra
3480124321
gh-119180: Set the name of the param to __annotate__ to "format" (#124730) 2024-12-30 08:19:38 -08:00
Yan Yanchii
30efede33c
gh-128195: Add _REPLACE_WITH_TRUE to the tier2 optimizer (GH-128203)
Add `_REPLACE_WITH_TRUE` to the tier2 optimizer
2024-12-24 05:17:47 +08:00
T. Wouters
180d417e9f
gh-114203: Optimise simple recursive critical sections (#128126)
Add a fast path to (single-mutex) critical section locking _iff_ the mutex
is already held by the currently active, top-most critical section of this
thread. This can matter a lot for indirectly recursive critical sections
without intervening critical sections.
2024-12-23 13:31:33 +01:00
Nico-Posada
3879ca0100
gh-128049: Fix type confusion bug with the return value of a custom ExceptionGroup split function (#128079) 2024-12-20 19:20:31 +00:00
Mark Shannon
128cc47fbd
GH-127705: Add debug mode for _PyStackRefs inspired by HPy debug mode (GH-128121) 2024-12-20 16:52:20 +00:00
Shantanu
45e6dd63b8
gh-128030: Avoid error from PyModule_GetFilenameObject for non-module (#128047)
I missed the extra `PyModule_Check` in #127660 because I was looking at
3.12 as the base implementation for import from. This meant that I
missed the `PyModuleCheck` introduced in #112661.
2024-12-20 00:22:26 -08:00
mpage
255762c09f
gh-127274: Defer nested methods (#128012)
Methods (functions defined in class scope) are likely to be cleaned
up by the GC anyway.

Add a new code flag, `CO_METHOD`, that is set for functions defined
in a class scope. Use that when deciding to defer functions.
2024-12-19 13:03:14 -08:00
Neil Schemenauer
1b15c89a17
gh-115999: Specialize STORE_ATTR in free-threaded builds. (gh-127838)
* Add `_PyDictKeys_StringLookupSplit` which does locking on dict keys and
  use in place of `_PyDictKeys_StringLookup`.

* Change `_PyObject_TryGetInstanceAttribute` to use that function
  in the case of split keys.

* Add `unicodekeys_lookup_split` helper which allows code sharing
  between `_Py_dict_lookup` and `_PyDictKeys_StringLookupSplit`.

* Fix locking for `STORE_ATTR_INSTANCE_VALUE`.  Create
  `_GUARD_TYPE_VERSION_AND_LOCK` uop so that object stays locked and
  `tp_version_tag` cannot change.

* Pass `tp_version_tag` to `specialize_dict_access()`, ensuring
  the version we store on the cache is the correct one (in case of
  it changing during the specalize analysis).

* Split `analyze_descriptor` into `analyze_descriptor_load` and
  `analyze_descriptor_store` since those don't share much logic.
  Add `descriptor_is_class` helper function.

* In `specialize_dict_access`, double check `_PyObject_GetManagedDict()`
  in case we race and dict was materialized before the lock.

* Avoid borrowed references in `_Py_Specialize_StoreAttr()`.

* Use `specialize()` and `unspecialize()` helpers.

* Add unit tests to ensure specializing happens as expected in FT builds.

* Add unit tests to attempt to trigger data races (useful for running under TSAN).

* Add `has_split_table` function to `_testinternalcapi`.
2024-12-19 10:21:17 -08:00
Mark Shannon
d2f1d917e8
GH-122548: Implement branch taken and not taken events for sys.monitoring (GH-122564) 2024-12-19 16:59:51 +00:00
Donghee Na
48c70b8f7d
gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build (gh-127737) 2024-12-19 11:08:17 +09:00
Kumar Aditya
91c55085a9
gh-128033: change PyMutex_LockFast to take PyMutex as argument (#128054)
Change `PyMutex_LockFast` to take `PyMutex` as argument.
2024-12-18 20:49:00 +05:30
Bénédikt Tran
7303f06846
gh-126742: Add _PyErr_SetLocaleString, use it for gdbm & dlerror messages (GH-126746)
- Add a helper to set an error from locale-encoded `char*`
- Use the helper for gdbm & dlerror messages

Co-authored-by: Victor Stinner <vstinner@python.org>
2024-12-17 12:12:45 +01:00
Berker Peksag
cfeaa992ba
Free arena on _PyCompile_AstOptimize failure in Py_CompileStringObject (GH-127910)
After commit 10a91d7e9 introduced arena cleanup, commit 2dfbd4f36
removed the free call when _PyCompile_AstOptimize fails.
2024-12-16 22:59:36 +02:00
Tomas R.
081673801e
gh-127864: Fix compiler warning (-Wstringop-truncation) (GH-127878) 2024-12-16 11:57:18 -05:00
mpage
2de048ce79
gh-115999: Specialize loading attributes from modules in free-threaded builds (#127711)
We use the same approach that was used for specialization of LOAD_GLOBAL in free-threaded builds:

_CHECK_ATTR_MODULE is renamed to _CHECK_ATTR_MODULE_PUSH_KEYS; it pushes the keys object for the following _LOAD_ATTR_MODULE_FROM_KEYS (nee _LOAD_ATTR_MODULE). This arrangement avoids having to recheck the keys version.

_LOAD_ATTR_MODULE is renamed to _LOAD_ATTR_MODULE_FROM_KEYS; it loads the value from the keys object pushed by the preceding _CHECK_ATTR_MODULE_PUSH_KEYS at the cached index.
2024-12-13 10:17:16 -08:00
Mark Shannon
e62e1ca455
GH-126833: Dumps graphviz representation of executor graph. (GH-126880) 2024-12-13 11:00:00 +00:00
Pieter Eendebak
5fc6bb2754
gh-126868: Add freelist for compact int objects (GH-126865) 2024-12-13 10:06:26 +00:00
velemas
f823910bbd
gh-127865: Fix build failure for systems without thread local support (GH-127866)
This PR fixes the build issue introduced by the commit 628f6eb from
GH-112207 on systems without thread local support.
2024-12-12 13:07:55 -05:00
mpage
c84928ed6d
gh-115999: Specialize CALL_KW in free-threaded builds (#127713)
* Enable specialization of CALL_KW

* Fix bug pushing frame in _PY_FRAME_KW

`_PY_FRAME_KW` pushes a pointer to the new frame onto the stack for
consumption by the next uop. When pushing the frame fails, we do not
want to push the result, `NULL`, to the stack because it is not
a valid stackref. This works in the default build because `PyStackRef_NULL`
 and `NULL` are the same value, so the `PyStackRef_XCLOSE()` in the error
handler ignores it. In the free-threaded build the values are not the same;
`PyStackRef_XCLOSE()` will attempt to decref a null pointer.
2024-12-11 15:18:22 -08:00
Petr Viktorin
690fe077f6
gh-126491: Revert "GH-126491: Lower heap size limit with faster marking (GH-127519)" (GH-127770)
Revert "GH-126491: Lower heap size limit with faster marking (GH-127519)"

This reverts commit 023b7d2141, which introduced
a refleak.
2024-12-10 11:53:56 +01:00
Shantanu
3983527c3a
gh-127651: Use __file__ in diagnostics if origin is missing (#127660)
See the left hand side in https://github.com/python/cpython/pull/123929/files#diff-c22186367cbe20233e843261998dc027ae5f1f8c0d2e778abfa454ae74cc59deL2840-L2849

---------

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-12-09 20:55:20 -08:00
Russell Keith-Magee
2041a95e68
gh-126925: Modify how iOS test results are gathered (#127592)
Adds a `use_system_log` config item to enable stdout/stderr redirection for
Apple platforms. This log streaming is then used by a new iOS test runner
script, allowing the display of test suite output at runtime. The iOS test
runner script can be used by any Python project, not just the CPython test
suite.
2024-12-09 13:28:57 +08:00
Sam Gross
a353455fca
gh-125610: Fix STORE_ATTR_INSTANCE_VALUE specialization check (GH-125612)
The `STORE_ATTR_INSTANCE_VALUE` opcode doesn't support objects with
non-NULL managed dictionaries, so don't specialize to that op in that case.
2024-12-06 10:48:24 -05:00
Mark Shannon
023b7d2141
GH-126491: Lower heap size limit with faster marking (GH-127519)
* Faster marking of reachable objects

* Changes calculation of work to do and work done.

* Merges transitive closure calculations
2024-12-06 10:46:59 +00:00