Commit graph

8945 commits

Author SHA1 Message Date
Raphael Gaschignard
2770d5caca
gh-105879: Add support for keyword arguments to eval and exec (#105885)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-05-02 08:55:29 -07:00
Mark Shannon
72867c962c
GH-118095: Unify the behavior of tier 2 FOR_ITER branch micro-ops (GH-118420)
* Target _FOR_ITER_TIER_TWO at POP_TOP following the matching END_FOR

* Modify _GUARD_NOT_EXHAUSTED_RANGE, _GUARD_NOT_EXHAUSTED_LIST and _GUARD_NOT_EXHAUSTED_TUPLE so that they also target the POP_TOP following the matching END_FOR
2024-05-02 16:17:59 +01:00
NGRsoftlab
7d2ffada0a
gh-116180: Check the globals argument in PyRun_* C API (GH-116637)
It used to crash when passing NULL or non-dict as globals.
Now it sets a SystemError.
2024-05-02 16:43:03 +03:00
Crowthebird
7c97dc8c95
gh-118216: Don't consider dotted __future__ imports (#118267) 2024-05-02 06:32:20 -07:00
Mark Shannon
67bba9dd0f
GH-117442: Check eval-breaker at start (rather than end) of tier 2 loops (GH-118482) 2024-05-02 13:10:31 +01:00
Eric Snow
526ca4c09e
gh-117953: Work Relative to Specific Extension Kinds in the Import Machinery (gh-118205)
This change will make some later changes simpler.
2024-05-01 17:40:28 -06:00
Mark Shannon
39981fd07a
GH-118095: Make sure that progress is made if there are pending calls being handled. (GH-118484) 2024-05-01 22:18:31 +01:00
Irit Katriel
6763bfcc0f
gh-118272: set stacktop to 0 before freeing contents, to avoid access to invalid objects during GC (#118478) 2024-05-01 21:51:40 +01:00
Hugo van Kemenade
3b3f8dea57
gh-117225: Move colorize functionality to own internal module (#118283) 2024-05-01 12:27:06 -06:00
Victor Stinner
b52c753e0f
gh-110850: Add PyTime_TimeRaw() function (#118394)
Add "Raw" variant of PyTime functions:

* PyTime_MonotonicRaw()
* PyTime_PerfCounterRaw()
* PyTime_TimeRaw()

Changes:

* Add documentation and tests. Tests release the GIL while calling
  raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
  the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-01 18:05:01 +00:00
Brandt Bucher
49baa656cb
GH-115802: Use the GHC calling convention in JIT code (GH-118287) 2024-05-01 08:05:53 -07:00
Anthony Shaw
beb653cc24
gh-117958: Expose JIT code via method in UOpExecutor (#117959) 2024-05-01 07:11:14 -07:00
Irit Katriel
c1bf4874c1
gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
2024-05-01 13:01:16 +02:00
Mark Shannon
f6fab21721
GH-118095: Make invalidating and clearing executors memory safe (GH-118459) 2024-05-01 11:34:50 +01:00
Guido van Rossum
7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Ken Jin
7fabcc727d
gh-117657: Don't specialize RESUME_CHECK when specialization is disabled (GH-118349) 2024-04-30 21:51:59 +00:00
Victor Stinner
e93c39b47e
gh-118422: Fix run_fileexflags() test (#118429)
Don't test the undefined behavior of fileno()
on a closed file, but use fstat() as a reliable
test if the file was closed or not.
2024-04-30 22:32:55 +02:00
Sam Gross
b2c3b70c71
gh-118332: Fix deadlock involving stop the world (#118412)
Avoid detaching thread state when stopping the world. When re-attaching
the thread state, the thread would attempt to resume the top-most
critical section, which might now be held by a thread paused for our
stop-the-world request.
2024-04-30 15:01:28 -04:00
Dino Viehland
4a1cf66c5c
gh-117657: Fix small issues with instrumentation and TSAN (#118064)
Small TSAN fixups for instrumentation
2024-04-30 11:38:05 -07:00
Irit Katriel
1f16b4ce56
gh-118272: Clear generator frame's locals when the generator is closed (#118277)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
2024-04-30 19:32:25 +01:00
Nice Zombies
9a75d56d5d
gh-118379: Use PyTuple_Pack instead of Py_BuildValue if possible (GH-118381) 2024-04-30 17:55:15 +03:00
Malcolm Smith
3b268f4edc
gh-116622: Redirect stdout and stderr to system log when embedded in an Android app (#118063) 2024-04-30 16:00:31 +02:00
Mark Shannon
5b05d452cd
GH-118095: Add tier 2 support for YIELD_VALUE (GH-118380) 2024-04-30 11:33:13 +01:00
Savannah Ostrowski
8b56d82c59
GH-118306: Update JIT to use LLVM 18 (GH-118307) 2024-04-29 21:09:16 +00:00
Alex Turner
2ba1aed596
gh-117657: TSAN fix race on gstate->young.count (#118313) 2024-04-29 20:26:26 +00:00
Eric Snow
529a160be6
gh-117953: Share More Machinery Code Between Builtin and Dynamic Extensions (gh-118204)
This change will make some later changes simpler. It also brings more consistent behavior and lower maintenance costs.
2024-04-29 12:53:04 -06:00
Sam Gross
7ccacb220d
gh-117783: Immortalize objects that use deferred reference counting (#118112)
Deferred reference counting is not fully implemented yet. As a temporary
measure, we immortalize objects that would use deferred reference
counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first
non-main thread is started. Additionally, some tests, including refleak
tests, suppress this behavior.
2024-04-29 14:36:02 -04:00
Eric Snow
44f57a952e
gh-117953: Split Up _PyImport_LoadDynamicModuleWithSpec() (gh-118203)
Basically, I've turned most of _PyImport_LoadDynamicModuleWithSpec() into two new functions (_PyImport_GetModInitFunc() and _PyImport_RunModInitFunc()) and moved the rest of it out into _imp_create_dynamic_impl().  There shouldn't be any changes in behavior.

This change makes some future changes simpler.  This is particularly relevant to potentially calling each module init function in the main interpreter first.  Thus the critical part of the PR is the addition of _PyImport_RunModInitFunc(), which is strictly focused on running the init func and validating the result.  A later PR will take it a step farther by capturing error information rather than raising exceptions.

FWIW, this change also helps readers by clarifying a bit more about what happens when an extension/builtin module is imported.
2024-04-29 09:29:07 -06:00
Tian Gao
375c94c75d
gh-107674: Lazy load line number to improve performance of tracing (GH-118127) 2024-04-29 09:54:52 +01:00
Mark Shannon
ab6eda0ee5
GH-118095: Allow a variant of RESUME_CHECK in tier 2 (GH-118286) 2024-04-29 07:54:05 +01:00
Jelle Zijlstra
2326d6c868
gh-109118: Make comprehensions work within annotation scopes, but without inlining (#118160)
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-28 06:21:28 -07:00
Sam Gross
5a90de0d4c
gh-116749: Disable GIL by default in free-threaded build (#118295)
Switch GIL to disabled by default in free-threaded build so that the
free-threaded CIs catch thread-safety issues.
2024-04-26 14:22:29 -04:00
Mark Shannon
3e06c7f719
GH-118095: Add dynamic exit support and FOR_ITER_GEN support to tier 2 (GH-118279) 2024-04-26 18:08:50 +01:00
neonene
63add11704
gh-117680: Fix msvc warning in instruction_sequence.c (#118326) 2024-04-26 17:20:30 +01:00
Tian Gao
0f99861332
gh-117385: Remove unhooked events on sys.settrace (GH-117386) 2024-04-26 17:01:44 +01:00
Eric Snow
09c2947581
gh-110693: Pending Calls Machinery Cleanups (gh-118296)
This does some cleanup in preparation for later changes.
2024-04-26 01:05:51 +00:00
Mark Shannon
f180b31e76
GH-118095: Handle RETURN_GENERATOR in tier 2 (GH-118180) 2024-04-25 11:32:47 +01:00
Nice Zombies
10bb90ed49
gh-102511: Speed up os.path.splitroot() with native helpers (GH-118089) 2024-04-25 10:07:38 +01:00
Eric Snow
85ec1c2dc6
gh-117953: Fix Refleaks Introduced by gh-118194 (gh-118250)
A couple of refleaks slipped through in gh-118194. This takes care of them.

(AKA _Py_ext_module_loader_info_init() does not steal references.)
2024-04-24 21:23:45 +00:00
Eric Snow
5865fa5f9b
gh-117953: Add Internal struct _Py_ext_module_loader_info (gh-118194)
This helps with a later change that splits up _PyImport_LoadDynamicModuleWithSpec().
2024-04-24 17:42:01 +00:00
Eric Snow
1acd249798
gh-117953: Let update_global_state_for_extension() Caller Decide If Singlephase or Not (gh-118193)
This change makes other upcoming changes simpler.
2024-04-24 16:28:35 +00:00
Eric Snow
03e3e31723
gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791)
See https://discuss.python.org/t/pep-734-multiple-interpreters-in-the-stdlib/41147/26.
2024-04-24 16:18:24 +00:00
Eric Snow
af3c1d817d
gh-117953: Cleanups For fix_up_extension() in import.c (gh-118192)
These are cleanups I've pulled out of gh-118116.  Mostly, this change moves code around to align with some future changes and to improve clarity a little.  There is one very small change in behavior: we now add the module to the per-interpreter caches after updating the global state, rather than before.
2024-04-24 09:55:48 -06:00
Mark Shannon
83235f7791
GH-115419: Move setting the instruction pointer to error exit stubs (GH-118088) 2024-04-24 14:41:30 +01:00
Mark Shannon
77cd0428b6
GH-118095: Convert DEOPT_IFs on likely side exits to EXIT_IFs (GH-118106)
Covert DEOPT_IFs on likely side exits to EXIT_IFs
2024-04-24 14:37:55 +01:00
Irit Katriel
0aa0fc3d3c
gh-117901: Add option for compiler's codegen to save nested instruction sequences for introspection (#118007) 2024-04-24 09:46:17 +00:00
Guido van Rossum
7e87d30f1f
gh-118074: Immortal executors are not GC-able (#118182)
Better version of gh-118117.
Just check for immortality instead of an address range check.
2024-04-23 13:38:23 -07:00
mpage
2e7771a03d
gh-117657: Quiet TSAN warnings about remaining non-atomic accesses of tstate->state (#118165)
Quiet TSAN warnings about remaining non-atomic accesses of `tstate->state`
2024-04-23 10:20:14 -07:00
Eric Snow
23950beff8
gh-117953: Small Cleanup of Extensions-Related Machinery Code (gh-118167)
This is a collection of very basic cleanups I've pulled out of gh-118116.  It is mostly renaming variables and moving a couple bits of code in functionally equivalent ways.
2024-04-23 08:25:50 -06:00
Joe Jevnik
eb927e9fc8
gh-68114: Fix handling for removed PyArg_ParseTuple 'w' formatters (GH-8204)
Co-authored-by: Joe Jevnik <joe@quantopian.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-04-23 13:15:15 +02:00