Commit graph

4553 commits

Author SHA1 Message Date
T. Wouters
bbfd9c92fa
gh-115999: Fix gdb support for libpython.so after thread-local bytecode change (#126440)
Fix the gdb pretty printer in the face of --enable-shared by delaying the attempt to load the _PyInterpreterFrame definition until after .so files are loaded.
2024-11-05 15:49:27 +01:00
Peter Bierma
1371295e67
gh-126366: Fix crash if __iter__ raises an exception during yield from (#126369) 2024-11-05 15:26:36 +05:30
mpage
2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for BINARY_OP (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
sobolevn
bfc1d2504c
gh-109413: Add more type hints to libregrtest (#126352) 2024-11-04 13:15:57 +03:00
Bénédikt Tran
38a604fd90
gh-99108: Cleanup references to inexisting Modules/_blake2. (GH-126270)
* Remove references to `Modules/_blake2`.

* Remove `Modules/_blake2` entry from CODEOWNERS

The folder does not exist anymore.

* Remove `Modules/_blake2` entry from `Tools/c-analyzer/TODO`
2024-11-01 16:18:44 +00:00
mpage
821759d631
gh-126211: Exclude preprocessor directives from statements containing escaping calls (#126213)
The cases generator inserts code to save and restore the stack pointer around
statements that contain escaping calls. To find the beginning of such statements,
we would walk backwards from the escaping call until we encountered a token that
was treated as a statement terminator. This set of terminators should include
preprocessor directives.
2024-11-01 08:53:03 -07:00
Sergey B Kirpichev
8c22eba877
gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
Avoid temporary tuple creation when all arguments either positional-only
or vararg.

Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight
performance regression for set methods, introduced by gh-115112.
2024-10-31 11:37:03 +01:00
Savannah Ostrowski
c29bbe2101
GH-125498: Update JIT builds to use LLVM 19 and preserve_none (GH-125499) 2024-10-30 12:03:31 -07:00
Pablo Galindo Salgado
9dfef4e5f4
gh-125588: Teach the python PEG generator the new f-string tokens (#125589)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-10-29 23:40:12 +00:00
Tomas R.
aab58a93ef
gh-118423: Add INSTRUCTION_SIZE macro to code generator (GH-125467) 2024-10-29 17:25:05 +00:00
Hood Chatham
dc2552d429
gh-124928: Emscripten node support: Clean up old node <= 16 flags (#124929)
Clean up configure flags for old node versions

These flags are only needed for node <= 16. Node 16 has been end of life since
October of 2023.
2024-10-29 06:28:54 +00:00
Sam Gross
00ea179879
gh-125985: Add free threading scaling micro benchmarks (#125986)
These consist of a number of short snippets that help identify scaling
bottlenecks in the free threaded interpreter.

The current bottlenecks are in calling functions in benchmarks that call
functions (due to `LOAD_ATTR` not yet using deferred reference counting)
and when accessing thread-local data.
2024-10-28 17:47:23 -04:00
Mark Shannon
25441592db
GH-125515: Reduce number of compiler warnings in generated code (GH-125697) 2024-10-28 10:30:31 +00:00
Erlend E. Aasland
26d627779f
gh-89640: properly detect float word ordering on Linux (#125571)
autoconf-archive patch by Dan Amelang.
2024-10-26 15:46:46 +00:00
Mark Shannon
57e3c59bb6
GH-125521: Remove if (true) from generated output to reduce C compiler warnings (GH-125700) 2024-10-22 10:11:29 +01:00
Mikhail Efimov
c1bdbe84c8
gh-124889: Rework Python generator cache (#125816) 2024-10-22 09:42:56 +01:00
Adam Turner
f36d37bbaf
gh-125741: Update build.yml for the new check_autoconf_regen job (#125772) 2024-10-21 11:54:54 +01:00
Michael Droettboom
c8fd4b12e3
gh-125207: Fix MSVC 1935 build with JIT (#125209)
* gh-125207: Use {0} array initializers

* Simplify, as suggested in PR

* Revert change to explicitly specify length
2024-10-18 15:51:29 -04:00
Wulian
cda0ec8e7c
gh-124102: Clean up unsupported VS and WiX detections (GH-124784) 2024-10-18 13:48:18 +01:00
Kirill Podoprigora
d83fcf8371
CI: Bump Python to 3.13 and mypy to 1.12 in mypy workflow (#125592)
* Bump mypy to 1.12 & Python to 3.13

* Remove unnecessary `type: ignore`
2024-10-16 14:27:19 +00:00
Kirill Podoprigora
bee112a94d
gh-124872: Replace enter/exit events with "switched" (#125532)
Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for gh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-10-16 13:53:21 +02:00
Kirill Podoprigora
d3c82b9cce
gh-125512: Revert "gh-124872: Replace enter/exit events with "switched" (#124776)" (#125513) 2024-10-15 17:42:16 +03:00
Richard Hansen
843d28f59d
gh-124872: Replace enter/exit events with "switched" (#124776)
Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for gh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).
2024-10-14 12:28:41 -07:00
Mikhail Efimov
187580d95c
gh-119786: [doc] broken link and typo fix in interpreter_definition.md (#125455) 2024-10-14 17:24:54 +00:00
Mark Shannon
06ca33020e
GH-125323: Convert DECREF_INPUTS_AND_REUSE_FLOAT into a function that takes PyStackRefs. (GH-125439) 2024-10-14 14:18:57 +01:00
Ken Jin
4b358ee647
gh-125323: Remove some unsafe Py_DECREFs in bytecodes.c, replacing them with PyStackRef_CLOSEs (GH-125324) 2024-10-14 09:17:51 +01:00
Donghee Na
08f6bf7171
gh-124612: Update autoconf container image (#125320) 2024-10-12 00:27:26 +09:00
Mark Shannon
c9014374c5
GH-125174: Make immortal objects more robust, following design from PEP 683 (GH-125251) 2024-10-10 18:19:08 +01:00
Tomas R.
6b533a659b
gh-125039: Make this_instr/prev_instr const in cases generator (GH-125071) 2024-10-09 13:54:39 +01:00
Donghee Na
b502573f7f
gh-124612: Use ghcr.io/python/autoconf instead of public image (#124657)
* gh-124612: Use ghcr.io/python/autoconf instead of public image

* Update
2024-10-09 09:33:18 +09:00
Cody Maloney
7dca7322cc
gh-125084: Resolve paths in generator common code (GH-125085)
In out of tree builds, the paths can contain `../ which needs to be
resolved for the relative path calculation to work.
2024-10-08 12:16:02 -05:00
Bénédikt Tran
e4292c0410
gh-123961: Convert _curses to a multi-phase init module (PEP-489) (#124965) 2024-10-08 13:42:44 +02:00
Mark Shannon
da071fa3e8
GH-119866: Spill the stack around escaping calls. (GH-124392)
* Spill the evaluation around escaping calls in the generated interpreter and JIT. 

* The code generator tracks live, cached values so they can be saved to memory when needed.

* Spills the stack pointer around escaping calls, so that the exact stack is visible to the cycle GC.
2024-10-07 14:56:39 +01:00
efimov-mikhail
1f9025a4e7
gh-124889: Remove redundant artificial rules in PEG parser (#124893)
Cache in C PEG-generator reworked:
we save artificial rules in cache by Node string representation as a key instead of Node object itself.
As a result total count of artificial rules in parsers.c is lowered from 283 to 170.
More natural number ordering is used for the names of artificial rules.

Auxiliary method CCallMakerVisitor._generate_artificial_rule_call is added.
Its purpose is abstracting work with artificial rules cache.

Explicit using of "is_repeat1" kwarg is added to visit_Repeat0 and visit_Repeat1 methods.
Its slightly improve code readabitily.
2024-10-03 13:58:56 +01:00
Diego Russo
b85923a0fe
GH-119726: Deduplicate AArch64 trampolines within a trace (GH-123872) 2024-10-02 12:07:20 -07:00
Savannah Ostrowski
6737333ac5
GH-124820: Move -mno-outline-atomics flag to aarch64 Linux JIT builds only (#124821) 2024-10-02 11:40:59 +02:00
Wulian
da1e5526ae
gh-124487: Updated installer messages and internal docs regarding minimum Windows version (GH-124822) 2024-10-01 15:44:18 +01:00
Steve Dower
fac5e7aa17
gh-124487: Require at least Windows 10 SDK and update install check (GH-124672) 2024-09-30 18:50:46 +01:00
Bénédikt Tran
4d8e7c40a0
gh-123961: Add a global state to _curses (#124729) 2024-09-29 15:17:20 +00:00
Bénédikt Tran
c00964ecd5
gh-124665: Add _PyCodec_UnregisterError and _codecs._unregister_error (#124677) 2024-09-29 02:25:23 +02:00
Jelle Zijlstra
25189188bf
generate_global_objects.py: Fix name of macro in error message (#124464)
_PyID does not exist but _Py_ID does.
2024-09-26 21:37:21 -07:00
Savannah Ostrowski
65f1237098
GH-123516: Improve JIT memory consumption by invalidating cold executors (GH-124443)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-09-27 00:35:42 +00:00
Brett Cannon
66cc6d4c50
Remove some unused files related to WASM/WASI (GH-124635) 2024-09-26 21:23:41 +00:00
Irit Katriel
78aeb38f7d
gh-124285: Fix bug where bool() is called multiple times for the same part of a boolean expression (#124394) 2024-09-25 15:51:25 +01:00
Savannah Ostrowski
b6471f4a39
GH-124408: Temporarily skip test_strftime_y2k for emulated Linux JIT CI (GH-124466) 2024-09-24 18:09:50 -07:00
Sam Gross
f4997bb3ac
gh-123923: Defer refcounting for f_funcobj in _PyInterpreterFrame (#124026)
Use a `_PyStackRef` and defer the reference to `f_funcobj` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
2024-09-24 20:08:18 +00:00
Jason Fried
d87482bc4e
gh-119333: Add C api to have contextvar enter/exit callbacks (#119335)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-09-23 20:40:17 -07:00
Hugo van Kemenade
e256a7590a
Lint and format Tools/build/check-warnings.py (#124382) 2024-09-24 02:47:05 +03:00
Wulian
27a62e7371
gh-124102: Update internal PCbuild docs to accurately list build dependencies (GH-124103) 2024-09-23 23:09:22 +00:00
Mark Shannon
c87b0e4a46
GH-124284: Add stats for refcount operations on immortal objects (GH-124288) 2024-09-23 19:10:55 +01:00