Commit graph

5961 commits

Author SHA1 Message Date
Pablo Galindo Salgado
0909d6d8e8
gh-91048: Add better error messages for remote debugging for CI builds (#134682) 2025-05-26 15:31:47 +01:00
Pieter Eendebak
fb09db1b93
gh-129594: Remove redundant check on varargs in _PyArg_CheckPositional (#129595) 2025-05-26 10:51:12 +02:00
Pablo Galindo Salgado
42b25ad4d3
gh-91048: Refactor and optimize remote debugging module (#134652)
Completely refactor Modules/_remote_debugging_module.c with improved
code organization, replacing scattered reference counting and error
handling with centralized goto error paths. This cleanup improves
maintainability and reduces code duplication throughout the module while
preserving the same external API.

Implement memory page caching optimization in Python/remote_debug.h to
avoid repeated reads of the same memory regions during debugging
operations. The cache stores previously read memory pages and reuses
them for subsequent reads, significantly reducing system calls and
improving performance.

Add code object caching mechanism with a new code_object_generation
field in the interpreter state that tracks when code object caches need
invalidation. This allows efficient reuse of parsed code object metadata
and eliminates redundant processing of the same code objects across
debugging sessions.

Optimize memory operations by replacing multiple individual structure
copies with single bulk reads for the same data structures. This reduces
the number of memory operations and system calls required to gather
debugging information from the target process.

Update Makefile.pre.in to include Python/remote_debug.h in the headers
list, ensuring that changes to the remote debugging header force proper
recompilation of dependent modules and maintain build consistency across
the codebase.

Also, make the module compatible with the free threading build as an extra :)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2025-05-25 20:19:29 +00:00
Brett Cannon
ad42dc1909
GH-130397: remove special-casing of C stack depth for WASI (#134469)
Removed special-casing for WASI when setting C stack depth limits. Since WASI has its own C stack checking this isn't a security risk.

Also disabled some tests that stopped passing. They all happened to have already been disabled under Emscripten.
2025-05-22 14:08:44 -07:00
Tomas R.
484e00379b
GH-131798: Optimize away isinstance calls in the JIT (GH-134369) 2025-05-22 12:52:47 -04:00
Eric Snow
4a4ac3ab4d
gh-132775: Make _PyXI_session Opaque (gh-134452)
This is mostly a refactor to clean things up a bit, most notably the "XI namespace" code.

Making the session opaque requires adding the following internal-only functions:

* _PyXI_NewSession()
* _PyXI_FreeSession()
* _PyXI_GetMainNamespace()
2025-05-22 10:14:04 -06:00
Brandt Bucher
ec736e7dae
GH-131798: Optimize cached class attributes and methods in the JIT (GH-134403) 2025-05-22 11:15:03 -04:00
Mark Shannon
fbe7b87c06
GH-131688: Mark instructions and uops with DECREF_INPUTS as escaping. (GH-133501) 2025-05-22 09:54:35 +01:00
Mark Shannon
29f6dc6323
GH-133932: Tagged ints are heap safe (GH-134244) 2025-05-22 09:54:04 +01:00
Eric Snow
a66bae8bb5
gh-132775: Use _PyFunction_VerifyStateless() and _PyCode_VerifyStateless() (gh-134439) 2025-05-21 14:16:55 -06:00
Eric Snow
88f8102a8f
gh-132775: Support Fallbacks in _PyObject_GetXIData() (gh-133482)
It now supports a "full" fallback to _PyFunction_GetXIData() and then `_PyPickle_GetXIData()`.  There's also room for other fallback modes if that later makes sense.
2025-05-21 07:23:48 -06:00
Peter Bierma
b8998fe2d8
gh-131185: Use a proper thread-local for cached thread states (gh-132510)
Switches over to a _Py_thread_local in place of autoTssKey, and also fixes a few other checks regarding PyGILState_Ensure after finalization.

Note that this doesn't fix concurrent use of PyGILState_Ensure with Py_Finalize; I'm pretty sure zapthreads doesn't work at all, and that needs to be fixed seperately.
2025-05-21 07:01:25 -06:00
Brandt Bucher
2f0570caf4
GH-131798: Narrow types more aggressively in the JIT (GH-134373) 2025-05-20 18:09:51 -04:00
Donghee Na
317c496223
gh-129748: Update mimalloc to use atomic store for mi_block_set_nextx (#134238) 2025-05-20 11:39:56 -04:00
Pablo Galindo Salgado
dd7f113057
gh-115999: Add PyCodeObject.co_tlbc to the debug offsets (#134286)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2025-05-20 15:31:37 +00:00
Mark Shannon
6dcb0fdfe0
GH-134282: Always borrow references LOAD_CONST (GH-134284) 2025-05-20 11:24:11 -04:00
Tomas R.
a7f317d730
GH-131798: Add _POP_CALL_TWO_LOAD_CONST_INLINE_BORROW (GH-134268) 2025-05-19 18:00:53 -04:00
Diego Russo
42d03f3933
GH-131798: Split CALL_LIST_APPEND into several uops (GH-134240) 2025-05-19 15:48:55 -04:00
Dino Viehland
3fa30d9e9c
gh-128045: Syncs w/ latest opcode metadata (#134231)
Fix opcode metadata
2025-05-19 11:08:50 -04:00
Dino Viehland
cc9add695d
gh-128045: Mark unknown opcodes as deopting to themselves (#128044)
* Mark unknown opcodes as deopting to themselves
2025-05-19 10:15:16 -04:00
Bénédikt Tran
ee36db5500
gh-125843: indicate which C function caused a curses.error (#125844)
- Rename error helpers with a `curses_set_error_*` prefix instead of `PyCurses*`.
- Cleanly report both NULL and ERR cases.
- Raise `curses.error` in `is_linetouched` instead of a `TypeError`.
2025-05-19 15:53:39 +02:00
Steve Dower
986c367028
gh-133779: Revert Windows generation of pyconfig.h and go back to a static header. (GH-133966)
Extension builders must specify Py_GIL_DISABLED if they want to link to the free-threaded builds.
This was usually the case already, but this change guarantees it in all circumstances.
2025-05-19 11:35:22 +01:00
Peter Hawkins
9ad0c7b0f1
gh-132641: fix race in lru_cache under free-threading (#133787)
Fix race in `lru_cache` by acquiring critical section on the cache object itself and call the lock held variant of dict functions to modify the underlying dict.
2025-05-13 17:38:57 +00:00
Eric Snow
8cf4947b0f
gh-132775: Add _PyFunction_GetXIData() (gh-133481) 2025-05-12 22:10:56 +00:00
Serhiy Storchaka
9f69a58623
gh-133767: Fix use-after-free in the unicode-escape decoder with an error handler (GH-129648)
If the error handler is used, a new bytes object is created to set as
the object attribute of UnicodeDecodeError, and that bytes object then
replaces the original data. A pointer to the decoded data will became invalid
after destroying that temporary bytes object. So we need other way to return
the first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal().

_PyBytes_DecodeEscape() does not have such issue, because it does not
use the error handlers registry, but it should be changed for compatibility
with _PyUnicode_DecodeUnicodeEscapeInternal().
2025-05-12 20:42:23 +03:00
Adam Turner
d29ddbd90c
gh-132983: Convert zstd `__new__` methods to Argument Clinic (#133860) 2025-05-12 08:51:53 +00:00
Nybblista
832058274d
gh-133713: Compare the f->stackpointer to the result of _PyFrame_Stackbase(f) (GH-133714) 2025-05-10 11:04:32 +08:00
Stan Ulbrych
4fd1095280
gh-133610: Remove PyUnicode_AsDecoded/Encoded functions (#133612) 2025-05-09 17:31:24 +02:00
Bénédikt Tran
5044e85265
gh-133644: Remove deprecated Python initialization getter functions (#133661)
Remove functions:

* Py_GetExecPrefix()
* Py_GetPath()
* Py_GetPrefix()
* Py_GetProgramFullPath()
* Py_GetProgramName()
* Py_GetPythonHome()
2025-05-09 11:39:23 +00:00
Nybblista
6d5a8c2ec1
gh-133476: Assert with the PyStackRef_IsTaggedInt function (GH-133477) 2025-05-09 08:17:50 +08:00
Tomas R.
c492ac7252
GH-131798: Split up and optimize CALL_ISINSTANCE (GH-133339) 2025-05-08 14:26:30 -07:00
Eric Snow
c81fa2b9cd
gh-132775: Add _PyCode_GetScriptXIData() (gh-133480)
This converts functions, code, str, bytes, bytearray, and memoryview objects to PyCodeObject,
and ensure that the object looks like a script.  That means no args, no return, and no closure.
_PyCode_GetPureScriptXIData() takes it a step further and ensures there are no globals.

We also add _PyObject_SupportedAsScript() to the internal C-API.
2025-05-08 15:07:46 +00:00
Xuehai Pan
f0f93ba5fa
gh-131942: Use the Python-specific Py_DEBUG macro rather than _DEBUG in Windows-related C code (GH-131944) 2025-05-08 15:01:25 +00:00
Bénédikt Tran
a2c4467d06
gh-133644: remove deprecated PyImport_ImportModuleNoBlock (#133655) 2025-05-08 13:08:43 +00:00
Serhiy Storchaka
dcf93c4c93
gh-133595: Clean up sqlite3.Connection APIs (GH-133605)
* All parameters of sqlite3.connect() except "database" are now keyword-only.
* The first three parameters of methods create_function() and
  create_aggregate() are now positional-only.
* The first parameter of methods set_authorizer(), set_progress_handler()
  and set_trace_callback() is now positional-only.
2025-05-08 15:42:00 +03:00
Serhiy Storchaka
4c914e7a36
gh-133583: Add support for fixed size unsigned integers in argument parsing (GH-133584)
* Add Argument Clinic converters: uint8, uint16, uint32, uint64.
* Add private C API: _PyLong_UInt8_Converter(),
  _PyLong_UInt16_Converter(), _PyLong_UInt32_Converter(),
  _PyLong_UInt64_Converter().
2025-05-08 12:27:50 +03:00
Eric Snow
27128e4fa8
gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133528)
This reverts commit 3c73cf5 (gh-133497), which itself reverted
the original commit d270bb5 (gh-133221).

We reverted the original change due to failing android tests.
The checks in _PyCode_CheckNoInternalState() were too strict,
so we've relaxed them.
2025-05-08 00:00:33 +00:00
Zachary Ware
74e2acddf6
Test fixes for 3.15 (GH-133599)
Followup to 942673ed19 (GH-133588)

* Update configure for Python 3.15

* Update magic number for 3.15

* Remove deprecated 'check_home' argument from sysconfig.is_python_build

* Add warningignore entries for Modules/_sqlite/clinic/connection.c.h

* Work around c-analyzer complaints about _testclinic deprecation tests

---------

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-05-07 14:50:39 -05:00
Hugo van Kemenade
9748fb3867 Python 3.15.0a0 2025-05-07 18:47:42 +03:00
Hugo van Kemenade
b092705907 Python 3.14.0b1 2025-05-06 18:33:52 +03:00
Irit Katriel
296cd128bf
Revert "gh-133395: add option for extension modules to specialize BINARY_OP/SUBSCR, apply to arrays (#133396)" (#133498) 2025-05-06 13:12:26 +03:00
Petr Viktorin
3c73cf51df
gh-132775: Revert "gh-132775: Add _PyCode_VerifyStateless() (gh-133221)" (#133497) 2025-05-06 13:09:41 +03:00
Adam Turner
529012e26f
Remove duplicate includes: Include/internal/ (#132624) 2025-05-06 02:08:42 +01:00
Eric Snow
ea598730ef
gh-132775: Add _PyCode_GetXIData() (gh-133475) 2025-05-05 23:46:03 +00:00
Eric Snow
d270bb5792
gh-132775: Add _PyCode_VerifyStateless() (gh-133221)
"Stateless" code is a function or code object which does not rely on external state or internal state.
It may rely on arguments and builtins, but not globals or a closure. I've left a comment in
pycore_code.h that provides more detail.

We also add _PyFunction_VerifyStateless(). The new functions will be used in several later changes
that facilitate "sharing" functions and code objects between interpreters.
2025-05-05 21:48:58 +00:00
Diego Russo
9cc77aaf9d
GH-131798: Split CALL_LEN into several uops (GH-133180) 2025-05-05 14:31:48 -07:00
Neil Schemenauer
893034cf93
gh-132917: Use RSS + swap for estimate of process memory usage (gh-133464) 2025-05-05 14:15:05 -07:00
Eric Snow
24ebb9ccfd
gh-132775: Unrevert "Add _PyCode_GetVarCounts()" (gh-133265)
This reverts commit 811edcf (gh-133232), which itself reverted the original commit 811edcf (gh-133128).

We reverted the original change due to failing s390 builds (a big-endian architecture).
It ended up that I had not accommodated op caches.
2025-05-05 13:24:29 -06:00
Peter Bierma
b275b8f342
gh-133140: Add PyUnstable_Object_IsUniquelyReferenced for free-threading (#133144) 2025-05-05 21:01:20 +02:00
Neil Schemenauer
5c245ffce7
gh-132917: Check resident set size (RSS) before GC trigger. (gh-133399)
For the free-threaded build, check the process resident set size (RSS)
increase before triggering a full automatic garbage collection.  If the RSS
has not increased 10% since the last collection then it is deferred.
2025-05-05 17:17:05 +00:00