Commit graph

8212 commits

Author SHA1 Message Date
Irit Katriel
608927b014
gh-103082: use IS_VALID_OPCODE instead of _PyOpcode_OpName to check if an opcode is defined (#107882) 2023-08-14 10:51:50 +01:00
Dong-hee Na
bf707749e8
gh-106797: Remove warning logs from Python/generated_cases.c.h and executor_cases.c.h (gh-107889)
gh-106797: Remove warning logs from Python/generated_cases.c.h
2023-08-13 04:36:46 +09:00
Mark Shannon
37d8b904f8
GH-107674: Avoid allocating boxed ints for sys.settrace line events (GH-107780) 2023-08-10 13:35:02 +01:00
Irit Katriel
bafedfbebd
gh-106149: move CFG and basicblock definitions into flowgraph.c, use them as opaque types in compile.c (#107639) 2023-08-10 13:03:47 +01:00
Mark Shannon
494e3d4436
GH-107774: Add missing audit event for PEP 669 (GH-107775) 2023-08-10 12:29:06 +01:00
Brandt Bucher
326f0ba1c5
GH-106485: Dematerialize instance dictionaries when possible (GH-106539) 2023-08-09 19:14:50 +00:00
Brandt Bucher
a9caf9cf90
GH-105848: Simplify the arrangement of CALL's stack (GH-107788) 2023-08-09 18:19:39 +00:00
Mark Shannon
52fbcf61b5
GH-107724: Fix the signature of PY_THROW callback functions. (GH-107725) 2023-08-09 09:30:50 +01:00
Brandt Bucher
ea72c6fe3b
GH-107596: Specialize str[int] (GH-107597) 2023-08-08 13:42:43 -07:00
Guido van Rossum
328d925244
gh-107758: Improvements to lltrace feature (#107757)
- The `dump_stack()` method could call a `__repr__` method implemented in Python,
  causing (infinite) recursion.
  I rewrote it to only print out the values for some fundamental types (`int`, `str`, etc.);
  for everything else it just prints `<type_name @ 0xdeadbeef>`.

- The lltrace-like feature for uops wrote to `stderr`, while the one in `ceval.c` writes to `stdout`;
  I changed the uops to write to stdout as well.
2023-08-07 21:36:25 -07:00
Eric Snow
430632d6f7
gh-107630: Initialize Each Interpreter's refchain Properly (gh-107733)
This finishes fixing the crashes in Py_TRACE_REFS builds.  We missed this part in gh-107567.
2023-08-07 13:14:56 -06:00
Ivin Lee
4e6fac7fcc
gh-106608: make uop trace variable length (#107531)
Executors are now more like tuples.
2023-08-04 21:10:46 -07:00
Brandt Bucher
05a824f294
GH-84436: Skip refcounting for known immortals (GH-107605) 2023-08-04 16:24:50 -07:00
Guido van Rossum
400835ea16
gh-106812: Refactor cases_generator to allow uops with array stack effects (#107564)
Introducing a new file, stacking.py, that takes over several responsibilities related to symbolic evaluation of push/pop operations, with more generality.
2023-08-04 09:35:56 -07:00
Mark Shannon
2ba7c7f7b1
Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Mark Shannon
fa45958450
GH-107263: Increase C stack limit for most functions, except _PyEval_EvalFrameDefault() (GH-107535)
* Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
2023-08-04 10:10:29 +01:00
Eric Snow
58ef741867
gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
2023-08-03 19:51:08 +00:00
Eric Snow
017f047183
gh-107471: Fix Refleaks in test_import (gh-107569)
gh-107184 introduced a refleak in test_import.SubinterpImportTests (specifically test_singlephase_check_with_setting_and_override and test_single_init_extension_compat).  We fix it here by making sure _testsinglephase is removed from sys.modules whenever we clear the runtime's internal state for the module.

The underlying problem is strictly contained in the internal function _PyImport_ClearExtension() (AKA _testinternalcapi.clear_extension()), which is only used in tests.

(This also fixes an intermittent segfault introduced in the same place, in test_disallowed_reimport.)
2023-08-02 20:55:09 +00:00
Mark Shannon
0d30a5a409
GH-100964: Break cycles involving exception state when returning from generator (GH-107563) 2023-08-02 18:44:20 +01:00
Irit Katriel
2bd04d4234
gh-105481: combine regen-opcode-targets with regen-opcode to avoid calculating the specialized opcodes in two places (#107540) 2023-08-01 21:05:48 +01:00
Brandt Bucher
dfb55d9d7f
Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506) 2023-07-31 14:18:38 -07:00
Brandt Bucher
5e584eb704
GH-104584: Fix incorrect uoperands (GH-107513) 2023-07-31 21:16:57 +00:00
Guido van Rossum
5eb80a61f5
GH-104909: Move unused cache entries from uops to macros (#107444)
There's no need to use a dummy uop to skip unused cache entries. The macro syntax lets you write `unused/1` instead.

Similarly, move `unused/5` from op `_LOAD_ATTR_INSTANCE_VALUE` to macro `LOAD_ATTR_INSTANCE_VALUE`.
2023-07-31 08:55:33 -07:00
Georg Brandl
f57b9fd2b6
no-issue: Fix typo in import.c (gh-107498) 2023-07-31 12:07:17 +00:00
Eric Snow
8ba4df91ae
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules.  The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL.  It turns out it's tricky to use the same thread state for different threads.  Who could have guessed?

We solve the problem by eliminating the one object we were still sharing between interpreters.  We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.

We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
2023-07-28 14:39:08 -06:00
Eric Snow
8bdae1424b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)
The _xxsubinterpreters module should not rely on internal API.  Some of the functions it uses were recently moved there however.  Here we move them back (and expose them properly).
2023-07-27 15:30:16 -06:00
Eric Snow
75c974f535
gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184)
This fixes a bug where incompatible modules could still be imported if attempted multiple times.
2023-07-27 15:08:38 -06:00
Eric Snow
b72947a8d2
gh-106931: Intern Statically Allocated Strings Globally (gh-107272)
We tried this before with a dict and for all interned strings.  That ran into problems due to interpreter isolation.  However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning.  Here we circle back to using a global cache, but only for statically allocated strings.  We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.

Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter.  Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
2023-07-27 13:56:59 -06:00
Mark Shannon
ac7a0f858a
GH-106898: Add the exception as an argument to the PY_UNWIND event callback function. (GH-107347) 2023-07-27 15:47:33 +01:00
Victor Stinner
1dbb427dd6
gh-107196: Remove _PyArg_VaParseTupleAndKeywordsFast() function (#107197)
Remove the private _PyArg_VaParseTupleAndKeywordsFast() function: it
is no longer used.
2023-07-27 16:36:54 +02:00
Mark Shannon
c6539b36c1
GH-106895: Raise a ValueError when attempting to disable events that cannot be disabled. (GH-107337) 2023-07-27 15:27:11 +01:00
Mark Shannon
766d2518ae
GH-106897: Add RERAISE event to sys.monitoring. (GH-107291)
* Ensures that exception handling events are balanced. Each [re]raise event has a matching unwind/handled event.
2023-07-27 13:32:30 +01:00
Irit Katriel
507d8bc39a
gh-106149: fix comment on stackdepth of generators (#107321) 2023-07-26 22:31:47 +00:00
Pablo Galindo Salgado
da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer (#107018) 2023-07-26 16:34:15 +01:00
Irit Katriel
b0202a4e5d
gh-106149: Simplify stack depth calculation. Replace asserts by exceptions. (#107255) 2023-07-26 13:32:47 +01:00
Guido van Rossum
233b878288
gh-107082: Fix instruction size computation for ENTER_EXECUTOR (#107256)
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-07-25 20:01:02 +00:00
Victor Stinner
188000ae4b
Remove unused internal _PyImport_GetModuleId() function (#107235) 2023-07-25 17:02:12 +00:00
Victor Stinner
1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner
2e0744955f
gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212)
Rename the internal PySymtable_Lookup() function to
_PySymtable_Lookup() and no longer export it.
2023-07-25 00:54:09 +00:00
Irit Katriel
7608fa8fd5
gh-106149: move _PyCfg_BasicblockLastInstr and make it local to flowgraph.c (#107180) 2023-07-24 22:08:59 +01:00
Carl Meyer
e5d5522612
gh-106917: fix super classmethod calls to non-classmethods (#106977) 2023-07-24 13:14:56 -07:00
Victor Stinner
fd66baf34a
gh-106320: Remove private _PyDict C API (#107145)
Move private _PyDict functions to the internal C API (pycore_dict.h):

* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()

No longer export these functions.
2023-07-24 14:02:03 +00:00
Victor Stinner
0d6dfd68d2
gh-106320: Remove private _PyObject C API (#107147)
Move private debug _PyObject functions to the internal C API
(pycore_object.h):

* _PyDebugAllocatorStats()
* _PyObject_CheckConsistency()
* _PyObject_DebugTypeStats()
* _PyObject_IsFreed()

No longer export most of these functions, except of
_PyObject_IsFreed().

Move test functions using _PyObject_IsFreed() from _testcapi to
_testinternalcapi. check_pyobject_is_freed() test no longer catch
_testcapi.error: the tested function cannot raise _testcapi.error.
2023-07-23 20:09:08 +00:00
Victor Stinner
0927a2b25c
GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069)
Rename private C API constants:

* Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS
* Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS
2023-07-22 21:35:27 +00:00
Victor Stinner
ee15844db8
gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
Victor Stinner
5e4af2a3e9
gh-106320: Move private _PySet API to the internal API (#107041)
* Add pycore_setobject.h header file.
* Move the following API to the internal C API:

  * _PySet_Dummy
  * _PySet_NextEntry()
  * _PySet_Update()
2023-07-22 17:04:34 +02:00
Victor Stinner
89f9875448
gh-106320: Move private _PyHash API to the internal C API (#107026)
* No longer export most private _PyHash symbols, only export the ones
  which are needed by shared extensions.
* Modules/_xxtestfuzz/fuzzer.c now uses the internal C API.
2023-07-22 13:49:37 +00:00
Kirill Podoprigora
85ed1d2442
gh-106916: Add missing error check _PyCompile_CleanDoc (#106921)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-21 13:56:58 +09:00
Brandt Bucher
8f4de57699
GH-106701: Move _PyUopExecute to Python/executor.c (GH-106924) 2023-07-20 20:37:19 +00:00
Irit Katriel
9c81fc2dbe
gh-105481: do not auto-generate pycore_intrinsics.h (#106913) 2023-07-20 17:46:04 +01:00