Commit graph

8328 commits

Author SHA1 Message Date
Irit Katriel
a1cc74c4ee
gh-107901: Fix missing line number on BACKWARD_JUMP at the end of a for loop (#108242) 2023-08-21 23:44:31 +01:00
Irit Katriel
10a91d7e98
gh-108113: Make it possible to create an optimized AST (#108154) 2023-08-21 16:31:30 +00:00
Serhiy Storchaka
80bdebdd85
gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929) 2023-08-21 14:16:31 +03:00
Nikita Sobolev
db6dc6ce41
gh-107526: Revert "gh-100357: Convert several functions in bltinsmodule to AC" (#107542) 2023-08-20 17:54:10 -07:00
Serhiy Storchaka
633ea217a8
gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
2023-08-19 14:51:03 +03:00
Victor Stinner
3ff5ef2ad3
gh-108014: Add Py_IsFinalizing() function (#108032)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 12:34:41 +02:00
Pablo Galindo Salgado
75b3db8445
gh-107944: Improve error message for function calls with bad keyword arguments (#107969) 2023-08-17 19:39:42 +01:00
Guido van Rossum
61c7249759
gh-106581: Project through calls (#108067)
This finishes the work begun in gh-107760. When, while projecting a superblock, we encounter a call to a short, simple function, the superblock will now enter the function using `_PUSH_FRAME`, continue through it, and leave it using `_POP_FRAME`, and then continue through the original code. Multiple frame pushes and pops are even possible. It is also possible to stop appending to the superblock in the middle of a called function, when running out of space or encountering an unsupported bytecode.
2023-08-17 11:29:58 -07:00
Mark Shannon
006e44f950
GH-108035: Remove the _PyCFrame struct as it is no longer needed for performance. (GH-108036) 2023-08-17 11:16:03 +01:00
Guido van Rossum
dc8fdf5fd5
gh-106581: Split CALL_PY_EXACT_ARGS into uops (#107760)
* Split `CALL_PY_EXACT_ARGS` into uops

This is only the first step for doing `CALL` in Tier 2.
The next step involves tracing into the called code object and back.
After that we'll have to do the remaining `CALL` specialization.
Finally we'll have to deal with `KW_NAMES`.

Note: this moves setting `frame->return_offset` directly in front of
`DISPATCH_INLINED()`, to make it easier to move it into `_PUSH_FRAME`.
2023-08-16 16:26:43 -07:00
Irit Katriel
665a4391e1
gh-105481: generate op IDs from bytecode.c instead of hard coding them in opcode.py (#107971) 2023-08-16 22:25:18 +00:00
Nikita Sobolev
bdd8ddfda1
gh-105724: Add location information to assert errors (GH-105935) 2023-08-16 11:35:38 +01:00
Ken Jin
e28b0dc86d
gh-107557: Setup abstract interpretation (#107847)
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Jules <57632293+juliapoo@users.noreply.github.com>
2023-08-15 18:04:17 +00:00
Steve Dower
34e1917912
gh-106242: Minor fixup to avoid compiler warnings (GH-107983) 2023-08-15 17:02:32 +00:00
Irit Katriel
971a4c2751
gh-103082: remove assumption that INSTRUMENTED_LINE is the last instrumented opcode (#107978) 2023-08-15 16:40:05 +01:00
Finn Womack
0932272431
gh-106242: Fix path truncation in os.path.normpath (GH-106816) 2023-08-15 16:33:00 +01:00
Irit Katriel
608927b014
gh-103082: use IS_VALID_OPCODE instead of _PyOpcode_OpName to check if an opcode is defined (#107882) 2023-08-14 10:51:50 +01:00
Dong-hee Na
bf707749e8
gh-106797: Remove warning logs from Python/generated_cases.c.h and executor_cases.c.h (gh-107889)
gh-106797: Remove warning logs from Python/generated_cases.c.h
2023-08-13 04:36:46 +09:00
Mark Shannon
37d8b904f8
GH-107674: Avoid allocating boxed ints for sys.settrace line events (GH-107780) 2023-08-10 13:35:02 +01:00
Irit Katriel
bafedfbebd
gh-106149: move CFG and basicblock definitions into flowgraph.c, use them as opaque types in compile.c (#107639) 2023-08-10 13:03:47 +01:00
Mark Shannon
494e3d4436
GH-107774: Add missing audit event for PEP 669 (GH-107775) 2023-08-10 12:29:06 +01:00
Brandt Bucher
326f0ba1c5
GH-106485: Dematerialize instance dictionaries when possible (GH-106539) 2023-08-09 19:14:50 +00:00
Brandt Bucher
a9caf9cf90
GH-105848: Simplify the arrangement of CALL's stack (GH-107788) 2023-08-09 18:19:39 +00:00
Mark Shannon
52fbcf61b5
GH-107724: Fix the signature of PY_THROW callback functions. (GH-107725) 2023-08-09 09:30:50 +01:00
Brandt Bucher
ea72c6fe3b
GH-107596: Specialize str[int] (GH-107597) 2023-08-08 13:42:43 -07:00
Guido van Rossum
328d925244
gh-107758: Improvements to lltrace feature (#107757)
- The `dump_stack()` method could call a `__repr__` method implemented in Python,
  causing (infinite) recursion.
  I rewrote it to only print out the values for some fundamental types (`int`, `str`, etc.);
  for everything else it just prints `<type_name @ 0xdeadbeef>`.

- The lltrace-like feature for uops wrote to `stderr`, while the one in `ceval.c` writes to `stdout`;
  I changed the uops to write to stdout as well.
2023-08-07 21:36:25 -07:00
Eric Snow
430632d6f7
gh-107630: Initialize Each Interpreter's refchain Properly (gh-107733)
This finishes fixing the crashes in Py_TRACE_REFS builds.  We missed this part in gh-107567.
2023-08-07 13:14:56 -06:00
Ivin Lee
4e6fac7fcc
gh-106608: make uop trace variable length (#107531)
Executors are now more like tuples.
2023-08-04 21:10:46 -07:00
Brandt Bucher
05a824f294
GH-84436: Skip refcounting for known immortals (GH-107605) 2023-08-04 16:24:50 -07:00
Guido van Rossum
400835ea16
gh-106812: Refactor cases_generator to allow uops with array stack effects (#107564)
Introducing a new file, stacking.py, that takes over several responsibilities related to symbolic evaluation of push/pop operations, with more generality.
2023-08-04 09:35:56 -07:00
Mark Shannon
2ba7c7f7b1
Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Mark Shannon
fa45958450
GH-107263: Increase C stack limit for most functions, except _PyEval_EvalFrameDefault() (GH-107535)
* Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
2023-08-04 10:10:29 +01:00
Eric Snow
58ef741867
gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
2023-08-03 19:51:08 +00:00
Eric Snow
017f047183
gh-107471: Fix Refleaks in test_import (gh-107569)
gh-107184 introduced a refleak in test_import.SubinterpImportTests (specifically test_singlephase_check_with_setting_and_override and test_single_init_extension_compat).  We fix it here by making sure _testsinglephase is removed from sys.modules whenever we clear the runtime's internal state for the module.

The underlying problem is strictly contained in the internal function _PyImport_ClearExtension() (AKA _testinternalcapi.clear_extension()), which is only used in tests.

(This also fixes an intermittent segfault introduced in the same place, in test_disallowed_reimport.)
2023-08-02 20:55:09 +00:00
Mark Shannon
0d30a5a409
GH-100964: Break cycles involving exception state when returning from generator (GH-107563) 2023-08-02 18:44:20 +01:00
Irit Katriel
2bd04d4234
gh-105481: combine regen-opcode-targets with regen-opcode to avoid calculating the specialized opcodes in two places (#107540) 2023-08-01 21:05:48 +01:00
Brandt Bucher
dfb55d9d7f
Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506) 2023-07-31 14:18:38 -07:00
Brandt Bucher
5e584eb704
GH-104584: Fix incorrect uoperands (GH-107513) 2023-07-31 21:16:57 +00:00
Guido van Rossum
5eb80a61f5
GH-104909: Move unused cache entries from uops to macros (#107444)
There's no need to use a dummy uop to skip unused cache entries. The macro syntax lets you write `unused/1` instead.

Similarly, move `unused/5` from op `_LOAD_ATTR_INSTANCE_VALUE` to macro `LOAD_ATTR_INSTANCE_VALUE`.
2023-07-31 08:55:33 -07:00
Georg Brandl
f57b9fd2b6
no-issue: Fix typo in import.c (gh-107498) 2023-07-31 12:07:17 +00:00
Eric Snow
8ba4df91ae
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules.  The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL.  It turns out it's tricky to use the same thread state for different threads.  Who could have guessed?

We solve the problem by eliminating the one object we were still sharing between interpreters.  We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.

We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
2023-07-28 14:39:08 -06:00
Eric Snow
8bdae1424b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)
The _xxsubinterpreters module should not rely on internal API.  Some of the functions it uses were recently moved there however.  Here we move them back (and expose them properly).
2023-07-27 15:30:16 -06:00
Eric Snow
75c974f535
gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184)
This fixes a bug where incompatible modules could still be imported if attempted multiple times.
2023-07-27 15:08:38 -06:00
Eric Snow
b72947a8d2
gh-106931: Intern Statically Allocated Strings Globally (gh-107272)
We tried this before with a dict and for all interned strings.  That ran into problems due to interpreter isolation.  However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning.  Here we circle back to using a global cache, but only for statically allocated strings.  We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.

Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter.  Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
2023-07-27 13:56:59 -06:00
Mark Shannon
ac7a0f858a
GH-106898: Add the exception as an argument to the PY_UNWIND event callback function. (GH-107347) 2023-07-27 15:47:33 +01:00
Victor Stinner
1dbb427dd6
gh-107196: Remove _PyArg_VaParseTupleAndKeywordsFast() function (#107197)
Remove the private _PyArg_VaParseTupleAndKeywordsFast() function: it
is no longer used.
2023-07-27 16:36:54 +02:00
Mark Shannon
c6539b36c1
GH-106895: Raise a ValueError when attempting to disable events that cannot be disabled. (GH-107337) 2023-07-27 15:27:11 +01:00
Mark Shannon
766d2518ae
GH-106897: Add RERAISE event to sys.monitoring. (GH-107291)
* Ensures that exception handling events are balanced. Each [re]raise event has a matching unwind/handled event.
2023-07-27 13:32:30 +01:00
Irit Katriel
507d8bc39a
gh-106149: fix comment on stackdepth of generators (#107321) 2023-07-26 22:31:47 +00:00
Pablo Galindo Salgado
da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer (#107018) 2023-07-26 16:34:15 +01:00