Commit graph

1162 commits

Author SHA1 Message Date
Mark Shannon
a519b87958
GH-111848: Convert remaining jumps to deopts into tier 2 code. (GH-112045) 2023-11-14 15:30:33 +00:00
Victor Stinner
4f04172c92
gh-111262: Add PyDict_Pop() function (#112028)
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-11-14 12:51:00 +00:00
Irit Katriel
36aab34fab
gh-107149: make new opcode util functions private rather than public and unstable (#112042) 2023-11-14 00:31:02 +00:00
Serhiy Storchaka
771bd3c94a
Add private _PyUnicode_AsUTF8NoNUL() function (GH-111957)
Like PyUnicode_AsUTF8(), but check for embedded null characters.
2023-11-10 21:31:36 +02:00
Mark Shannon
25c4956488
GH-109369: Exit tier 2 if executor is invalid (GH-111657) 2023-11-09 11:19:51 +00:00
Irit Katriel
30ec968bef
gh-111354: remove comparisons with enum values, variable reuse, unused imports in genobject.c (#111708) 2023-11-09 10:27:20 +00:00
Sam Gross
31c90d5838
gh-111569: Implement Python critical section API (gh-111571)
Critical sections are helpers to replace the global interpreter lock
with finer grained locking.  They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves.  Critical
sections are implicitly ended whenever the GIL would be released.  They
are resumed when the GIL would be acquired.  Nested critical sections
behave as if the sections were interleaved.
2023-11-08 15:39:29 -07:00
Mark Shannon
06efb60264
GH-111848: Tidy up tier 2 handling of FOR_ITER specialization by using DEOPT_IF instead of jumps. (GH-111849) 2023-11-08 13:31:55 +00:00
Mark Shannon
931f4438c9
GH-111485: Allow arbitrary annotations on instructions and micro-ops. (GH-111697) 2023-11-07 09:42:39 +00:00
Brandt Bucher
3e99c9cbf6
GH-111485: Make BEFORE_WITH a uop (GH-111812) 2023-11-06 16:42:49 -08:00
Eric Snow
d4426e8d00
gh-76785: Move _Py_excinfo Functions Out of the Internal C-API (gh-111715)
I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in gh-111530 (9322ce9).  Since then I've had a nagging sense that I should have added the type and functions in its own PR.  While I do plan on using _Py_excinfo outside crossinterp.c very soon (see gh-111572/gh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR.  Hence, here we are.

(FWIW, I may combine that with gh-111572, which I may, in turn, combine with gh-111573.  We'll see.)
2023-11-06 11:09:22 -07:00
Antoine Pitrou
0e9c364f4a
GH-110829: Ensure Thread.join() joins the OS thread (#110848)
Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-11-04 13:59:24 +00:00
Tian Gao
e0afed7e27
gh-103615: Use local events for opcode tracing (GH-109472)
* Use local monitoring for opcode trace

* Remove f_opcode_trace_set

* Add test for setting f_trace_opcodes after settrace
2023-11-03 16:39:50 +00:00
Michael Droettboom
2bc01cc0c7
gh-111652: Fix --enable-pystats build (GH-111653) 2023-11-03 15:21:16 +00:00
scoder
24ddaee5ca
gh-106168: Revert the "size before item" setting (#111683)
gh-106168: Update the size only after setting the item, to avoid temporary inconsistencies.
Also remove the "what's new" sentence regarding the size setting since tuples cannot grow after allocation.
2023-11-03 11:02:39 +00:00
Irit Katriel
d49aba5a7a
gh-111354: Simplify _PyGen_yf by moving some of its work to the compiler and frame state (#111648) 2023-11-03 10:01:36 +00:00
Serhiy Storchaka
26c0e5e03a
gh-108082: Remove _PyErr_WriteUnraisableMsg() (GH-111643)
Replace the remaining calls with PyErr_FormatUnraisable().
2023-11-03 09:45:53 +02:00
Irit Katriel
52cc4af6ae
gh-111354: simplify detection of RESUME after YIELD_VALUE at except-depth 1 (#111459) 2023-11-02 10:18:43 +00:00
Eric Snow
9322ce90ac
gh-76785: Crossinterp utils additions (gh-111530)
This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers).

Specifically:

* _Py_excinfo, etc.:  the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c)
* _PyXI_exception_info, etc.:  helpers for passing an exception beween interpreters (wraps _Py_excinfo)
* _PyXI_namespace, etc.:  helpers for copying a dict of attrs between interpreters
* _PyXI_Enter(), _PyXI_Exit():  functions that abstract out the transitions between one interpreter and a second that will do some work temporarily

Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations.  I plan on proposing these as public API at some point.
2023-11-01 17:36:40 -06:00
Mark Shannon
b14e882428
GH-111485: Use micro-ops to split specialization code from base action (GH-111561) 2023-11-01 10:53:27 +00:00
Mark Shannon
2904d99839
GH-111485: Remove some special cases from the code generator and bytecodes.c (GH-111540) 2023-10-31 13:21:07 +00:00
Mark Shannon
d27acd4461
GH-111485: Increment next_instr consistently at the start of the instruction. (GH-111486) 2023-10-31 10:09:54 +00:00
Michael Droettboom
84b4533e84
gh-109329: Count tier2 opcode misses (#110561)
This keeps a separate 'miss' counter for each micro-opcode, incremented whenever a guard uop takes a deoptimization side exit.
2023-10-30 17:02:45 -07:00
Eric Snow
c6fe0869ab
gh-76785: Move the Cross-Interpreter Code to Its Own File (gh-111502)
This is partly to clear this stuff out of pystate.c, but also in preparation for moving some code out of _xxsubinterpretersmodule.c.  This change also moves this stuff to the internal API (new: Include/internal/pycore_crossinterp.h).  @vstinner did this previously and I undid it.  Now I'm re-doing it. :/
2023-10-30 16:53:10 -06:00
Victor Stinner
801741ff81
gh-90815: Fix mimalloc atomic.h on Windows arm64 (#111527)
mi_atomic_load_explicit() casts 'p' argument to drop the 'const'
qualifier on Windows arm64 platform. Fix the compiler warning:

    'function': different 'const' qualifiers
    (compiling source file ..\Objects\mimalloc\options.c)
2023-10-30 22:33:49 +00:00
Sam Gross
6dfb8fe023
gh-110481: Implement biased reference counting (gh-110764) 2023-10-30 16:06:09 +00:00
Dino Viehland
05f2f0ac92
gh-90815: Add mimalloc memory allocator (#109914)
* Add mimalloc v2.12

Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.

Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c
 - src/region.c

mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.

* mimalloc: minimal changes for use in Python:

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
  https://github.com/python/cpython/pull/31164/

Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-30 15:43:11 +00:00
Savannah Ostrowski
4a929d432b
GH-111339: Fix initialization and finalization of static optimizer types (GH-111430) 2023-10-29 13:53:25 -07:00
gsallam
21f068d80c
gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines (#109666) 2023-10-27 03:57:29 +00:00
Irit Katriel
a0c414c35d
gh-111354: define names for RESUME oparg values (#111365) 2023-10-26 16:30:18 +01:00
Irit Katriel
67a91f78e4
gh-109094: replace frame->prev_instr by frame->instr_ptr (#109095) 2023-10-26 13:43:10 +00:00
Pablo Galindo Salgado
90a1b2859f
gh-67224: Show source lines in tracebacks when using the -c option when running Python (#111200) 2023-10-26 15:17:28 +09:00
scoder
a8a89fcd1f
gh-106320: Re-add some PyLong/PyDict C-API functions (GH-#111162)
* gh-106320: Re-add _PyLong_FromByteArray(), _PyLong_AsByteArray() and _PyLong_GCD() to the public header files since they are used by third-party packages and there is no efficient replacement.

See https://github.com/python/cpython/issues/111140
See https://github.com/python/cpython/issues/111139

* gh-111262: Re-add _PyDict_Pop() to have a C-API until a new public one is designed.
2023-10-25 11:33:48 +02:00
Brandt Bucher
e5168ff3f8
GH-109214: _SET_IP before _PUSH_FRAME (but not _POP_FRAME) (GH-111001) 2023-10-24 13:27:42 -07:00
Radislav Chugunov
47d3e2ed93
gh-109894: Fix initialization of static MemoryError in subinterpreter (gh-110911)
Fixes #109894

* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-10-23 17:06:59 -06:00
Mark Shannon
52e902ccf0
GH-109369: Add machinery for deoptimizing tier2 executors, both individually and globally. (GH-110384) 2023-10-23 14:49:09 +01:00
Eric Snow
c58c63fdf6
gh-84570: Add Timeouts to SendChannel.send() and RecvChannel.recv() (gh-110567) 2023-10-17 23:05:49 +00:00
Eric Snow
a53d7cb672
gh-84570: Send-Wait Fixes for _xxinterpchannels (gh-111006)
There were a few things I did in gh-110565 that need to be fixed. I also forgot to add tests in that PR.

(Note that this PR exposes a refleak introduced by gh-110246. I'll take care of that separately.)
2023-10-17 16:32:00 -06:00
Donghee Na
2dcc57008b
gh-109693: Remove pycore_atomic.h (gh-110992) 2023-10-18 00:33:50 +09:00
Victor Stinner
6db6b30ac2
gh-85283: Build winsound extension with limited C API (#110978)
Replace type->tp_name with PyType_GetQualName().
2023-10-17 15:57:10 +02:00
Victor Stinner
be5e8a0103
gh-110964: Remove private _PyArg functions (#110966)
Move the following private functions and structures to
pycore_modsupport.h internal C API:

* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()

Changes:

* Python/getargs.h now includes pycore_modsupport.h to export
  functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
  is used.
* Add pycore_modsupport.h includes when a C extension uses one of
  these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
  directly or indirectly (via code generated by Argument Clinic)
  pycore_modsupport.h:

  * _csv
  * _curses_panel
  * _dbm
  * _gdbm
  * _multiprocessing.posixshmem
  * _sqlite.row
  * _statistics
  * grp
  * resource
  * syslog

* _testcapi: bad_get() no longer uses METH_FASTCALL calling
  convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
  PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
  by _testcapi sub-modules which need the internal C API
  (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
  watchers.c.
* Remove Include/cpython/modsupport.h header file.
  Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
2023-10-17 14:30:31 +02:00
Donghee Na
86559ddfec
gh-109693: Update _gil_runtime_state.locked to use pyatomic.h (gh-110836) 2023-10-17 07:32:50 +09:00
Donghee Na
b2ab210aae
gh-109693: Update pyruntimestate._finalizing to use pyatomic.h (gh-110837) 2023-10-13 16:40:15 +00:00
Pablo Galindo Salgado
e1d8c65e1d
gh-110805: Allow the repl to show source code and complete tracebacks (#110775) 2023-10-13 09:25:37 +00:00
Donghee Na
2566434e59
gh-109693: Update _gil_runtime_state.last_holder to use pyatomic.h (#110605) 2023-10-13 10:07:27 +09:00
Pablo Galindo Salgado
e7331365b4
gh-110721: Use the traceback module for PyErr_Display() and fallback to the C implementation (#110702) 2023-10-12 14:52:14 +00:00
Irit Katriel
7dd3c2b800
gh-109094: remove redundant arg to _PyFrame_PushTrampolineUnchecked (GH-110759) 2023-10-12 11:02:42 +01:00
Mark Shannon
19b7ead5eb
GH-109214: Convert _SAVE_CURRENT_IP to _SET_IP in tier 2 trace creation. (GH-110755) 2023-10-12 10:34:32 +01:00
Donghee Na
5bc1b7f08d
gh-109693: Update pycore_interp.h to use pyatomic.h (#110604) 2023-10-10 23:17:08 +09:00
Donghee Na
67e8d416cc
gh-109693: Use pyatomic.h for signal module (gh-110480) 2023-10-10 08:26:29 +09:00