Commit graph

4380 commits

Author SHA1 Message Date
Kumar Aditya
dc3e4350a5
GH-99205: remove _static field from PyThreadState and PyInterpreterState (GH-99385) 2022-11-14 16:35:37 -08:00
Eric Snow
e874c2f198
gh-81057: Move the Remaining Import State Globals to _PyRuntimeState (gh-99488)
https://github.com/python/cpython/issues/81057
2022-11-14 15:56:16 -07:00
Eric Snow
a088290f9d
gh-81057: Move Global Variables Holding Objects to _PyRuntimeState. (gh-99487)
This moves nearly all remaining object-holding globals in core code (other than static types).

https://github.com/python/cpython/issues/81057
2022-11-14 13:50:56 -07:00
Irit Katriel
a3ac9232f8
gh-87092: expose the compiler's codegen to python for unit tests (GH-99111) 2022-11-14 13:56:40 +00:00
Erlend E. Aasland
c95f554a40
gh-83638: Add sqlite3.Connection.autocommit for PEP 249 compliant behaviour (#93823)
Introduce the autocommit attribute to Connection and the autocommit
parameter to connect() for PEP 249-compliant transaction handling.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Géry Ogam <gery.ogam@gmail.com>
2022-11-12 23:44:41 +01:00
Eric Snow
7f3a4b967c
gh-81057: Move PyImport_Inittab to _PyRuntimeState (gh-99402)
We actually don't move PyImport_Inittab.  Instead, we make a copy that we keep on _PyRuntimeState and use only that after Py_Initialize().  We also prevent folks from modifying PyImport_Inittab (the best we can) after that point.

https://github.com/python/cpython/issues/81057
2022-11-11 17:06:05 -07:00
Eric Snow
67807cfc87
gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)
The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject.  State for the "small block" allocator was stored in another 13.  That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057.  (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.)

https://github.com/python/cpython/issues/81057
2022-11-11 16:30:46 -07:00
Eric Snow
f531b6879b
gh-81057: Add PyInterpreterState.static_objects (gh-99397)
As we consolidate global variables, we find some objects that are almost suitable to add to _PyRuntimeState.global_objects, but have some small/sneaky bit of per-interpreter state (e.g. a weakref list). We're adding PyInterpreterState.static_objects so we can move such objects there. (We'll removed the _not_used field once we've added others.)

https://github.com/python/cpython/issues/81057
2022-11-11 14:24:18 -07:00
Eric Snow
dd36b71fa6
gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)
We also move the closely related max_module_number and add comments documenting the group of struct members.

https://github.com/python/cpython/issues/81057
2022-11-11 14:16:28 -07:00
Eric Snow
fe55ff3f68
gh-81057: Generate a Separate Initializer For Each Part of the Global Objects Initializer (gh-99389)
Up until now we had a single generated initializer macro for all the statically declared global objects in _PyRuntimeState, including several one-offs (e.g. the empty tuple). The one-offs don't need to be generated, but were because we had one big initializer. Having separate initializers for set of generated global objects allows us to generate only the ones we need to.  This allows us to add initializers for one-off global objects without having to generate them.

https://github.com/python/cpython/issues/81057
2022-11-11 13:23:41 -07:00
Mark Shannon
1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Brandt Bucher
c7f5708714
GH-98686: Get rid of "adaptive" and "quick" instructions (GH-99182) 2022-11-09 10:50:09 -08:00
Kumar Aditya
6e3cc72afe
GH-90699: disallow _Py_IDENTIFIER in core code (GH-99210) 2022-11-09 08:53:21 -08:00
Victor Stinner
c03e05c2e7
gh-98724: Fix Py_CLEAR() macro side effects (#99100)
The Py_CLEAR(), Py_SETREF() and Py_XSETREF() macros now only evaluate
their argument once. If an argument has side effects, these side
effects are no longer duplicated.

Add test_py_clear() and test_py_setref() unit tests to _testcapi.
2022-11-09 14:06:36 +01:00
Victor Stinner
0124b5dd28
gh-90868: Add _PyStaticObject_CheckRefcnt() function (#99261)
Add _PyStaticObject_CheckRefcnt() function to make
_PyStaticObjects_CheckRefcnt() shorter. Use
_PyObject_ASSERT_FAILED_MSG() to log the object causing the fatal
error.
2022-11-09 08:40:40 +01:00
Eric Snow
52f91c642b
gh-90868: Adjust the Generated Objects (gh-99223)
We do the following:

* move the generated _PyUnicode_InitStaticStrings() to its own file
* move the generated _PyStaticObjects_CheckRefcnt() to its own file
* include pycore_global_objects.h in extension modules instead of pycore_runtime_init.h

These changes help us avoid including things that aren't needed.

https://github.com/python/cpython/issues/90868
2022-11-08 10:03:03 -07:00
Victor Stinner
4d5fcca273
gh-91248: Add PyFrame_GetVar() function (#95712)
Add PyFrame_GetVar() and PyFrame_GetVarString() functions to get a
frame variable by its name.

Move PyFrameObject C API tests from test_capi to test_frame.
2022-11-08 17:40:27 +01:00
Kumar Aditya
be0d5008b3
GH-90699: Remove remaining _Py_IDENTIFIER stdlib usage (GH-99067) 2022-11-07 12:06:23 -08:00
Mark Shannon
4a1c58d504
GH-96793: Specialize FOR_ITER for generators. (GH-98772) 2022-11-07 14:49:51 +00:00
Pablo Galindo Salgado
99e2e60cb2
gh-99139: Improve NameError error suggestion for instances (#99140) 2022-11-06 13:52:06 +00:00
Kaushik Kulkarni
67ade403a2
gh-98284: better error message for undefined abstractmethod (#97971) 2022-11-05 09:31:57 -07:00
Kumar Aditya
0ee59a9ca3
GH-90699: Remove _Py_IDENTIFIER usage from _ctypes (GH-99054) 2022-11-03 13:20:10 -07:00
Mark Shannon
f4adb97506
GH-96793: Implement PEP 479 in bytecode. (GH-99006)
* Handle converting StopIteration to RuntimeError in bytecode.

* Add custom instruction for converting StopIteration into RuntimeError.
2022-11-03 11:38:51 +00:00
Brandt Bucher
276d77724f
GH-98686: Quicken everything (GH-98687) 2022-11-02 10:42:57 -07:00
Kumar Aditya
18fc232e07
GH-90699: Remove _Py_IDENTIFIER usage from _asyncio module (#99010) 2022-11-02 10:16:06 -07:00
Kumar Aditya
780757ac58
GH-90699: Remove _Py_IDENTIFIER usage from _json module (GH-98956) 2022-11-02 09:03:38 -07:00
Irit Katriel
6d683d8525
gh-87092: do not allocate PyFutureFeatures dynamically (GH-98913) 2022-11-02 15:13:07 +00:00
Eric Snow
4702552885
gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)
Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads.  By default, we were disallowing all three for "isolated" interpreters.  We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API.

Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads.  The default for "isolated" interpreters disables fork, exec, and daemon threads.  Regular threads are allowed by default.  We continue always allowing everything For the main interpreter and the legacy API.

In the code, we add `_PyInterpreterConfig.allow_exec` and  `_PyInterpreterConfig.allow_daemon_threads`.  We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
2022-10-31 12:35:54 -07:00
David Hewitt
e98923c0be
gh-98410: move getbufferproc and releasebufferproc to buffer.h (#31158)
This adds them to the Limited API.
2022-10-31 15:01:32 +01:00
Irit Katriel
39448adc9d
gh-98811: use full source location to simplify __future__ imports error checking. This also fixes an incorrect error offset. (GH-98812) 2022-10-31 13:08:03 +00:00
Nikita Sobolev
76f989dc3e
gh-98783: Fix crashes when str subclasses are used in _PyUnicode_Equal (#98806) 2022-10-30 02:23:20 -04:00
Mark Shannon
22863df7ca
GH-96793: Change FOR_ITER to not pop the iterator on exhaustion. (GH-96801)
Change FOR_ITER to have the same stack effect regardless of whether it branches or not.
Performance is unchanged as FOR_ITER (and specialized forms jump over the cleanup code).
2022-10-27 11:55:03 +01:00
Wenzel Jakob
e60892f9db
gh-98586: Add vector call APIs to the Limited API (GH-98587)
Expose the facilities for making vector calls through Python's limited API.
2022-10-27 11:45:42 +02:00
Eric Snow
f32369480d
gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() (gh-98609)
(see https://github.com/python/cpython/issues/98608)

This change does the following:

1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig`  instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
   * allow_fork
   * allow_subprocess
   * allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`

The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three.  None of that changes here; the defaults are preserved.

Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved.  This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`.  I'd rather just avoid that mess from the start for `_PyInterpreterConfig`.  We can preserve it later if we find an actual need.

This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).

(Note that this PR adds "private" symbols.  We'll probably make them public, and add docs, in a separate change.)
2022-10-26 11:16:30 -06:00
Pablo Galindo Salgado
7cfbb49fcd
gh-91058: Add error suggestions to 'import from' import errors (#98305) 2022-10-25 23:56:59 +01:00
Thomas Wouters
286549b7cc Post 3.12.0a1 2022-10-25 04:17:18 +02:00
Thomas Wouters
4ae1a0ecaf Python 3.12.0a1 2022-10-25 00:08:22 +02:00
Carl Meyer
82ccbf69a8
gh-91051: allow setting a callback hook on PyType_Modified (GH-97875) 2022-10-21 14:41:51 +01:00
Noam Cohen
a371a7e03e
gh-95023: Added os.setns and os.unshare functions (#95046)
Added os.setns and os.unshare to easily switch between namespaces
on Linux.

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Victor Stinner <vstinner@python.org>
2022-10-20 11:08:54 +02:00
Eric Snow
9c8dde0fa5
gh-98417: Store int_max_str_digits on the Interpreter State (GH-98418) 2022-10-19 13:27:46 -07:00
Victor Stinner
1863302d61
gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Ken Jin
b399115ef1
gh-95756: Lazily created cached co_* attrs (GH-97791) 2022-10-11 11:26:08 +08:00
Noam Cohen
5405537813
gh-95011: Migrate syslog module to Argument Clinic (GH-95012) 2022-10-08 21:31:57 +03:00
Pablo Galindo Salgado
83eb827247
gh-97922: Run the GC only on eval breaker (#97920) 2022-10-08 07:57:09 -07:00
Carl Meyer
e82d977eb0
gh-91052: Add PyDict_Unwatch for unwatching a dictionary (#98055) 2022-10-07 17:37:46 -07:00
Nikita Sobolev
24a6645894
gh-97955: Migrate zoneinfo to Argument Clinic (#97958) 2022-10-07 11:06:23 -07:00
zikcheng
24a4b34158
Make _symtable_entry.ste_type's comment consistent wit _Py_block_ty (#92414)
_Py_block_ty defines four types of block, FunctionBlock, ClassBlock, ModuleBlock and AnnotationBlock.
But _symtable_entry.ste_type only comments three of them, I think it's better both sides are consistent.
2022-10-07 10:53:07 -07:00
Nikita Sobolev
83cbe84dc2
gh-64373: Convert _functools to Argument Clinic (#96640) 2022-10-07 10:36:40 -07:00
Carl Meyer
a4b7794887
GH-91052: Add C API for watching dictionaries (GH-31787) 2022-10-07 01:08:00 +01:00
Mark Shannon
76449350b3
GH-91079: Decouple C stack overflow checks from Python recursion checks. (GH-96510) 2022-10-05 01:34:03 +01:00