Commit graph

6400 commits

Author SHA1 Message Date
Victor Stinner
310e2d2517
bpo-36854: Fix refleak in subinterpreter (GH-17331)
finalize_interp_clear() now explicitly clears the codec registry and
then trigger a GC collection to clear all references.
2019-11-22 10:58:00 +01:00
Mark Shannon
fee552669f
Produce cleaner bytecode for 'with' and 'async with' by generating separate code for normal and exceptional paths. (#6641)
Remove BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY bytecodes. Implement finally blocks by code duplication.
Reimplement frame.lineno setter using line numbers rather than bytecode offsets.
2019-11-21 09:11:43 +00:00
Victor Stinner
b93f31fcd9
bpo-38858: Fix Py_Finalize() when called from a subinterpreter (GH-17297)
Use _Py_IsMainInterpreter() in Py_Initialize() and Py_Finalize() to
detect if the current interpreter is the main interpreter or not.
2019-11-20 18:39:12 +01:00
Victor Stinner
fff7bbfdb6
bpo-38858: Add _Py_IsMainInterpreter(tstate) (GH-17293) 2019-11-20 17:34:39 +01:00
Victor Stinner
4dedd0f0dd
bpo-37340: Remove PyMethod_ClearFreeList() and PyCFunction_ClearFreeList() (GH-17284)
Remove PyMethod_ClearFreeList() and PyCFunction_ClearFreeList()
functions: the free lists of bound method objects have been removed.

Remove also _PyMethod_Fini() and _PyCFunction_Fini() functions.
2019-11-20 12:59:12 +01:00
Victor Stinner
7247407c35
bpo-36854: Move _PyRuntimeState.gc to PyInterpreterState (GH-17287)
* Rename _PyGC_InitializeRuntime() to _PyGC_InitState()
* finalize_interp_clear() now also calls _PyGC_Fini() in
  subinterpreters (clear the GC state).
2019-11-20 12:25:50 +01:00
Victor Stinner
e7e699e4df
bpo-38858: Fix reference leak in pycore_init_types() (GH-17286)
Only call _PyGC_Init(), _PyExc_Init() and _PyErr_Init() in
new_interpreter().
2019-11-20 12:08:13 +01:00
Victor Stinner
67e0de6f0b
bpo-36854: gcmodule.c gets its state from tstate (GH-17285)
* Add GCState type for readability
* gcmodule.c now gets its gcstate from tstate
* _PyGC_DumpShutdownStats() now expects tstate rather than runtime
* Rename "state" to "gcstate" for readability: to avoid confusion
  between "state" and "tstate" for example.
* collect() now only expects tstate: it gets gcstate from tstate.
* Pass tstate to _PyErr_xxx() functions
2019-11-20 11:48:18 +01:00
Victor Stinner
9da7430675
bpo-36854: Clear the current thread later (GH-17279)
Clear the current thread later in the Python finalization.

* The PyInterpreterState_Delete() function is now responsible
  to call PyThreadState_Swap(NULL).
* The tstate_delete_common() function is now responsible to clear the
  "autoTSSKey" thread local storage and it only clears it once the
  thread state is fully cleared. It allows to still get the current
  thread from TSS in tstate_delete_common().
2019-11-20 11:17:17 +01:00
Victor Stinner
7eee5beaf8
bpo-38858: Factorize Py_EndInterpreter() code (GH-17273)
* Factorize code in common between Py_FinalizeEx() and
  Py_EndInterpreter().
* Py_EndInterpreter() now also calls _PyWarnings_Fini().
* Call _PyExc_Fini() and _PyGC_Fini() later in the finalization.
2019-11-20 10:38:34 +01:00
Victor Stinner
be143ec996
bpo-38835: Don't use PyFPE_START_PROTECT and PyFPE_END_PROTECT (GH-17231)
The PyFPE_START_PROTECT() and PyFPE_END_PROTECT() macros are empty:
they have been doing nothing for the last year  (since commit
735ae8d139), so stop using them.
2019-11-20 02:51:30 +01:00
Victor Stinner
01b1cc12e7
bpo-36710: Add PyInterpreterState.runtime field (GH-17270)
Add PyInterpreterState.runtime field: reference to the _PyRuntime
global variable. This field exists to not have to pass runtime in
addition to tstate to a function.  Get runtime from tstate:
tstate->interp->runtime.

Remove "_PyRuntimeState *runtime" parameter from functions already
taking a "PyThreadState *tstate" parameter.

_PyGC_Init() first parameter becomes "PyThreadState *tstate".
2019-11-20 02:27:56 +01:00
Brandt Bucher
33b671e724 bpo-38823: Fix refleak in marshal init error path (GH-17260) 2019-11-20 01:59:32 +01:00
Victor Stinner
444b39bb64
bpo-38631: Avoid Py_FatalError() in handle_legacy_finalizers() (GH-17266)
* Rename _PyGC_Initialize() to _PyGC_InitializeRuntime()
* Add _PyGC_Init(): initialize _PyRuntime.gc.garbage list
* Call _PyGC_Init() before _PyTypes_Init()
2019-11-20 01:18:11 +01:00
Victor Stinner
ef5aa9af7c
bpo-38858: Reorganize pycore_init_types() (GH-17265)
* Call _PyLong_Init() and _PyExc_Init() earlier
* new_interpreter() reuses pycore_init_types()
2019-11-20 00:38:03 +01:00
Victor Stinner
b5e170f127
bpo-38644: Add _PyEval_EvalCode() (GH-17183)
_PyFunction_Vectorcall() now pass tstate to function calls.
2019-11-16 01:03:22 +01:00
Victor Stinner
4d231bcc77
bpo-38644: Add _PyObject_Call() (GH-17089)
* Add pycore_call.h internal header file.
* Add _PyObject_Call(): PyObject_Call() with tstate
* Add _PyObject_CallNoArgTstate(): _PyObject_CallNoArg() with tstate
* Add _PyObject_FastCallDictTstate(): _PyObject_FastCallDict()
  with tstate
* _PyObject_Call_Prepend() now takes tstate
* Replace _PyObject_FastCall() calls
  with _PyObject_VectorcallTstate() calls
2019-11-14 13:36:21 +01:00
Victor Stinner
b9e681261c
bpo-38644: Add _PyEval_EvalFrame() with tstate (GH-17131)
Add _PyEval_EvalFrame() static inline function to get eval_frame from
tstate->interp.
2019-11-14 12:20:46 +01:00
Victor Stinner
7e43373317
bpo-38644: Add _PyObject_VectorcallTstate() (GH-17052)
* Add _PyObject_VectorcallTstate() function: similar to
  _PyObject_Vectorcall(), but with tstate parameter
* Add tstate parameter to _PyObject_MakeTpCall()
2019-11-08 10:05:17 +01:00
Victor Stinner
d12d0e7c0f
bpo-38733: PyErr_Occurred() caller must hold the GIL (GH-17080)
bpo-3605, bpo-38733: Optimize _PyErr_Occurred(): remove "tstate ==
NULL" test.

Py_FatalError() no longer calls PyErr_Occurred() if called without
holding the GIL. So PyErr_Occurred() no longer has to support
tstate==NULL case.

_Py_CheckFunctionResult(): use directly _PyErr_Occurred() to avoid
explicit "!= NULL" test.
2019-11-07 12:42:07 +01:00
Jeroen Demeyer
bf17d41826 bpo-37645: add new function _PyObject_FunctionStr() (GH-14890)
Additional note: the `method_check_args` function in `Objects/descrobject.c` is written in such a way that it applies to all kinds of descriptors. In particular, a future re-implementation of `wrapper_descriptor` could use that code.

CC @vstinner @encukou 


https://bugs.python.org/issue37645



Automerge-Triggered-By: @encukou
2019-11-05 07:48:04 -08:00
Benjamin Peterson
62161ce989
closes bpo-37633: Reëxport some function compatibility wrappers for macros in `pythonrun.h`. (GH-17056) 2019-11-04 21:34:14 -08:00
Victor Stinner
1726909094
bpo-38644: Pass tstate to _Py_CheckFunctionResult() (GH-17050)
* Add tstate parameter to _Py_CheckFunctionResult()
* Add _PyErr_FormatFromCauseTstate()
* Replace PyErr_XXX(...) with _PyErr_XXX(state, ...)
2019-11-05 01:22:12 +01:00
Victor Stinner
be434dc038
bpo-38644: Pass tstate to Py_EnterRecursiveCall() (GH-16997)
* Add _Py_EnterRecursiveCall() and _Py_LeaveRecursiveCall() which
  require a tstate argument.
* Pass tstate to _Py_MakeRecCheck() and  _Py_CheckRecursiveCall().
* Convert Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() macros
  to static inline functions.

_PyThreadState_GET() is the most efficient way to get the tstate, and
so using it with _Py_EnterRecursiveCall() and
_Py_LeaveRecursiveCall() should be a little bit more efficient than
using Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() which use
the "slower" PyThreadState_GET().
2019-11-05 00:51:22 +01:00
Victor Stinner
f4b1e3d7c6
bpo-38644: Add Py_EnterRecursiveCall() to the limited API (GH-17046)
Provide Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as
regular functions for the limited API. Previously, there were defined
as macros, but these macros didn't work with the limited API which
cannot access PyThreadState.recursion_depth field.

Remove _Py_CheckRecursionLimit from the stable ABI.

Add Include/cpython/ceval.h header file.
2019-11-04 19:48:34 +01:00
Max Bernstein
bdac32e9fe closes bpo-38648: Remove double tp_free slot in Python-ast.c. (GH-17002)
This looks like a typo due to copy-paste.
2019-10-30 18:08:06 -07:00
Pablo Galindo
6c3e66a34b
bpo-38640: Allow break and continue in always false while loops (GH-16992) 2019-10-30 11:53:26 +00:00
Serhiy Storchaka
26ae9f6d3d
bpo-38535: Fix positions for AST nodes for calls without arguments in decorators. (GH-16861) 2019-10-26 16:46:05 +03:00
Hansraj Das
7320ec05f7 Fix typo in formatter_unicode (GH-16831)
numbers's -> number's
2019-10-25 09:44:02 -07:00
Serhiy Storchaka
5bc6a7c06e
bpo-38540: Fix possible leak in PyArg_Parse for "es#" and "et#". (GH-16869) 2019-10-21 11:35:07 +03:00
Hansraj Das
2798b60c55 Typo fix - implemention should be implementation (GH-16806) 2019-10-15 13:49:13 -07:00
Vinay Sajip
0b60f64e43
bpo-11410: Standardize and use symbol visibility attributes across POSIX and Windows. (GH-16347) 2019-10-15 08:26:12 +01:00
Pablo Galindo
fd5c414880
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755)
The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 05:18:05 +01:00
Pablo Galindo
e3babbd03c
Correct signature of __build_class__ (GH-16735) 2019-10-13 16:35:41 +01:00
Hansraj Das
a8e0d3141e Typo fix: "throuhgh" should be "through". (GH-16704) 2019-10-10 20:43:13 -07:00
Dong-hee Na
a05fcd3c7a bpo-38425: Fix ‘res’ may be used uninitialized warning (GH-16688) 2019-10-10 09:41:26 +02:00
Hansraj Das
01171ebd96 Typo fix: "empy" should be "empty". (GH-16666) 2019-10-08 19:24:02 -07:00
Hansraj Das
5dfbb4d503 Fix typo in _warnings.warn_explicit() docstring (GH-16625) 2019-10-08 10:56:07 +02:00
Victor Stinner
c02b41b1fb
bpo-38353: getpath.c: allocates strings on the heap (GH-16585)
* _Py_FindEnvConfigValue() now returns a string allocated
  by PyMem_RawMalloc().
* calculate_init() now decodes VPATH macro.
* Add calculate_open_pyenv() function.
* Add substring() and joinpath2() functions.

* Fix add_exe_suffix()

And a few cleanup changes.
2019-10-04 19:53:43 +02:00
Joannah Nanjekye
8855e47d09 bpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558)
Revert the removal of PyThreadState_DeleteCurrent() with documentation.
2019-10-04 13:35:42 +02:00
Victor Stinner
03a8a56fac
bpo-38353: Add subfunctions to getpath.c (GH-16572)
Following symbolic links is now limited to 40 attempts, just to
prevent loops.

Add subfunctions:

* Add resolve_symlinks()
* Add calculate_argv0_path_framework()
* Add calculate_which()
* Add calculate_program_macos()

Fix also _Py_wreadlink(): readlink() result type is Py_ssize_t, not
int.
2019-10-04 02:22:39 +02:00
Victor Stinner
61691d8336
bpo-38353: Cleanup includes in the internal C API (GH-16548)
Use forward declaration of types to avoid includes in the internal C
API. Add also comment to justify other includes.
2019-10-02 23:51:20 +02:00
Victor Stinner
8462a4936b
bpo-38304: PyConfig_InitPythonConfig() cannot fail anymore (GH-16509)
PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() no
longer return PyStatus: they cannot fail anymore.
2019-10-01 12:06:16 +02:00
Victor Stinner
3c30a76f3d
bpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)
For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.

We may revisit this in the future if we further explore the idea of offering a stable embedding API.

(cherry picked from commit bdace21b76)
2019-10-01 10:56:37 +02:00
Victor Stinner
fb4ae152a9
bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.

* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
  with "_Py".
* test_embed: add test_init_warnoptions().
2019-09-30 01:40:17 +02:00
Brandt Bucher
f185a73249 bpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467) 2019-09-28 17:12:49 -07:00
Victor Stinner
441b10cf28
bpo-38304: Add PyConfig.struct_size (GH-16451)
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
2019-09-28 04:28:35 +02:00
Victor Stinner
12f2f177fc
bpo-38234: Py_Initialize() sets global path configuration (GH-16421)
* Py_InitializeFromConfig() now writes PyConfig path configuration to
  the global path configuration (_Py_path_config).
* Add test_embed.test_get_pathconfig().
* Fix typo in _PyWideStringList_Join().
2019-09-26 15:51:50 +02:00
Victor Stinner
8bf39b606e
bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
  with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
  Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import  and base_executable
  fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
  calculated base_executable: _PyConfig_InitPathConfig() copies
  executable to base_executable.
* Complete path config documentation.
2019-09-26 02:22:35 +02:00
Victor Stinner
1ce152a42e
bpo-38234: Py_SetPath() uses the program full path (GH-16357)
Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).

Fix also memory leaks in pathconfig_set_from_config().
2019-09-24 17:44:15 +02:00