Commit graph

10969 commits

Author SHA1 Message Date
Guido van Rossum
495da29225 bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975
2019-03-07 12:38:08 -08:00
Davide Rizzo
dc078947a5 bpo-36139: Fix mmap_object_dealloc(): hold the GIL to call PyMem_Free() (GH-12199) 2019-03-06 18:08:31 +01:00
Davide Rizzo
bb9593af0a closes bpo-36139: release GIL around munmap(). (GH-12073) 2019-03-06 07:52:34 -08:00
Emmanuel Arias
b71e28ea91 bpo-36209: Fix typo on hashlib error message (GH-12194) 2019-03-06 15:35:35 +01:00
Victor Stinner
c656e25667
bpo-36142: Add _PyPreConfig_SetAllocator() (GH-12187)
* _PyPreConfig_Write() now reallocates the pre-configuration with the
  new memory allocator.
* It is no longer needed to force the "default raw memory allocator"
  to clear pre-configuration and core configuration. Simplify the
  code.
* _PyPreConfig_Write() now does nothing if called after
  Py_Initialize(): no longer check if the allocator is the same.
* Remove _PyMem_GetDebugAllocatorsName(): dev mode sets again
  allocator to "debug".
2019-03-06 01:13:43 +01:00
Victor Stinner
7d2ef3ef50
bpo-36142: _PyPreConfig_Write() sets the allocator (GH-12186)
* _PyPreConfig_Write() now sets the memory allocator.
* _PyPreConfig_Write() gets a return type: _PyInitError.
* _Py_InitializeCore() now reads and writes the pre-configuration
  (set the memory allocator, configure the locale) before reading and
  writing the core configuration.
2019-03-06 00:36:56 +01:00
Serhiy Storchaka
adfffc7343
Fix the C function signature for _collections._tuplegetter.__reduce__. (GH-12180)
Correctly fixes bpo-36197.
2019-03-05 18:41:09 +02:00
Stéphane Wirtel
359a2f3dab bpo-33012: Fix compilation warnings in memoryobject.c and _collectionsmodule.c (GH-12179)
Cast function pointers to (void(*)(void)) before casting to (PyCFunction)
to make "warning: cast between incompatible function types" false alarm quiet.
2019-03-05 16:10:53 +01:00
Victor Stinner
6dcb54228e
bpo-36142: Add _PyPreConfig_ReadFromArgv() (GH-12173)
The new function is now responsible to parse -E and -I command line
arguments.
2019-03-05 02:44:12 +01:00
Victor Stinner
cad1f747da
bpo-36142: Add _PyPreConfig structure (GH-12172)
* Add _PyPreConfig structure
* Move 'ignored' and 'use_environment' fields from _PyCoreConfig
  to _PyPreConfig
* Add a new "_PyPreConfig preconfig;" field to _PyCoreConfig
2019-03-05 02:01:27 +01:00
Christian Heimes
b7bc283ab6 bpo-36179: Fix ref leaks in _hashopenssl (GH-12158)
Fix two unlikely reference leaks in _hashopenssl. The leaks only occur in
out-of-memory cases. Thanks to Charalampos Stratakis.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue36179
2019-03-04 07:45:41 -08:00
Victor Stinner
4d61e6e3b8
Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f21.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf65.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a2c.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967e2.
2019-03-04 14:21:28 +01:00
Inada Naoki
8c17d928eb add missing break statement (GH-12147)
Bug introduced by 848037c.
2019-03-03 08:22:39 -08:00
ngie-eign
848037c147 Use names SEEK_SET, etc instead of magic number (GH-12057)
The previous code hardcoded `SEEK_SET`, etc. While it's very unlikely
that these values will change, it's best to use the definitions to avoid
there being mismatches in behavior with the code in the future.

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
2019-03-03 16:28:26 +09:00
Eric Snow
bcfa450f21
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003) 2019-03-01 16:50:31 -07:00
Victor Stinner
95e2cbf32f
bpo-36142: Move command line parsing to coreconfig.c (GH-12123)
* Add _PyCoreConfig_ReadFromArgv() function which parses command line
  options: move code from main.c to coreconfig.c.
* Add _PyCoreConfig_Write() to write the new configuration: coerce
  the LC_CTYPE locale, set Py_xxx global configuration variables,
  etc.
* _PyCoreConfig_ReadFromArgv() now only changes the LC_CTYPE locale
  temporarily. _PyCoreConfig_Write() becomes responsible to set the
  LC_CTYPE locale.
* Add _Py_SetArgcArgv() and _Py_ClearArgcArgv() functions
* Rename many "pymain_xxx()" functions
* Add "const" to some function parameters
* Reorganize main.c to declare functions in the order in which they
  are called.
2019-03-01 16:25:19 +01:00
Victor Stinner
62be763348
bpo-36142: Remove _PyMain structure (GH-12120)
* Move fields from _PyMain to _PyCoreConfig:

  * skip_first_line
  * run_command
  * run_module
  * run_filename

* Replace _PyMain.stdin_is_interactive with a new
  stdin_is_interactive(config) function
* Rename _PyMain to _PyArgv. Add "const _PyArgv *args" field
  to _PyCmdline.
2019-03-01 13:10:14 +01:00
Victor Stinner
dfe884759d
bpo-36142: Rework error reporting in pymain_main() (GH-12113)
Add a new _Py_INIT_EXIT() macro to be able to exit Python with an
exitcode using _PyInitError API. Rewrite function calls by
pymain_main() to use _PyInitError.

Changes:

* Remove _PyMain.err and _PyMain.status field
* Add _Py_INIT_EXIT() macro and _PyInitError.exitcode field.
* Rename _Py_FatalInitError() to _Py_ExitInitError().
2019-03-01 12:14:41 +01:00
Victor Stinner
f684d83d86
bpo-36142: Exclude coreconfig.h from Py_LIMITED_API (GH-12111)
The whole coreconfig.h header is now excluded from Py_LIMITED_API.
Move functions definitions into a new internal pycore_coreconfig.h
header.

* Move Include/coreconfig.h to Include/cpython/coreconfig.h
* coreconfig.h header is now excluded from Py_LIMITED_API
* Move functions to pycore_coreconfig.h
2019-03-01 03:44:13 +01:00
Dima Pasechnik
f57cd8288d bpo-36106: Resolve sinpi name clash with libm (IEEE-754 violation). (GH-12027)
The standard math library (libm) may follow IEEE-754 recommendation to
include an implementation of sinPi(), i.e. sinPi(x):=sin(pi*x).
And this triggers a name clash, found by FreeBSD developer
Steve Kargl, who worken on putting sinpi into libm used on FreeBSD
(it has to be named "sinpi", not "sinPi", cf. e.g.
https://en.cppreference.com/w/c/experimental/fpext4).
2019-02-26 08:36:11 +02:00
Zackery Spytz
6673decfa0 bpo-24643: Fix "#define timezone _timezone" clashes on Windows (GH-12019) 2019-02-25 15:56:44 -08:00
Sergey Fedoseev
234531b446 bpo-36030: Add _PyTuple_FromArray() function (GH-11954) 2019-02-25 17:59:12 +01:00
Serhiy Storchaka
a24107b04c
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112) 2019-02-25 17:59:46 +02:00
Serhiy Storchaka
6a44f6eef3
bpo-36048: Use __index__() instead of __int__() for implicit conversion if available. (GH-11952)
Deprecate using the __int__() method in implicit conversions of Python
numbers to C integers.
2019-02-25 17:57:58 +02:00
Eric Snow
ef4ac967e2
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)
This involves moving the global "pending calls" state to PyInterpreterState.

https://bugs.python.org/issue33608
2019-02-24 15:40:47 -08:00
animalize
463572c8be bpo-36101: remove non-ascii characters in docstring (GH-12018)
* remove non-ascii characters in docstring
* clinic.py emits a warning when docstring has non-ascii character
2019-02-25 08:18:48 +09:00
Davin Potts
e895de3e7f
bpo-35813: Tests and docs for shared_memory (#11816)
* Added tests for shared_memory submodule.

* Added tests for ShareableList.

* Fix bug in allocationn size during creation of empty ShareableList illuminated by existing test run on Linux.

* Initial set of docs for shared_memory module.

* Added docs for ShareableList, added doctree entry for shared_memory submodule, name refactoring for greater clarity.

* Added examples to SharedMemoryManager docs, for ease of documentation switched away from exclusively registered functions to some explicit methods on SharedMemoryManager.

* Wording tweaks to docs.

* Fix test failures on Windows.

* Added tests around SharedMemoryManager.

* Documentation tweaks.

* Fix inappropriate test on Windows.

* Further documentation tweaks.

* Fix bare exception.

* Removed __copyright__.

* Fixed typo in doc, removed comment.

* Updated SharedMemoryManager preliminary tests to reflect change of not supporting all registered functions on SyncManager.

* Added Sphinx doctest run controls.

* CloseHandle should be in a finally block in case MapViewOfFile fails.

* Missed opportunity to use with statement.

* Switch to self.addCleanup to spare long try/finally blocks and save one indentation, change to use decorator to skip test instead.

* Simplify the posixshmem extension module.

Provide shm_open() and shm_unlink() functions.  Move other
functionality into the shared_memory.py module.

* Added to doc around size parameter of SharedMemory.

* Changed PosixSharedMemory.size to use os.fstat.

* Change SharedMemory.buf to a read-only property as well as NamedSharedMemory.size.

* Marked as provisional per PEP411 in docstring.

* Changed SharedMemoryTracker to be private.

* Removed registered Proxy Objects from SharedMemoryManager.

* Removed shareable_wrap().

* Removed shareable_wrap() and dangling references to it.

* For consistency added __reduce__ to key classes.

* Fix for potential race condition on Windows for O_CREX.

* Remove unused imports.

* Update access to kernel32 on Windows per feedback from eryksun.

* Moved kernel32 calls to _winapi.

* Removed ShareableList.copy as redundant.

* Changes to _winapi use from eryksun feedback.

* Adopt simpler SharedMemory API, collapsing PosixSharedMemory and WindowsNamedSharedMemory into one.

* Fix missing docstring on class, add test for ignoring size when attaching.

* Moved SharedMemoryManager to managers module, tweak to fragile test.

* Tweak to exception in OpenFileMapping suggested by eryksun.

* Mark a few dangling bits as private as suggested by Giampaolo.
2019-02-23 22:08:16 -06:00
E. M. Bray
abe5922743 fix _abc.c compile error on Cygwin (GH-8445) 2019-02-24 10:30:14 +09:00
Eric Snow
64d6cc826d
bpo-35724: Explicitly require the main interpreter for signal-handling. (GH-11530)
Ensure that the main interpreter is active (in the main thread) for signal-handling operations. This is increasingly relevant as people use subinterpreters more.

https://bugs.python.org/issue35724
2019-02-23 15:40:43 -07:00
Eric Snow
be3b295838
bpo-35886: Make PyInterpreterState an opaque type in the public API. (GH-11731)
Move PyInterpreterState into the "internal" header files.
2019-02-23 11:35:52 -07:00
Pablo Galindo
175421b58c bpo-36016: Add generation option to gc.getobjects() (GH-11909) 2019-02-23 12:02:06 +09:00
Joe Jevnik
f36f89257b bpo-36068: Make _tuplegetter objects serializable (GH-11981) 2019-02-21 13:00:40 -08:00
tjb900
4371c0a9c0 bpo-34572: change _pickle unpickling to use import rather than retrieving from sys.modules (GH-9047)
Fix C implementation of pickle.loads to use importlib's locking mechanisms, and thereby avoid using partially-loaded modules.
2019-02-18 15:30:51 +00:00
animalize
4a7f44a2ed bpo-34294: re module, fix wrong capturing groups in rare cases. (GH-11546)
Need to reset capturing groups between two SRE(match) callings in loops, this fixes wrong capturing groups in rare cases.

Also add a missing index in re.rst.
2019-02-18 15:26:37 +02:00
Pablo Galindo
09fbcd6085
bpo-35942: Improve the error message if __fspath__ returns invalid types in path_converter (GH-11831)
The error message emitted when returning invalid types from __fspath__ in interfaces that allow passing PathLike objects has been improved and now it does explain the origin of the error.
2019-02-18 10:46:34 +00:00
Michael Felt
1bf8845f74 bpo-34720: Fix test_importlib.test_bad_traverse for AIX (GH-9391)
Fix  Modules/_testmultiphase.c so that it exits with non-zero status on AIX just as other systems do (non zero exit status, e.g. as result of a segmentation fault) when a NULL pointer is accessed for data.


https://bugs.python.org/issue34720
2019-02-17 04:02:56 -08:00
Gregory P. Smith
38f11cc3f6
bpo-1054041: Exit properly after an uncaught ^C. (#11862)
* bpo-1054041: Exit properly by a signal after a ^C.

An uncaught KeyboardInterrupt exception means the user pressed ^C and
our code did not handle it.  Programs that install SIGINT handlers are
supposed to reraise the SIGINT signal to the SIG_DFL handler in order
to exit in a manner that their calling process can detect that they
died due to a Ctrl-C.  https://www.cons.org/cracauer/sigint.html

After this change on POSIX systems

 while true; do python -c 'import time; time.sleep(23)'; done

can be stopped via a simple Ctrl-C instead of the shell infinitely
restarting a new python process.

What to do on Windows, or if anything needs to be done there has not
yet been determined.  That belongs in its own PR.

TODO(gpshead): A unittest for this behavior is still needed.

* Do the unhandled ^C check after pymain_free.

* Return STATUS_CONTROL_C_EXIT on Windows.

* Fix ifdef around unistd.h include.

* 📜🤖 Added by blurb_it.

* Add STATUS_CTRL_C_EXIT to the os module on Windows

* Add unittests.

* Don't send CTRL_C_EVENT in the Windows test.

It was causing CI systems to bail out of the entire test suite.

See https://dev.azure.com/Python/cpython/_build/results?buildId=37980
for example.

* Correct posix test (fail on macOS?) check.

* STATUS_CONTROL_C_EXIT must be unsigned.

* Improve the error message.

* test typo :)

* Skip if the bash version is too old.

...and rename the windows test to reflect what it does.

* min bash version is 4.4, detect no bash.

* restore a blank line i didn't mean to delete.

* PyErr_Occurred() before the Py_DECREF(co);

* Don't add os.STATUS_CONTROL_C_EXIT as a constant.

* Update the Windows test comment.

* Refactor common logic into a run_eval_code_obj fn.
2019-02-16 12:57:40 -08:00
Alexey Izbyshev
36433221f0 bpo-35984: _xxsubinterpreters: Fix memory leak in _channel_send() (GH-11845)
https://bugs.python.org/issue35984
2019-02-15 14:28:58 -08:00
Zackery Spytz
bb3c05d7ef closes bpo-35991: Fix a potential double free in Modules/_randommodule.c. (GH-11849) 2019-02-13 23:43:17 -08:00
Anthony Sottile
73d600239b Remove stray quote in os.replace docstring. (GH-11556) 2019-02-12 20:15:54 -08:00
Pablo Galindo
4207907c2b
Fix division by 0 when checking for overflow in math.prod (GH-11808) 2019-02-10 19:56:58 +00:00
Raymond Hettinger
2dda72a2e8 lru_cache: Add more comments. Fix comment typos. Clarify a comment. (GH-11795) 2019-02-08 18:55:02 -08:00
Pablo Galindo
bc09851586 bpo-35606: Implement math.prod (GH-11359) 2019-02-06 23:04:02 -08:00
Paul Ganssle
89427cd0fe bpo-32417: Make timedelta arithmetic respect subclasses (#10902)
* Make timedelta return subclass types

Previously timedelta would always return the `date` and `datetime`
types, regardless of what it is added to. This makes it return
an object of the type it was added to.

* Add tests for timedelta arithmetic on subclasses

* Make pure python timedelta return subclass types

* Add test for fromtimestamp with tz argument

* Add tests for subclass behavior in now

* Add news entry.

Fixes:
bpo-32417
bpo-35364

* More descriptive variable names in tests

Addresses Victor's comments
2019-02-04 14:42:04 -05:00
Mark Becwar
b82bfac436 bpo-29734: nt._getfinalpathname handle leak (GH-740)
Make sure that failure paths call CloseHandle outside of the function that failed
2019-02-02 13:08:23 -08:00
Tony Roberts
4860f01ac0 bpo-33895: Relase GIL while calling functions that acquire Windows loader lock (GH-7789)
LoadLibrary, GetProcAddress, FreeLibrary and GetModuleHandle acquire the system loader lock. Calling these while holding the GIL will cause a deadlock on the rare occasion that another thread is detaching and needs to destroy its thread state at the same time.
2019-02-02 09:16:42 -08:00
Davin Potts
e5ef45b8f5
bpo-35813: Added shared_memory submodule of multiprocessing. (#11664)
Added shared_memory submodule to multiprocessing in time for first alpha with cross-platform tests soon to follow.
2019-02-01 22:52:23 -06:00
Victor Stinner
325e4bac5a
bpo-35537: Fix function name in os.posix_spawnp() errors (GH-11719) 2019-02-01 15:47:24 +01:00
Joannah Nanjekye
80c5dfe74b bpo-35537: Add setsid parameter to os.posix_spawn() and os.posix_spawnp() (GH-11608) 2019-02-01 11:05:22 +01:00
Raymond Hettinger
05f1b93f58
Speed-up argument parsing for common cases in deque.__init__()(GH-11717) 2019-01-31 22:13:43 -08:00