Commit graph

10614 commits

Author SHA1 Message Date
Paul Ganssle
58dc03c737 Cleanup dangling reference in get_timezone_utc_capi (#5317) 2018-01-25 14:58:07 +01:00
Serhiy Storchaka
f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Victor Stinner
2b822a0bb1
bpo-32652: Defer pymain_set_global_config() call (#5303)
In Py_Main(), don't call pymain_set_global_config() early: only call
it when the whole configuration has been read.

Add an unit test to prevent future regression.
2018-01-25 09:18:36 +01:00
Paul Ganssle
04af5b1ba9 bpo-10381: Add timezone to datetime C API (#5032)
* Add timezone to datetime C API

* Add documentation for timezone C API macros

* Add dedicated tests for datetime type check macros

* Remove superfluous C API test

* Drop support for TimeZoneType in datetime C API

* Expose UTC singleton to the datetime C API

* Update datetime C-API documentation to include links

* Add reference count information for timezone constructors
2018-01-24 17:29:30 -05:00
Barry Warsaw
6f6eb35f9b
bpo-32248 - Implement ResourceReader and get_resource_reader() for zipimport (#5248) 2018-01-24 15:36:21 -05:00
Yury Selivanov
22feeb88b4
bpo-32643: Drop support for a few private Task and Future APIs. (#5293)
Specifically, it's not possible to subclass Task/Future classes
and override the following methods:

* Future._schedule_callbacks
* Task._step
* Task._wakeup
2018-01-24 11:31:01 -05:00
Victor Stinner
8ded5b8037
bpo-32030: Add _PyCoreConfig.module_search_paths (#4954)
_PyCoreConfig_Read() is now responsible to compute sys.path.
So sys.path is now computed before calling _Py_InitializeCore().

Changes:

* Add module_search_path, module_search_paths, executable, prefix,
  base_prefix, exec_prefix and base_exec_prefix to _PyCoreConfig.
* _PyMainInterpreterConfig_Read() now only converts wchar_t** lists
  into a Python list, it doesn't compute sys.path anymore.
2018-01-24 17:03:28 +01:00
Victor Stinner
5de15f1d5f
bpo-28046: Remove MACHDEPPATH from Modules/Setup.dist (#5289)
Makefile $MACHDEPPATH uses $PLATDIR, but this variable was removed.
So remove also $MACHDEPPATH.

Remove also $EXTRAMACHDEPPATH.
2018-01-24 17:02:41 +01:00
Yury Selivanov
9d411c119f
bpo-32296: Make get_running_loop() another 4-5x faster (#5277) 2018-01-23 15:10:03 -05:00
Yury Selivanov
f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Serhiy Storchaka
1211c9a989
bpo-32503: Avoid creating too small frames in pickles. (#5127) 2018-01-20 16:42:44 +02:00
Christian Heimes
ff5be6e810
bpo-32598: Use autoconf to detect usable OpenSSL (#5242)
Add https://www.gnu.org/software/autoconf-archive/ax_check_openssl.html
to auto-detect compiler flags, linker flags and libraries to compile
OpenSSL extensions. The M4 macro uses pkg-config and falls back to
manual detection.

Add autoconf magic to detect usable X509_VERIFY_PARAM_set1_host()
and related functions.

Refactor setup.py to use new config vars to compile _ssl and _hashlib
modules.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-20 13:19:21 +01:00
Victor Stinner
05d68a8bd8
bpo-9566: Fix size_t=>int downcast warnings (#5230)
* Use wider types (int => Py_ssize_t) to avoid integer overflows.
* Fix gc.get_freeze_count(): use Py_ssize_t type rather than int, since gc_list_size() returns a Py_ssize_t.
2018-01-18 11:15:25 +01:00
Victor Stinner
e860089fd9
bpo-32226: Fix memory leak in generic_alias_dealloc() (#5212) 2018-01-17 23:08:18 +01:00
Paul Ganssle
9f1b7b93f5 bpo-32403: Faster date and datetime constructors (#4993)
* Add tests for date subclass alternate constructors

* Switch over alternate date constructors to fast path

* Switch datetime constructors to fastpath, fix bpo-32404

* Add fast path for datetime in date subclass constructor

* Set fold in constructor in datetime.combine

* Add news entries.
2018-01-16 13:06:31 -05:00
Serhiy Storchaka
4d9aec0220
bpo-31572: Get rid of PyObject_HasAttr() and _PyObject_HasAttrId() in the _io module. (#3726) 2018-01-16 18:34:21 +02:00
INADA Naoki
378edee0a3
bpo-32544: Speed up hasattr() and getattr() (GH-5173)
AttributeError was raised always when attribute is not found.
This commit skip raising AttributeError when `tp_getattro` is `PyObject_GenericGetAttr`.
It makes hasattr() and getattr() about 4x faster when attribute is not found.
2018-01-16 20:52:41 +09:00
Antoine Pitrou
94e1696d04
bpo-14976: Reentrant simple queue (#3346)
Add a queue.SimpleQueue class, an unbounded FIFO queue with a reentrant C implementation of put().
2018-01-16 00:27:16 +01:00
Victor Stinner
cb064fc232
bpo-31900: Fix localeconv() encoding for LC_NUMERIC (#4174)
* Add _Py_GetLocaleconvNumeric() function: decode decimal_point and
  thousands_sep fields of localeconv() from the LC_NUMERIC encoding,
  rather than decoding from the LC_CTYPE encoding.
* Modify locale.localeconv() and "n" formatter of str.format() (for
  int, float and complex to use _Py_GetLocaleconvNumeric()
  internally.
2018-01-15 15:58:02 +01:00
Victor Stinner
7ed7aead95
bpo-29240: Fix locale encodings in UTF-8 Mode (#5170)
Modify locale.localeconv(), time.tzname, os.strerror() and other
functions to ignore the UTF-8 Mode: always use the current locale
encoding.

Changes:

* Add _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx(). On decoding or
  encoding error, they return the position of the error and an error
  message which are used to raise Unicode errors in
  PyUnicode_DecodeLocale() and PyUnicode_EncodeLocale().
* Replace _Py_DecodeCurrentLocale() with _Py_DecodeLocaleEx().
* PyUnicode_DecodeLocale() now uses _Py_DecodeLocaleEx() for all
  cases, especially for the strict error handler.
* Add _Py_DecodeUTF8Ex(): return more information on decoding error
  and supports the strict error handler.
* Rename _Py_EncodeUTF8_surrogateescape() to _Py_EncodeUTF8Ex().
* Replace _Py_EncodeCurrentLocale() with _Py_EncodeLocaleEx().
* Ignore the UTF-8 mode to encode/decode localeconv(), strerror()
  and time zone name.
* Remove PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize()
  and PyUnicode_EncodeLocale() now ignore the UTF-8 mode: always use
  the "current" locale.
* Remove _PyUnicode_DecodeCurrentLocale(),
  _PyUnicode_DecodeCurrentLocaleAndSize() and
  _PyUnicode_EncodeCurrentLocale().
2018-01-15 10:45:49 +01:00
Serhiy Storchaka
5b76bdba07
bpo-31993: Do not use memoryview when pickle large strings. (#5154)
PyMemoryView_FromMemory() created a memoryview referring to
the internal data of the string.  When the string is destroyed
the memoryview become referring to a freed memory.
2018-01-13 00:28:31 +02:00
Serhiy Storchaka
0a2da50e18
bpo-31993: Do not create frames for large bytes and str objects (#5114)
when serialize into memory buffer with C pickle implementations.

This optimization already is performed when serialize into memory
with Python pickle implementations or into a file with both
implementations.
2018-01-11 13:03:20 +02:00
Victor Stinner
cb3ae5588b
bpo-29240: Ignore UTF-8 Mode in time module (#5148)
time.strftime() must use the current LC_CTYPE encoding, not UTF-8
if the UTF-8 mode is enabled.

Add _PyUnicode_DecodeCurrentLocale() function.
2018-01-11 10:37:59 +01:00
Victor Stinner
2cba6b8579
bpo-29240: readline now ignores the UTF-8 Mode (#5145)
Add new fuctions ignoring the UTF-8 mode:

* _Py_DecodeCurrentLocale()
* _Py_EncodeCurrentLocale()
* _PyUnicode_DecodeCurrentLocaleAndSize()
* _PyUnicode_EncodeCurrentLocale()

Modify the readline module to use these functions.

Re-enable test_readline.test_nonascii().
2018-01-10 22:46:15 +01:00
David Carlier
b4ebaa7099 bpo-32493: Not only AIX, but FreeBSD has uuid_create support (#5089)
Allow building the _uuid extension module on FreeBSD and OpenBSD.
2018-01-09 20:38:07 +01:00
Max Bélanger
af11a15c58 bpo-32282: Remove unnecessary check for VersionHelpers.h in socketmodule.c on Windows 2018-01-07 13:05:18 +11:00
Olivier Grisel
3cd7c6e6eb bpo-31993: Do not allocate large temporary buffers in pickle dump. (#4353)
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.

Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.

The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
2018-01-06 17:18:54 +02:00
Nathaniel J. Smith
735ae8d139 bpo-29137: Remove fpectl module (#4789)
This module has never been enabled by default, never worked correctly
on x86-64, and caused ABI problems that caused C extension
compatibility. See bpo-29137 for details/discussion.
2018-01-05 23:15:34 -08:00
Michael Felt
502d551c6d bpo-32390: Fix compilation failure on AIX after f_fsid was added to os.statvfs() (#4972) 2018-01-05 13:01:58 +01:00
Serhiy Storchaka
ce5b0e9db1
bpo-32226: Make __class_getitem__ an automatic class method. (#5098) 2018-01-05 00:21:41 +02:00
Serhiy Storchaka
fbb490fd2f
bpo-32308: Replace empty matches adjacent to a previous non-empty match in re.sub(). (#4846) 2018-01-04 11:06:13 +02:00
Benjamin Peterson
0a37a30037
closes bpo-32460: ensure all non-static globals have initializers (#5061) 2017-12-31 10:04:13 -08:00
Michael Felt
0d3ccb4395 bpo-32399: Starting with AIX6.1 there is support in libc.a for uuid (RFC4122) (#4974)
Starting with AIX6.1 there is support in libc.a for uuid (RFC4122)
This patch provides the changes needed for this integration with the OS.

On AIX the base function is uuid_create() rather than uuid_generate_time()
The AIX uuid_t typedef is more aligned to the UUID field based definition
while the Linux typedef that is more aligned with UUID bytes
(or perhaps UUID bytes_le) definitions.
2017-12-30 22:39:20 +01:00
Andrew Svetlov
fc35932afd
Fix typo (#5049) 2017-12-30 15:40:27 +02:00
Benjamin Peterson
bbdb17d19b
return the new file descriptor from os.dup2 (closes bpo-32441) (#5041) 2017-12-29 13:13:06 -08:00
Yury Selivanov
e0aef4f3cd
bpo-31721: Allow Future._log_traceback to only be set to False (#5009) 2017-12-25 16:16:10 -05:00
INADA Naoki
3070b71e5e
bpo-32422: Reduce lru_cache memory usage (GH-5008) 2017-12-26 02:03:24 +09:00
Yury Selivanov
0cf16f9ea0
bpo-32363: Disable Task.set_exception() and Task.set_result() (#4923) 2017-12-25 10:48:15 -05:00
INADA Naoki
4856b0f34a
bpo-32402: io: Add missing NULL check. (GH-4971)
_PyUnicode_FromId() may return NULL.

Reported by coverity scan: CID 1426868, 1426867.
2017-12-24 10:29:19 +09:00
Yury Selivanov
719ccbca69
bpo-32415: Fix "error is already set" (#4999) 2017-12-23 16:29:26 -05:00
Andrew Svetlov
0f47fa2c89 bpo-32357: Use PySet_GET_SIZE macro in _is_coroutine() from _asynciomodule.c (#4990) 2017-12-23 15:06:46 -05:00
Yury Selivanov
ca9b36cd1a
bpo-32415: Add asyncio.Task.get_loop() and Future.get_loop() (#4992) 2017-12-23 15:04:15 -05:00
Victor Stinner
9bee329130
bpo-32030: Add _Py_FindEnvConfigValue() (#4963)
Add a new _Py_FindEnvConfigValue() function: code shared between
Windows and Unix implementations of _PyPathConfig_Calculate() to read
the pyenv.cfg file.

_Py_FindEnvConfigValue() now uses _Py_DecodeUTF8_surrogateescape()
instead of using a Python Unicode string, the Python API must not be
used early during Python initialization. Same change in Unix
search_for_exec_prefix(): use _Py_DecodeUTF8_surrogateescape().

Cleanup also encode_current_locale(): PyMem_RawFree/PyMem_Free can be
called with NULL.

Fix also "NUL byte" => "NULL byte" typo.
2017-12-21 16:49:13 +01:00
Victor Stinner
9dd762013f
bpo-32030: Add _Py_EncodeLocaleRaw() (#4961)
Replace Py_EncodeLocale() with _Py_EncodeLocaleRaw() in:

* _Py_wfopen()
* _Py_wreadlink()
* _Py_wrealpath()
* _Py_wstat()
* pymain_open_filename()

These functions are called early during Python intialization, only
the RAW memory allocator must be used.
2017-12-21 16:20:32 +01:00
Paul Ganssle
09dc2f508c bpo-15873: Implement [date][time].fromisoformat (#4699)
Closes bpo-15873.
2017-12-21 00:33:49 -05:00
INADA Naoki
507434fd50
bpo-15216: io: TextIOWrapper.reconfigure() accepts encoding, errors and newline (GH-2343) 2017-12-21 09:59:53 +09:00
Victor Stinner
31e99080f6
bpo-32030: Fix usage of memory allocators (#4953)
* _Py_InitializeCore() doesn't call _PyMem_SetupAllocators() anymore
  if the PYTHONMALLOC environment variable is not set.
* pymain_cmdline() now sets the allocator to the default, instead of
  setting the allocator in subfunctions.
* Py_SetStandardStreamEncoding() now calls
  _PyMem_SetDefaultAllocator() to get a known allocator, to be able
  to release the memory with the same allocator.
2017-12-20 23:41:38 +01:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
83cb778b4a correct the typos (#4950) 2017-12-20 23:36:10 +02:00
Victor Stinner
9cfc00262c
bpo-32030: Complete _PyCoreConfig_Read() (#4946)
* Add _PyCoreConfig.install_signal_handlers
* Remove _PyMain.config: _PyMainInterpreterConfig usage is now
  restricted to pymain_init_python_main().
* Rename _PyMain.core_config to _PyMain.config
* _PyMainInterpreterConfig_Read() now creates the xoptions dictionary
   from the core config
* Fix _PyMainInterpreterConfig_Read(): don't replace xoptions and
  argv if they are already set.
2017-12-20 19:36:46 +01:00
Victor Stinner
ca719ac42b
bpo-32030: Add _PyCoreConfig.warnoptions (#4936)
Merge _PyCoreConfig_ReadEnv() into _PyCoreConfig_Read(), and
_Py_CommandLineDetails usage is now restricted to pymain_cmdline().

Changes:

* _PyCoreConfig: Add nxoption, xoptions, nwarnoption and warnoptions
* Add _PyCoreConfig.program: argv[0] or ""
* Move filename, command, module and xoptions from
  _Py_CommandLineDetails to _PyMain. xoptions _Py_OptList becomes
  (int, wchar_t**) list.
* Add pymain_cmdline() function
* Rename copy_argv() to copy_wstrlist(). Rename clear_argv() to
  clear_wstrlist(). Remove _Py_OptList structure: use (int,
  wchar_t**) list instead.
* Rename pymain_set_flag_from_env() to pymain_get_env_flag()
* Rename pymain_set_flags_from_env() to pymain_get_env_flags()
* _PyMainInterpreterConfig_Read() now creates the warnoptions from
  _PyCoreConfig.warnoptions
* Inline pymain_add_warning_dev_mode() and
  pymain_add_warning_bytes_flag() into config_init_warnoptions()
* Inline pymain_get_program_name() into _PyCoreConfig_Read()
* _Py_CommandLineDetails: Replace warning_options with nwarnoption
  and warnoptions. Replace env_warning_options with nenv_warnoption
  and env_warnoptions.
* pymain_warnings_envvar() now has a single implementation for
  Windows and Unix: use config_get_env_var_dup() to also get the
  variable as wchar_t* on Unix.
2017-12-20 18:00:19 +01:00