Commit graph

10434 commits

Author SHA1 Message Date
Serhiy Storchaka
fbb490fd2f
bpo-32308: Replace empty matches adjacent to a previous non-empty match in re.sub(). (#4846) 2018-01-04 11:06:13 +02:00
Benjamin Peterson
0a37a30037
closes bpo-32460: ensure all non-static globals have initializers (#5061) 2017-12-31 10:04:13 -08:00
Michael Felt
0d3ccb4395 bpo-32399: Starting with AIX6.1 there is support in libc.a for uuid (RFC4122) (#4974)
Starting with AIX6.1 there is support in libc.a for uuid (RFC4122)
This patch provides the changes needed for this integration with the OS.

On AIX the base function is uuid_create() rather than uuid_generate_time()
The AIX uuid_t typedef is more aligned to the UUID field based definition
while the Linux typedef that is more aligned with UUID bytes
(or perhaps UUID bytes_le) definitions.
2017-12-30 22:39:20 +01:00
Andrew Svetlov
fc35932afd
Fix typo (#5049) 2017-12-30 15:40:27 +02:00
Benjamin Peterson
bbdb17d19b
return the new file descriptor from os.dup2 (closes bpo-32441) (#5041) 2017-12-29 13:13:06 -08:00
Yury Selivanov
e0aef4f3cd
bpo-31721: Allow Future._log_traceback to only be set to False (#5009) 2017-12-25 16:16:10 -05:00
INADA Naoki
3070b71e5e
bpo-32422: Reduce lru_cache memory usage (GH-5008) 2017-12-26 02:03:24 +09:00
Yury Selivanov
0cf16f9ea0
bpo-32363: Disable Task.set_exception() and Task.set_result() (#4923) 2017-12-25 10:48:15 -05:00
INADA Naoki
4856b0f34a
bpo-32402: io: Add missing NULL check. (GH-4971)
_PyUnicode_FromId() may return NULL.

Reported by coverity scan: CID 1426868, 1426867.
2017-12-24 10:29:19 +09:00
Yury Selivanov
719ccbca69
bpo-32415: Fix "error is already set" (#4999) 2017-12-23 16:29:26 -05:00
Andrew Svetlov
0f47fa2c89 bpo-32357: Use PySet_GET_SIZE macro in _is_coroutine() from _asynciomodule.c (#4990) 2017-12-23 15:06:46 -05:00
Yury Selivanov
ca9b36cd1a
bpo-32415: Add asyncio.Task.get_loop() and Future.get_loop() (#4992) 2017-12-23 15:04:15 -05:00
Victor Stinner
9bee329130
bpo-32030: Add _Py_FindEnvConfigValue() (#4963)
Add a new _Py_FindEnvConfigValue() function: code shared between
Windows and Unix implementations of _PyPathConfig_Calculate() to read
the pyenv.cfg file.

_Py_FindEnvConfigValue() now uses _Py_DecodeUTF8_surrogateescape()
instead of using a Python Unicode string, the Python API must not be
used early during Python initialization. Same change in Unix
search_for_exec_prefix(): use _Py_DecodeUTF8_surrogateescape().

Cleanup also encode_current_locale(): PyMem_RawFree/PyMem_Free can be
called with NULL.

Fix also "NUL byte" => "NULL byte" typo.
2017-12-21 16:49:13 +01:00
Victor Stinner
9dd762013f
bpo-32030: Add _Py_EncodeLocaleRaw() (#4961)
Replace Py_EncodeLocale() with _Py_EncodeLocaleRaw() in:

* _Py_wfopen()
* _Py_wreadlink()
* _Py_wrealpath()
* _Py_wstat()
* pymain_open_filename()

These functions are called early during Python intialization, only
the RAW memory allocator must be used.
2017-12-21 16:20:32 +01:00
Paul Ganssle
09dc2f508c bpo-15873: Implement [date][time].fromisoformat (#4699)
Closes bpo-15873.
2017-12-21 00:33:49 -05:00
INADA Naoki
507434fd50
bpo-15216: io: TextIOWrapper.reconfigure() accepts encoding, errors and newline (GH-2343) 2017-12-21 09:59:53 +09:00
Victor Stinner
31e99080f6
bpo-32030: Fix usage of memory allocators (#4953)
* _Py_InitializeCore() doesn't call _PyMem_SetupAllocators() anymore
  if the PYTHONMALLOC environment variable is not set.
* pymain_cmdline() now sets the allocator to the default, instead of
  setting the allocator in subfunctions.
* Py_SetStandardStreamEncoding() now calls
  _PyMem_SetDefaultAllocator() to get a known allocator, to be able
  to release the memory with the same allocator.
2017-12-20 23:41:38 +01:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
83cb778b4a correct the typos (#4950) 2017-12-20 23:36:10 +02:00
Victor Stinner
9cfc00262c
bpo-32030: Complete _PyCoreConfig_Read() (#4946)
* Add _PyCoreConfig.install_signal_handlers
* Remove _PyMain.config: _PyMainInterpreterConfig usage is now
  restricted to pymain_init_python_main().
* Rename _PyMain.core_config to _PyMain.config
* _PyMainInterpreterConfig_Read() now creates the xoptions dictionary
   from the core config
* Fix _PyMainInterpreterConfig_Read(): don't replace xoptions and
  argv if they are already set.
2017-12-20 19:36:46 +01:00
Victor Stinner
ca719ac42b
bpo-32030: Add _PyCoreConfig.warnoptions (#4936)
Merge _PyCoreConfig_ReadEnv() into _PyCoreConfig_Read(), and
_Py_CommandLineDetails usage is now restricted to pymain_cmdline().

Changes:

* _PyCoreConfig: Add nxoption, xoptions, nwarnoption and warnoptions
* Add _PyCoreConfig.program: argv[0] or ""
* Move filename, command, module and xoptions from
  _Py_CommandLineDetails to _PyMain. xoptions _Py_OptList becomes
  (int, wchar_t**) list.
* Add pymain_cmdline() function
* Rename copy_argv() to copy_wstrlist(). Rename clear_argv() to
  clear_wstrlist(). Remove _Py_OptList structure: use (int,
  wchar_t**) list instead.
* Rename pymain_set_flag_from_env() to pymain_get_env_flag()
* Rename pymain_set_flags_from_env() to pymain_get_env_flags()
* _PyMainInterpreterConfig_Read() now creates the warnoptions from
  _PyCoreConfig.warnoptions
* Inline pymain_add_warning_dev_mode() and
  pymain_add_warning_bytes_flag() into config_init_warnoptions()
* Inline pymain_get_program_name() into _PyCoreConfig_Read()
* _Py_CommandLineDetails: Replace warning_options with nwarnoption
  and warnoptions. Replace env_warning_options with nenv_warnoption
  and env_warnoptions.
* pymain_warnings_envvar() now has a single implementation for
  Windows and Unix: use config_get_env_var_dup() to also get the
  variable as wchar_t* on Unix.
2017-12-20 18:00:19 +01:00
Marcel Plch
776407fe89 bpo-31901: atexit callbacks should be run at subinterpreter shutdown (#4611)
Change atexit behavior and PEP-489 multiphase init support.
2017-12-20 11:17:58 +01:00
Victor Stinner
1976086362
bpo-32030: Cleanup pymain_main() (#4935)
* Reorganize pymain_main() to make the code more flat
* Clear configurations before pymain_update_sys_path()
* Mark Py_FatalError() and _Py_FatalInitError() with _Py_NO_RETURN
* Replace _PyMain.run_code variable with a new RUN_CODE() macro
* Move _PyMain.cf into a local variable in pymain_run_python()
2017-12-20 01:41:59 +01:00
Victor Stinner
c4bca95106
bpo-32030: Add _PyCoreConfig.argv (#4934)
* Add argc and argv to _PyCoreConfig
* _PyMainInterpreterConfig_Read() now builds its argv from
  _PyCoreConfig.arg
* Move _PyMain.env_warning_options into _Py_CommandLineDetails
* Reorder pymain_free()
2017-12-19 23:48:17 +01:00
Zackery Spytz
e40ad79653 Fix GCC warning in _asynciomodule.c (#4928) 2017-12-19 13:48:13 -05:00
Michael Felt
c5ae169e1b bpo-26439 Fix ctypes.util.find_library failure on AIX (#4507)
Implement find_library() support in ctypes/util for AIX.

Add some AIX specific tests.
2017-12-19 13:58:49 +01:00
Yury Selivanov
a9d7e552c7
bpo-32357: Optimize asyncio.iscoroutine() for non-native coroutines (#4915) 2017-12-19 07:18:45 -05:00
Victor Stinner
5d8624647d
bpo-32030: Fix compiler warnings (#4921)
Fix compiler warnings in Py_FinalizeEx(): only define variables if
they are needed, add #ifdef.

Other cleanup changes:

* _PyWarnings_InitWithConfig() is no more needed: call
  _PyWarnings_Init() instead.
* Inline pymain_init_main_interpreter() in its caller. This
  subfunction is no more justifed.
2017-12-19 11:35:58 +01:00
Yury Selivanov
9818142b1b
bpo-32331: Fix socket.type when SOCK_NONBLOCK is available (#4877) 2017-12-18 20:02:54 -05:00
Victor Stinner
6efcb6d3d5
bpo-32030: Fix compilation on FreeBSD, #include <fenv.h> (#4919)
* main.c: add missing #include <fenv.h> on FreeBSD
* indent also other #ifdef in main.c
* cleanup Programs/python.c
2017-12-18 23:42:55 +01:00
Segev Finer
b2a6083eb0 bpo-19764: Implemented support for subprocess.Popen(close_fds=True) on Windows (#1218)
Even though Python marks any handles it opens as non-inheritable there
is still a race when using `subprocess.Popen` since creating a process
with redirected stdio requires temporarily creating inheritable handles.
By implementing support for `subprocess.Popen(close_fds=True)` we fix
this race.

In order to implement this we use PROC_THREAD_ATTRIBUTE_HANDLE_LIST
which is available since Windows Vista. Which allows to pass an explicit
list of handles to inherit when creating a process.

This commit also adds `STARTUPINFO.lpAttributeList["handle_list"]`
which can be used to control PROC_THREAD_ATTRIBUTE_HANDLE_LIST
directly.
2017-12-18 10:28:19 +01:00
Nathaniel J. Smith
902ab80b59 bpo-30050: Allow disabling full buffer warnings in signal.set_wakeup_fd (#4792) 2017-12-17 23:10:18 -05:00
Yury Selivanov
1b7c11ff0e
bpo-32348: Optimize asyncio.Future schedule/add/remove callback. (#4907) 2017-12-17 20:19:47 -05:00
Andrew Svetlov
44d1a5912e
bpo-32250: Implement asyncio.current_task() and asyncio.all_tasks() (#4799) 2017-12-16 21:58:38 +02:00
Serhiy Storchaka
45700fb757 Add tests for using PEP560 with classes implemented in C. (#4883)
Based on tests from #4878
2017-12-16 10:25:56 +01:00
Victor Stinner
9454060e84
bpo-29240, bpo-32030: Py_Main() re-reads config if encoding changes (#4899)
bpo-29240, bpo-32030: If the encoding change (C locale coerced or
UTF-8 Mode changed), Py_Main() now reads again the configuration with
the new encoding.

Changes:

* Add _Py_UnixMain() called by main().
* Rename pymain_free_pymain() to pymain_clear_pymain(), it can now be
  called multipled times.
* Rename pymain_parse_cmdline_envvars() to pymain_read_conf().
* Py_Main() now clears orig_argc and orig_argv at exit.
* Remove argv_copy2, Py_Main() doesn't modify argv anymore. There is
  no need anymore to get two copies of the wchar_t** argv.
* _PyCoreConfig: add coerce_c_locale and coerce_c_locale_warn.
* Py_UTF8Mode is now initialized to -1.
* Locale coercion (PEP 538) now respects -I and -E options.
2017-12-16 04:54:22 +01:00
Serhiy Storchaka
a5552f023e
bpo-32240: Add the const qualifier to declarations of PyObject* array arguments. (#4746) 2017-12-15 13:11:11 +02:00
Benjamin Peterson
e425bd7517
move pygetopt.h to internal (closes bpo-32264) (#4830) 2017-12-14 23:48:12 -08:00
Andrew Svetlov
f74ef458ab
bpo-32311: Implement asyncio.create_task() shortcut (#4848)
* Implement functionality
* Add documentation
2017-12-15 07:04:38 +02:00
Victor Stinner
41264f1cd4
bpo-32030: Add _PyMainInterpreterConfig.executable (#4876)
* Add new fields to _PyMainInterpreterConfig:

  * executable
  * prefix
  * base_prefix
  * exec_prefix
  * base_exec_prefix

* _PySys_EndInit() now sets sys attributes from
  _PyMainInterpreterConfig
2017-12-15 02:05:29 +01:00
Victor Stinner
da273412c4
bpo-32030: Add _PyCoreConfig_Copy() (#4874)
Each interpreter now has its core_config and main_config copy:

* Add _PyCoreConfig_Copy() and _PyMainInterpreterConfig_Copy()
* Move _PyCoreConfig_Read(), _PyCoreConfig_Clear() and
  _PyMainInterpreterConfig_Clear() from Python/pylifecycle.c to
  Modules/main.c
* Fix _Py_InitializeEx_Private(): call _PyCoreConfig_ReadEnv() before
  _Py_InitializeCore()
2017-12-15 01:46:02 +01:00
Victor Stinner
358e5e17a5
bpo-32329: Fix -R option for hash randomization (#4873)
bpo-32329, bpo-32030:

* The -R option now turns on hash randomization when the
  PYTHONHASHSEED environment variable is set to 0 Previously, the
  option was ignored.
* sys.flags.hash_randomization is now properly set to 0 when hash
  randomization is turned off by PYTHONHASHSEED=0.
* _PyCoreConfig_ReadEnv() now reads the PYTHONHASHSEED environment
  variable. _Py_HashRandomization_Init() now only apply the
  configuration, it doesn't read PYTHONHASHSEED anymore.
2017-12-15 00:51:22 +01:00
Giuseppe Scrivano
96a5e50a5d bpo-32143: add f_fsid to os.statvfs() (#4571)
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2017-12-14 17:46:46 -05:00
Anthony Sottile
233ef249cc bpo-32277: Fix exception raised from chmod(..., follow_symlinks=False) (#4797) 2017-12-14 18:57:55 +02:00
Victor Stinner
374c6e178a
bpo-32030: Add _PyMainInterpreterConfig.warnoptions (#4855)
Add warnoptions and xoptions fields to _PyMainInterpreterConfig.
2017-12-14 12:05:26 +01:00
Mike
53f7a7c281 bpo-32297: Few misspellings found in Python source code comments. (#4803)
* Fix multiple typos in code comments

* Add spacing in comments (test_logging.py, test_math.py)

* Fix spaces at the beginning of comments in test_logging.py
2017-12-14 13:04:53 +02:00
Victor Stinner
b5fd9ad05e
bpo-32030: Rewrite _PyMainInterpreterConfig (#4854)
_PyMainInterpreterConfig now contains Python objects, whereas
_PyCoreConfig contains wchar_t* strings.

Core config:

* Rename _PyMainInterpreterConfig_ReadEnv() to _PyCoreConfig_ReadEnv()
* Move 3 strings from _PyMainInterpreterConfig to _PyCoreConfig:
  module_search_path_env, home, program_name.
* Add _PyCoreConfig_Clear()
* _PyPathConfig_Calculate() now takes core config rather than main
  config
* _PyMainInterpreterConfig_Read() now requires also a core config

Main config:

* Add _PyMainInterpreterConfig.module_search_path: sys.path list
* Add _PyMainInterpreterConfig.argv: sys.argv list
* _PyMainInterpreterConfig_Read() now computes module_search_path
2017-12-14 02:20:52 +01:00
Victor Stinner
11a247df88
bpo-32030: Add _PyPathConfig_ComputeArgv0() (#4845)
Changes:

* Split _PySys_SetArgvWithError() into subfunctions for Py_Main():

  * Create the Python list object
  * Set sys.argv to the list
  * Compute argv0
  * Prepend argv0 to sys.path

* Add _PyPathConfig_ComputeArgv0()
* Remove _PySys_SetArgvWithError()
* Py_Main() now splits the code to compute sys.argv/path0 and the
  code to update the sys module: add pymain_compute_argv()
  subfunction.
2017-12-13 21:05:57 +01:00
Yury Selivanov
a70232f288
bpo-32296: Implement asyncio.get_event_loop and _get_running_loop in C. (#4827)
asyncio.get_event_loop(), and, subsequently asyncio._get_running_loop()
are one of the most frequently executed functions in asyncio.  They also
can't be sped up by third-party event loops like uvloop.

When implemented in C they become 4x faster.
2017-12-13 14:49:42 -05:00
Victor Stinner
d5dda98fa8
pymain_set_sys_argv() now copies argv (#4838)
bpo-29240, bpo-32030:

* Rename pymain_set_argv() to pymain_set_sys_argv()
* pymain_set_sys_argv() now creates of copy of argv and modify the
  copy, rather than modifying pymain->argv
* Call pymain_set_sys_argv() earlier: before pymain_run_python(), but
  after pymain_get_importer().
* Add _PySys_SetArgvWithError() to handle errors
2017-12-13 17:31:16 +01:00
Victor Stinner
91106cd9ff
bpo-29240: PEP 540: Add a new UTF-8 Mode (#855)
* Add -X utf8 command line option, PYTHONUTF8 environment variable
  and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
  UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
  _winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
  mode. As a side effect, open() now uses the UTF-8 encoding by
  default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
  in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
  enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
  return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
  always copy flag values, rather than only copying if the new value
  is greater than the old value.
2017-12-13 12:29:09 +01:00