Commit graph

9370 commits

Author SHA1 Message Date
Victor Stinner
4a06647534 Add assertions on tracemalloc_reentrant_key
Issue #26588.
2016-03-22 17:45:09 +01:00
Victor Stinner
0cfc058d61 Issue #26588: more assertions 2016-03-22 17:40:07 +01:00
Victor Stinner
84aab09421 Issue #26588: add debug traces
Try to debug random failure on buildbots.
2016-03-22 16:13:31 +01:00
Victor Stinner
10b73e1748 Add C functions _PyTraceMalloc_Track()
Issue #26530:

* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
  memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
2016-03-22 13:39:05 +01:00
Victor Stinner
e492ae50e2 tracemalloc now supports domains
Issue #26588:

* The _tracemalloc now supports tracing memory allocations of multiple address
  spaces (domains).
* Add domain parameter to tracemalloc_add_trace() and
  tracemalloc_remove_trace().
* tracemalloc_add_trace() now starts by removing the previous trace, if any.
* _tracemalloc._get_traces() now returns a list of (domain, size,
  traceback_frames): the domain is new.
* Add tracemalloc.DomainFilter
* tracemalloc.Filter: add an optional domain parameter to the constructor and a
  domain attribute
* Sublte change: use Py_uintptr_t rather than void* in the traces key.
* Add tracemalloc_config.use_domain, currently hardcoded to 1
2016-03-22 12:58:23 +01:00
Victor Stinner
58100059ac Remove _Py_hashtable_delete()
Issue #26588: Remove _Py_hashtable_delete() from hashtable.h since the function
is not used. Keep the C code in hashtable.c as commented code if someone needs
it later.
2016-03-22 12:25:04 +01:00
Victor Stinner
c9553876ae Simplify implementation of hashtable.c
Issue #26588: Remove copy_data, free_data and get_data_size callbacks from
hashtable.h. These callbacks are not used in Python and makes the code more
complex.

Remove also the _Py_HASHTABLE_ENTRY_DATA_AS_VOID_P() macro which uses an unsafe
pointer dereference (can cause memory alignment issue). Replace the macro usage
with _Py_HASHTABLE_ENTRY_READ_DATA() which is implemented with the safe
memcpy() function.
2016-03-22 12:13:01 +01:00
Martin Panter
0b2d71bc70 Issue #24266: Merge readline Ctrl+C handling from 3.5 2016-03-22 09:28:58 +00:00
Martin Panter
d6990d221b Issue #24266: Cancel history search mode with Ctrl+C in Readline 7 2016-03-22 07:24:05 +00:00
Martin Panter
12eb2ab490 Issue #15699: Merge readline fixup from 3.5 2016-03-22 02:26:18 +00:00
Martin Panter
f6e9f47aa7 Issue #15699: Reunite comment with variable 2016-03-22 02:19:29 +00:00
Victor Stinner
b32a7edb22 Issue #26588: Fix compilation warning on Windows 2016-03-21 23:05:08 +01:00
Victor Stinner
285cf0a601 hashtable.h now supports keys of any size
Issue #26588: hashtable.h now supports keys of any size, not only
sizeof(void*). It allows to support key larger than sizeof(void*), but also to
use less memory for key smaller than sizeof(void*).
2016-03-21 22:00:58 +01:00
Victor Stinner
928bff0b26 cleanup iobase.c
casting iobase_finalize to destructor is not needed
2016-03-19 10:36:36 +01:00
Victor Stinner
19a8e844e4 Add socket finalizer
Issue #26590: Implement a safe finalizer for the _socket.socket type. It now
releases the GIL to close the socket. Use PyErr_ResourceWarning() to raise the
ResourceWarning to pass the socket object to the warning logger, to get the
traceback where the socket was created (allocated).
2016-03-21 16:36:48 +01:00
Victor Stinner
322bc12c31 Ooops, revert changeset ea9efa06c137
Change pushed by mistake, the patch is still under review :-/

"""
_tracemalloc: add domain to trace keys

* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
"""
2016-03-21 14:36:39 +01:00
Victor Stinner
51b846c47a _tracemalloc: add domain to trace keys
* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
2016-03-18 21:52:22 +01:00
Berker Peksag
3c3d7f4b99 Issue #18787: spwd.getspnam() now raises a PermissionError if the user
doesn't have privileges.
2016-03-19 11:44:17 +02:00
Victor Stinner
f664dc5834 ResourceWarning: Revert change on socket and scandir
io.FileIO has a safe implementation of destructor, but not socket nor scandir.
2016-03-19 02:01:48 +01:00
Victor Stinner
914cde89d4 On ResourceWarning, log traceback where the object was allocated
Issue #26567:

* Add a new function PyErr_ResourceWarning() function to pass the destroyed
  object
* Add a source attribute to warnings.WarningMessage
* Add warnings._showwarnmsg() which uses tracemalloc to get the traceback where
  source object was allocated.
2016-03-19 01:03:51 +01:00
Victor Stinner
13be7db34c Fix usage of PyMem_Malloc() in overlapped.c
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
2016-03-16 23:25:02 +01:00
Victor Stinner
6150f31963 Fix usage of PyMem_Malloc() in overlapped.c
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
2016-03-16 23:25:02 +01:00
Victor Stinner
861d9abfcf faulthandler now works in non-Python threads
Issue #26563:

* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
  PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
  autoInterpreterState in last resort. The function now accepts NULL for interp
  and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
2016-03-16 22:45:24 +01:00
Victor Stinner
c36674a2c5 Fix usage of PyMem_Malloc() in os.stat()
Issue #26563: Replace PyMem_Malloc() with PyMem_RawMalloc() in the Windows
implementation of os.stat(), since the code is called without holding the GIL.
2016-03-16 14:30:16 +01:00
Victor Stinner
ad524375af Fail if PyMem_Malloc() is called without holding the GIL
Issue #26563: Debug hooks on Python memory allocators now raise a fatal error
if functions of the PyMem_Malloc() family are called without holding the GIL.
2016-03-16 12:12:53 +01:00
Victor Stinner
0611c26a58 On memory error, dump the memory block traceback
Issue #26564: _PyObject_DebugDumpAddress() now dumps the traceback where a
memory block was allocated on memory block. Use the tracemalloc module to get
the traceback.
2016-03-15 22:22:13 +01:00
Victor Stinner
af584a02a5 Merge 3.5 (_tracemalloc) 2016-03-15 21:57:23 +01:00
Victor Stinner
9528334e16 _tracemalloc: store lineno as unsigned int
Issue #26564. Cleanup the code, lineno is never negative.
2016-03-15 21:57:02 +01:00
Victor Stinner
b380010782 Merge 3.5 (test_faulthandler) 2016-03-15 17:24:13 +01:00
Victor Stinner
57003f81ea faulthandler: Test Py_FatalError() with GIL released
Issue #26558.
2016-03-15 17:23:35 +01:00
Victor Stinner
7105e9f3de _tracemalloc: filename cannot be NULL 2016-03-15 14:28:04 +01:00
Victor Stinner
c4aec3628b Check the GIL in PyObject_Malloc()
Issue #26558: The debug hook of PyObject_Malloc() now checks that the GIL is
held when the function is called.
2016-03-14 22:26:53 +01:00
Victor Stinner
13f0c6166f _pickle: Fix load_counted_tuple(), use Py_ssize_t for size
Fix a warning on Windows 64-bit.
2016-03-14 18:09:39 +01:00
Victor Stinner
21b47117ac _pickle: Fix load_counted_tuple(), use Py_ssize_t for size
Fix a warning on Windows 64-bit.
2016-03-14 18:09:39 +01:00
Victor Stinner
c44f70770b posix_getcwd(): limit to INT_MAX on Windows
It's more to fix a conversion warning during compilation, I don't think that
Windows support current working directory larger than 2 GB ...
2016-03-14 18:07:53 +01:00
Victor Stinner
d222653f8f Issue #26558: Remove useless check in tracemalloc
The first instruction of tracemalloc_add_trace() is traceback_new() which
already checks the GIL.
2016-03-14 17:01:32 +01:00
Victor Stinner
34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Ned Deily
8cd8883d27 Issue #26505: merge from 3.5 2016-03-07 14:52:46 -05:00
Ned Deily
adbf8cecbe Issue #26505: Fix typos in getaddrinfo license text.
Patch by Alex Willmer.
2016-03-07 14:51:59 -05:00
Alexander Belopolsky
a2998a63c8 Closes #19475: Added timespec to the datetime.isoformat() method.
Added an optional argument timespec to the datetime isoformat() method
to choose the precision of the time component.

Original patch by Alessandro Cucci.
2016-03-06 14:58:43 -05:00
Serhiy Storchaka
5608411a96 Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. 2016-03-06 14:02:26 +02:00
Serhiy Storchaka
d55162517d Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. 2016-03-06 14:00:45 +02:00
Serhiy Storchaka
a0d416f0d1 Issue #26482: Allowed pickling recursive dequeues. 2016-03-06 08:55:21 +02:00
Raymond Hettinger
d79d5b1a50 More logicial order. Move space saving step to just before it is used. 2016-03-04 09:55:07 -08:00
Berker Peksag
6282e656e9 Issue #26335: Make mmap.write() return the number of bytes written like
other write methods.

Patch by Jakub Stasiak.
2016-03-02 19:30:18 +02:00
Raymond Hettinger
6f86a3308a Factor-out common subexpression. 2016-03-02 00:30:58 -08:00
Raymond Hettinger
589106b206 Put block length computations in a more logical order. 2016-03-02 00:06:21 -08:00
Ned Deily
3d2778c8da Issue #26406: merge from 3.5 2016-02-23 22:10:06 +11:00
Ned Deily
e1d4e58777 Issue #26406: Avoid unnecessary serialization of getaddrinfo(3) calls on
current versions of OpenBSD and NetBSD.  Patch by A. Jesse Jiryu Davis.
2016-02-23 22:05:29 +11:00
Benjamin Peterson
4a180a693f merge 3.5 (closes #25939) 2016-02-17 22:18:35 -08:00