Commit graph

102742 commits

Author SHA1 Message Date
Victor Stinner
f966e5397e
bpo-29564:_PyMem_DumpTraceback() suggests enabling tracemalloc (GH-10510)
If tracemalloc is not tracing Python memory allocations,
_PyMem_DumpTraceback() now suggests to enable tracemalloc
to get the traceback where the memory block has been allocated.
2018-11-13 15:14:58 +01:00
Eddie Elizondo
474eedfb3d bpo-34784: Fix PyStructSequence_NewType with heap-allocated StructSequence (GH-9665) 2018-11-13 13:09:31 +01:00
Victor Stinner
1a6be91e6f
bpo-35081: Make some _PyGC macros internal (GH-10507)
* Move "GC" macros together:

  * PyObject_IS_GC()
  * _Py_AS_GC()
  * _PyObject_GC_IS_TRACKED()
  * _PyObject_GC_MAY_BE_TRACKED()

* Mark other GC macros as internal (#ifdef Py_BUILD_CORE):

  * _PyGCHead_NEXT(g), _PyGCHead_SET_NEXT(g, p)
  * _PyGCHead_PREV(g), _PyGCHead_SET_PREV(g, p)
  * _PyGCHead_FINALIZED(g), _PyGCHead_SET_FINALIZED(g)
  * _PyGC_FINALIZED(o), _PyGC_SET_FINALIZED(o)
  * _PyObject_GC_TRACK(o), _PyObject_GC_UNTRACK(o)
  * _PyGC_PREV_MASK_FINALIZED
  * _PyGC_PREV_MASK_COLLECTING
  * _PyGC_PREV_SHIFT
  * _PyGC_PREV_MASK

* Replace _PyGC_generation0 with _PyRuntime.gc.generation0
* _queuemodule.c: replace _PyObject_GC_UNTRACK()
  with with PyObject_GC_UnTrack()
* Document that  _PyObject_GC_TRACK() _PyObject_GC_UNTRACK() macros
  have been removed from the public C API.
2018-11-13 12:52:18 +01:00
Paul Ganssle
0d12672b30 bpo-35081: Remove Py_BUILD_CORE from datetime.h (GH-10416)
Datetime macros like PyDate_Check() have two implementations, one using
the C API capsule and one using direct access to the datetime type
symbols defined in _datetimemodule.c. Since the direct access versions
of the macros are only used in _datetimemodule.c, they have been moved
out of "datetime.h" and into _datetimemodule.c.

The _PY_DATETIME_IMPL macro is currently necessary in order to avoid
both duplicate definitions of these macros in _datetimemodule.c and
unnecessary declarations of C API capsule-related macros and varibles in
datetime.h.

Co-Authored-By: Victor Stinner <vstinner@redhat.com>
2018-11-13 09:02:25 +01:00
Gregory P. Smith
3015fb8ce4
bpo-35214: Add _Py_ prefix to MEMORY_SANITIZER def. (GH-10503)
Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
2018-11-12 22:01:22 -08:00
Windson yang
0dc1e45dfd Fix a couple documentation typos. (GH-10498)
reproduciblity -> reproducibility
PyPA are the group -> PyPA is the group
2018-11-12 19:56:25 -08:00
Gregory P. Smith
e6c77d8301
Disable getc_unlocked() with MemorySanitizer. (GH-10499)
clang's MemorySanitizer understand getc() but does not understand
getc_unlocked().  Workaround: Don't use it on msan builds.
2018-11-12 19:47:13 -08:00
Victor Stinner
2c07c493d2
bpo-29564: warnings suggests to enable tracemalloc (GH-10486)
The warnings module now suggests to enable tracemalloc if the source
is specified, tracemalloc module is available, but tracemalloc is not
tracing memory allocations.
2018-11-13 02:41:00 +01:00
Gregory P. Smith
1584a00815
bpo-35214: Initial clang MemorySanitizer support (GH-10479)
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
2018-11-12 12:07:14 -08:00
Windson yang
a9655b7f71 Improve grammar in Glossary. (GH-10474)
a asynchronous generator -> an asynchronous generator
2018-11-12 09:42:38 -08:00
Andrés Delfino
7610f4fe93 Correct grammar mistake in stdtypes.rst (GH-10481) 2018-11-12 09:24:00 -08:00
Andrew Svetlov
7438792607
bpo-30064: Refactor sock_* asyncio API (#10419) 2018-11-12 19:00:22 +02:00
Andrés Delfino
9404e7737b Linkify PEP 8 in unix.rst (GH-10482) 2018-11-12 08:31:57 -08:00
Victor Stinner
621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Giampaolo Rodola
19c46a4c96
bpo-33695 shutil.copytree() + os.scandir() cache (#7874) 2018-11-12 06:18:15 -08:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
cd449806fa Minor grammar improvement to io documentation. (GH-10329)
Independently of -> Independent of
2018-11-11 20:06:18 -08:00
Jakub Stasiak
7432f0929f Update subprocess.Popen documentation wrt universal_newlines arg (GH-10337)
* universal_newlines defaulting to False would suggest, that not
  specifying universal_newlines explicitly and setting text to True
  should cause an error, which is not the case.
* The run function didn't have the universal_newlines parameter
  documented
* The check_output function didn't have its text parameter documented
2018-11-11 19:40:42 -08:00
Julien Palard
082875dcd6
bpo-33878: Doc: Fix missing case by simplifying. (GH-7762)
The documentation was not covering multiple targets enclosed by
parenthesis nor multiple targets enclosed by brackets, adding them all
would be heavy, an else cover them all and is lighter to read.
2018-11-12 00:59:39 +01:00
Victor Stinner
5f2df88b63
bpo-35177: Add dependencies between header files (GH-10361)
* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:

  * Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
    twice
  * Add "extern { ... }" for C++
  * Undefine "Yield" macro conflicting with winbase.h

* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files
2018-11-12 00:56:19 +01:00
Alexey Izbyshev
fd3a91cbf9 closes bpo-35204: Disable thread and memory sanitizers for address_in_range(). (GH-10442)
This function may access memory which is mapped but is considered
free by libc allocator. It behaves so by design, therefore we
need to suppress sanitizer reports.

GCC doesn't support MSan, so disable only TSan for it.
2018-11-11 15:14:51 -08:00
Raymond Hettinger
f9ec1b9f52
Neaten the code without any algorithmic change. (GH-10466)
Remove unneeded assertion (we already know so is a PySetObject *).
2018-11-11 14:35:47 -08:00
Andrés Delfino
b086c8afdb bpo-33699: Describe try's else clause with the rest of the try clause (GH-7252)
https://bugs.python.org/issue33699
2018-11-11 11:33:51 -08:00
Andrés Delfino
76e8fd7b7e dict insertion order is guaranteed since 3.7 (GH-10431) 2018-11-11 08:56:47 -08:00
Terry Jan Reedy
50ff02b431
bpo-34864: Document two IDLE on MacOS issues. (GH-10456)
The System Preferences Dock "prefer tabs always" setting disables some
IDLE features.  Menus are a bit different than as described for Windows
and Linux.
2018-11-10 23:26:31 -05:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
b9498e2367 bpo-35202: Remove unused imports in Lib directory. (GH-10446) 2018-11-10 09:22:02 +02:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
43a74abb3a bpo-35202: Remove unused imports in idlelib (GH-10438) 2018-11-10 01:45:31 -05:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
2d1bc537fe bpo-35202: Remove unused imports in Lib directory. (GH-10445) 2018-11-10 07:45:28 +02:00
David Kleuker
9d43fa1bb2 Doc: Simplify Copyright line in README (GH-10287)
This also makes it consistent with other places like:

- https://docs.python.org/dev/copyright.html
- https://www.python.org/ footer
- e42b705188/Python/getcopyright.c (L7)
2018-11-09 19:45:40 -08:00
Géry Ogam
009b2f0204 Correct a typo in the Unittest documentation (GH-10397)
Co-Authored-By: maggyero <gery.ogam@gmail.com>
2018-11-09 14:34:54 -05:00
Victor Stinner
d17a693fa0
bpo-35199: Add an internal _PyTuple_ITEMS() macro (GH-10434)
* _PyTuple_ITEMS() gives access to the tuple->ob_item field and cast the
  first argument to PyTupleObject*. This internal macro is only usable if
  Py_BUILD_CORE is defined.
* Replace &PyTuple_GET_ITEM(ob, 0) with _PyTuple_ITEMS(ob).
* Replace PyTuple_GET_ITEM(op, 1) with &_PyTuple_ITEMS(ob)[1].
2018-11-09 16:56:48 +01:00
Victor Stinner
130893debf
bpo-35081: Internal headers require Py_BUILD_CORE (GH-10363)
* All internal header files now require Py_BUILD_CORE or
  Py_BUILD_CORE_BUILTIN to be defined.
* _json.c is now compiled with Py_BUILD_CORE_BUILTIN to access
  pycore_accu.h header.
* Add an example to Modules/Setup to show how to build _json
  as a built-in module; it requires non trivial compiler options.
2018-11-09 13:03:37 +01:00
Raymond Hettinger
0a18e0510a Hoist the float conversion out of the inner loop. (GH-10430)
Currently, the *n* and *total* variables get converted to floats each time they are multiplied by random().  This minor tweak does the conversion just once and gets a small speedup (approx 3%).
2018-11-09 02:39:50 -08:00
Raymond Hettinger
cf5863faab Optimize set.pop() to advance a pointer instead of indexing. (GH-10429)
Gives approx 20% speed-up using clang depending on the number of elements in the set (the less dense the set, the more the speed-up).

Uses the same entry++ logic used elsewhere in the setobject.c code.
2018-11-09 02:31:56 -08:00
Raymond Hettinger
b83942c755 Cleanup and improve the regex tokenizer example. (GH-10426)
1) Convert weird field name "typ" to the more standard "type".
2) For the NUMBER type, convert the value to an int() or float().
3) Simplify ``group(kind)`` to the shorter and faster ``group()`` call.
4) Simplify logic go a single if-elif chain to make this easier to extend.
5) Reorder the tests to match the order the tokens are specified.
   This isn't necessary for correctness but does make the example
   easier to follow.
6) Move the "column" calculation before the if-elif chain so that
   users have the option of using this value in error messages.
2018-11-09 01:19:33 -08:00
Raymond Hettinger
216aaaa056 Fixing wording in comment. (GH-10425)
Since the n==1 case just returns *max*, it cannot be larger
than the magnitude of the vector entry.
2018-11-09 01:06:02 -08:00
Alexey Izbyshev
7a69cf47a9 bpo-35194: Fix a wrong constant in cp932 codec (GH-10420)
This typo doesn't affect the result because wrong bits are discarded
on implicit conversion to unsigned char, but it trips UBSan
with -fsanitize=implicit-integer-truncation.





https://bugs.python.org/issue35194
2018-11-08 23:12:06 -08:00
Lisa Roach
0f221d09ca
bpo-24412: Adds cleanUps for setUpClass and setUpModule. (GH-9190) 2018-11-08 18:34:33 -08:00
Gregory P. Smith
49fa4a9f1e
bpo-35193: Fix an off by one error in the RETURN_VALUE case. (GH-10418)
Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.

Do a bounds check within find_op so it can return before going past the end as a safety measure.

7db3c48833 (diff-a33329ae6ae0bb295d742f0caf93c137)
introduced this off by one error while fixing another one nearby.

This bug was shipped in all Python 3.6 and 3.7 releases.

The included unittest won't fail unless you do a clang msan build.
2018-11-08 17:55:07 -08:00
Vincent Michel
fd512d7645 bpo-35065: Remove StreamReaderProtocol._untrack_reader (#10212)
The call to `_untrack_reader` is performed too soon, causing the protocol
to forget about the reader before `connection_lost` can run and feed the
EOF to the reader. See bpo-35065.
2018-11-08 14:21:47 +02:00
Zackery Spytz
5d95312fb8 Replace dead code with an assertion in winreg.c. (GH-10028) 2018-11-08 11:45:00 +02:00
Serhiy Storchaka
a44d34e179
bpo-34966: Improve support of method aliases in pydoc. (GH-9823)
Pydoc now does not duplicate docstrings for aliases of inherited methods.
2018-11-08 08:48:11 +02:00
Pablo Galindo
6843ffe453
Revert "bpo-32409: Fix regression in activate.bat on international Windows (GH-10295)" (GH-10403)
This reverts commit c64583b6d3 due to multiple buildbot failures when building it.
2018-11-07 23:55:40 +00:00
Julien Palard
556d50d03d
Doc: Make all versions sidebars the same for consistency. (GH-10288) 2018-11-08 00:11:49 +01:00
Pablo Galindo
b93925047a
Fix the construction of subprocess.CalledProcessError in test_venv (GH-10400)
The constructor of subprocess.CalledProcessError in the check_output function had an extra None in it.
2018-11-07 22:21:17 +00:00
Julien Palard
beed84ca5e
bpo-35015: Doc: Fix internationalisation of the availability directive. (GH-10360) 2018-11-07 22:42:40 +01:00
Elvis Pranskevichus
a6e956bcb0 bpo-34726: Fix handling of hash-based pycs in zipimport. (GH-10327)
Current support for hash-based bytecode files in `zipimport` is rather
sparse, which leads to test failures when the test suite is ran with
the ``SOURCE_DATE_EPOCH`` environment variable set.

This teaches zipimport to handle hash-based pycs properly.
2018-11-07 20:34:59 +02:00
Andrés Delfino
bfe1839aa9 Add link to PEP 525 in Expressions. (GH-10333) 2018-11-07 20:12:12 +02:00
Andrés Delfino
ae31e3fbf4 glob uses fnmatch.filter instead of fnmatch since 2001. (GH-10102) 2018-11-07 20:09:11 +02:00
Andrés Delfino
c5eec4426d Add a reference to the name mangling description in the tutorial to the index. (GH-10138) 2018-11-07 19:59:45 +02:00
Andrés Delfino
cdb96f45b6 Add future_stmt to simple_stmt production list. (GH-8239) 2018-11-07 19:32:18 +02:00