Commit graph

6907 commits

Author SHA1 Message Date
Miss Islington (bot)
8673b77e25
bpo-41299: Reduce lag in Windows threading timeouts by using a higher precision time source (GH-26568) (GH-26580)
(cherry picked from commit 449e6f0ef3)

Co-authored-by: Ryan Hileman <lunixbochs@gmail.com>

Co-authored-by: Ryan Hileman <lunixbochs@gmail.com>
2021-06-20 21:12:27 +01:00
Miss Islington (bot)
976598d36b
bpo-11105: Do not crash when compiling recursive ASTs (GH-20594)
When compiling an AST object with a direct / indirect reference
cycles, on the conversion phase because of exceeding amount of
calls, a segfault was raised. This patch adds recursion guards to
places for preventing user inputs to not to crash AST but instead
raise a RecursionError.
(cherry picked from commit f3491242e4)

Co-authored-by: Batuhan Taskaya <batuhan@python.org>
2021-06-03 13:27:00 -07:00
Mark Shannon
cea0585b79
[3.10] bpo-44298: Backport #26513 to 3.10 (#26516)
* Backport 937cebc93 to 3.10

* Update importlib
2021-06-03 19:57:31 +01:00
Miss Islington (bot)
2d780237d9
bpo-28146: Fix a confusing error message in str.format() (GH-24213)
Automerge-Triggered-By: GH:pitrou
(cherry picked from commit 4aeee0b47b)

Co-authored-by: Irit Katriel <iritkatriel@yahoo.com>
2021-05-13 14:24:49 -07:00
Mark Shannon
0acdf255a5
[3.10] bpo-43933: Force RETURN_VALUE bytecodes to have line numbers (GH-26061)
* Guarantee that line number is set for returns.
2021-05-13 14:11:41 +01:00
Miss Islington (bot)
13de28f17a
bpo-44063: set the missing end locations on the compiler (GH-25956) (#25972)
(cherry picked from commit b2ec37a722)

Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2021-05-07 23:40:09 +03:00
Miss Islington (bot)
569ca81adf
bpo-40943: Fix skipitem() didn't raise SystemError (GH-25937)
`convertitem()` raises `SystemError` when 'GH-' is used without `PY_SSIZE_T_CLEAN`.
This commit makes `skipitem()` raise it too.
(cherry picked from commit 4ebf4a6bfa)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2021-05-06 20:18:42 -07:00
Mark Shannon
8d4b18429d
Correct location for syntax error in try-except (GH-25939) 2021-05-06 13:38:50 +01:00
Dennis Sweeney
80a2a4ed7d
bpo-38530: Refactor and improve AttributeError suggestions (GH-25776)
- Make case-swaps half the cost of any other edit
- Refactor Levenshtein code to not use memory allocator, and to bail early on no match.
- Add comments to Levenshtein distance code
- Add test cases for Levenshtein distance behind a debug macro
- Set threshold to `(name_size + item_size + 3) * MOVE_COST / 6`.
  - Reasoning: similar to `difflib.SequenceMatcher.ratio()` >= 2/3:
```
"Multiset Jaccard similarity" >= 2/3
matching letters / total letters >= 2/3
(name_size - distance + item_size - distance) / (name_size + item_size) >= 2/3
1 - (2*distance) / (name_size + item_size) >= 2/3
1/3 >= (2*distance) / (name_size + item_size)
(name_size + item_size) / 6 >= distance
With rounding:
(name_size + item_size + 3) // 6 >= distance
```

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-05-03 16:47:27 +01:00
Pablo Galindo
39494285e1
bpo-43754: Fix compiler warning in Python/compile.c (GH-25855)
This fixes the following warning:

'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
2021-05-03 16:20:46 +01:00
Pablo Galindo
c2931d31f8
bpo-43916: Move the _PyStructSequence_InitType function to the internal API (GH-25854) 2021-05-03 15:50:24 +01:00
Batuhan Taskaya
ad106c68eb
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583) 2021-05-03 10:43:00 +03:00
Brandt Bucher
0ad1e0384c
bpo-43754: Eliminate bindings for partial pattern matches (GH-25229) 2021-05-02 13:02:10 -07:00
Mark Shannon
5979e81a21
bpo-43933: Set frame.f_lineno during call to __exit__ (GH-25719)
* Set line number of __exit__ call in a with statement to be that of the with keyword.
2021-04-30 14:32:47 +01:00
Jakub Kulík
9032cf5cb1
bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris (GH-25096) 2021-04-30 15:21:42 +02:00
Victor Stinner
4908fae3d5
bpo-43916: PyStdPrinter_Type uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25749)
The PyStdPrinter_Type type now uses the
Py_TPFLAGS_DISALLOW_INSTANTIATION flag to disallow instantiation,
rather than seting a tp_init method which always fail.

Write also unit tests for PyStdPrinter_Type.
2021-04-30 14:56:27 +02:00
Victor Stinner
3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
Mark Shannon
069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
Brandt Bucher
dbe60ee09d
bpo-43892: Validate the first term of complex literal value patterns (GH-25735) 2021-04-29 17:19:28 -07:00
Steve Dower
87655e2cf5
bpo-42800: Rename AUDIT_READ to PY_AUDIT_READ (GH-25736) 2021-04-30 01:08:55 +01:00
Ryan Hileman
9a2c2a9ec3
bpo-42800: add audit hooks for f_code and tb_frame (GH-24182)
Accessing the following attributes will now fire PEP 578 style audit hooks as ("object.__getattr__", obj, name):
* PyTracebackObject: tb_frame
* PyFrameObject: f_code
* PyGenObject: gi_code, gi_frame
* PyCoroObject: cr_code, cr_frame
* PyAsyncGenObject: ag_code, ag_frame
Add an AUDIT_READ attribute flag aliased to READ_RESTRICTED.
Update obsolete flag documentation.
2021-04-30 00:15:55 +01:00
Mark Shannon
c76da79b37
bpo-42739: Don't use sentinels to mark end of line table. (GH-25657)
* Add length parameter to PyLineTable_InitAddressRange and doen't use sentinel values at end of table. Makes the line number table more robust.

* Update PyCodeAddressRange to match PEP 626.
2021-04-29 13:12:51 +01:00
Nick Coghlan
1e7b858575
bpo-43892: Make match patterns explicit in the AST (GH-25585)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-28 22:58:44 -07:00
Victor Stinner
32c5a17444
bpo-43962: Fix _PyInterpreterState_IDIncref() (GH-25683)
_PyInterpreterState_IDIncref() now calls
_PyInterpreterState_IDInitref() and always increments id_refcount.
2021-04-28 13:40:44 +02:00
Dennis Sweeney
284c52da09
bpo-38530: Require 50% similarity in NameError and AttributeError suggestions (GH-25584) 2021-04-27 01:22:27 +01:00
Serhiy Storchaka
face87c94e
bpo-42609: Check recursion depth in the AST validator and optimizer (GH-23744) 2021-04-25 13:38:00 +03:00
Batuhan Taskaya
8cc3cfa8af
bpo-42737: annotations with complex targets no longer causes any runtime effects (GH-23952) 2021-04-25 05:31:20 +03:00
Segev Finer
5e437fb872
bpo-30555: Fix WindowsConsoleIO fails in the presence of fd redirection (GH-1927)
This works by not caching the handle and instead getting the handle from
the file descriptor each time, so that if the actual handle changes by
fd redirection closing/opening the console handle beneath our feet, we
will keep working correctly.
2021-04-23 23:00:27 +01:00
Pablo Galindo
a77aac4fca
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525)
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
2021-04-23 14:27:05 +01:00
Raymond Hettinger
a07da09ad5
bpo-43475: Fix worst case collision behavior for NaN instances (GH-25493) 2021-04-22 08:34:57 -07:00
Pablo Galindo
b0544ba77c
bpo-38605: Revert making 'from __future__ import annotations' the default (GH-25490)
This reverts commits 044a1048ca and 1be456ae9d, adapting the code to changes that happened after it.
2021-04-21 12:41:19 +01:00
Pablo Galindo
7a04116246
bpo-25460: Surround suggestions by quotes (GH-25473) 2021-04-19 23:35:53 +01:00
Pablo Galindo
0b1c169c4a
bpo-38530: Cover more error paths in error suggestion functions (GH-25462) 2021-04-17 23:28:45 +01:00
Pablo Galindo
3ab4bea5a3
bpo-38530: Include builtins in NameError suggestions (GH-25460) 2021-04-17 22:26:54 +01:00
Pablo Galindo
0ad81d4db2
bpo-38530: Match exactly AttributeError and NameError when offering suggestions (GH-25443) 2021-04-16 17:12:03 +01:00
Inada Naoki
8a232c7b17
bpo-41323: compiler: Reuse tuple in constant tuple folding (GH-25419) 2021-04-16 14:01:04 +09:00
Mark Shannon
11e0b295de
bpo-43846: Use less stack for large literals and calls (GH-25403)
* Modify compiler to reduce stack consumption for large expressions.

* Add more tests for stack usage.

* Add NEWS item.

* Raise SystemError for truly excessive stack use.
2021-04-15 14:28:56 +01:00
Pablo Galindo
3fc65b97d0
bpo-38530: Optimize the calculation of string sizes when offering suggestions (GH-25412) 2021-04-15 00:03:43 +01:00
Pablo Galindo
e07f4ab26a
bpo-38530: Make sure that failing to generate suggestions on failure will not propagate exceptions (GH-25408) 2021-04-14 18:58:28 +01:00
Pablo Galindo
5bf8bf2267
bpo-38530: Offer suggestions on NameError (GH-25397)
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:

    >>> schwarzschild_black_hole = None
    >>> schwarschild_black_hole
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
2021-04-14 15:10:33 +01:00
Pablo Galindo
37494b441a
bpo-38530: Offer suggestions on AttributeError (#16856)
When printing AttributeError, PyErr_Display will offer suggestions of similar 
attribute names in the object that the exception was raised from:

>>> collections.namedtoplo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?
2021-04-14 02:36:07 +01:00
Mark Shannon
9e7b2076fb
bpo-43760: Speed up check for tracing in interpreter dispatch (#25276)
* Remove redundant tracing_possible field from interpreter state.

* Move 'use_tracing' from tstate onto C stack, for fastest possible checking in dispatch logic.

* Add comments stressing the importance stack discipline when dealing with CFrames.

* Add NEWS
2021-04-13 11:08:14 +01:00
Victor Stinner
77d668b122
bpo-43680: _pyio.open() becomes a static method (GH-25354)
The Python _pyio.open() function becomes a static method to behave as
io.open() built-in function: don't become a bound method when stored
as a class variable. It becomes possible since static methods are now
callable in Python 3.10. Moreover, _pyio.OpenWrapper becomes a simple
alias to _pyio.open.

init_set_builtins_open() now sets builtins.open to io.open, rather
than setting it to io.OpenWrapper, since OpenWrapper is now an alias
to open in the io and _pyio modules.
2021-04-12 10:44:53 +02:00
Victor Stinner
09bbebea16
bpo-43753: Add Py_Is() and Py_IsNone() functions (GH-25227)
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
2021-04-11 00:17:39 +02:00
Matthew Suozzo
75a06f067b
bpo-43798: Add source location attributes to alias (GH-25324)
* Add source location attributes to alias.
* Move alias star construction to pegen helper.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-04-10 22:56:28 +02:00
Mark Shannon
28d28e053d
bpo-43760: Streamline dispatch sequence for machines without computed gotos. (GH-25244)
* Do fetch and decode at end of opcode then jump directly to switch.
  Should allow compilers that don't support computed-gotos, specifically MSVC,
  to generate better code.
2021-04-08 11:22:55 +01:00
Victor Stinner
3d55aa9e73
bpo-42923: Fix _Py_DumpExtensionModules() for NULL sysdict (GH-25262)
Fix Py_FatalError() is called before interp->sysdict is set.
2021-04-07 23:12:45 +02:00
Victor Stinner
d27f8d2e07
bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)
Rename AST functions of pycore_ast.h to use the "_PyAST_" prefix.
Remove macros creating aliases without prefix. For example, Module()
becomes _PyAST_Module(). Update Grammar/python.gram to use
_PyAST_xxx() functions.
2021-04-07 21:34:22 +02:00
tomKPZ
7a7ba3d343
bpo-43495 : Push missing frame block in compile.c (GH-24865) 2021-04-07 15:43:45 +01:00
Victor Stinner
d36d6a9c18
bpo-43244: Remove Yield macro from pycore_ast.h (GH-25243)
* pycore_ast.h no longer defines the Yield macro.
* Fix a compiler warning on Windows: "warning C4005: 'Yield': macro
  redefinition".
* Python-ast.c now defines directly functions with their real
  _Py_xxx() name, rather than xxx().
* Remove "#undef Yield" in C files including pycore_ast.h.
2021-04-07 13:01:09 +02:00