Commit graph

7067 commits

Author SHA1 Message Date
Serhiy Storchaka
c3007ab3c6
[3.10] bpo-44636: Collapse union of equal types (GH-27178) (GH-27181)
The result of `int | int` is now `int`.

Fix comparison of the union type with non-hashable objects.
`int | str == {}` no longer raises a TypeError.
(cherry picked from commit d9f923280f)
2021-07-16 14:48:20 +03:00
Miss Islington (bot)
705988056e
bpo-44646: Fix the hash of the union type. (GH-27179) (#27180)
It no longer depends on the order of arguments.
hash(int | str) == hash(str | int)

Co-authored-by: Jack DeVries <58614260+jdevries3133@users.noreply.github.com>
(cherry picked from commit aeaa553d65)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-07-16 11:02:59 +02:00
Miss Islington (bot)
6aa59c68dc
bpo-44184: Apply GH-26274 to the non-GC-type branch of subtype_dealloc (GH-27165) (GH-27174)
The non-GC-type branch of subtype_dealloc is using the type of an object after freeing in the same unsafe way as GH-26274 fixes. (I believe the old news entry covers this change well enough.)

https://bugs.python.org/issue44184
(cherry picked from commit 074e7659f2)

Co-authored-by: T. Wouters <thomas@python.org>
2021-07-16 01:18:16 +02:00
Miss Islington (bot)
2ce8af3cbc
bpo-42073: allow classmethod to wrap other classmethod-like descriptors (GH-27115) (GH-27162)
Patch by Erik Welch.

bpo-19072 (GH-8405) allows `classmethod` to wrap other descriptors, but this does
not work when the wrapped descriptor mimics classmethod.  The current PR fixes
this.

In Python 3.8 and before, one could create a callable descriptor such that this
works as expected (see Lib/test/test_decorators.py for examples):
```python
class A:
    @myclassmethod
    def f1(cls):
        return cls

    @classmethod
    @myclassmethod
    def f2(cls):
        return cls
```
In Python 3.8 and before, `A.f2()` return `A`. Currently in Python 3.9, it
returns `type(A)`.  This PR make `A.f2()` return `A` again.

As of GH-8405, classmethod calls `obj.__get__(type)` if `obj` has `__get__`.
This allows one to chain `@classmethod` and `@property` together.  When
using classmethod-like descriptors, it's the second argument to `__get__`--the
owner or the type--that is important, but this argument is currently missing.
Since it is None, the "owner" argument is assumed to be the type of the first
argument, which, in this case, is wrong (we want `A`, not `type(A)`).

This PR updates classmethod to call `obj.__get__(type, type)` if `obj` has
`__get__`.

Co-authored-by: Erik Welch <erik.n.welch@gmail.com>
(cherry picked from commit b83861f026)
2021-07-15 15:42:11 +02:00
Miss Islington (bot)
cc1a47c849
bpo-44632: Fix support of TypeVar in the union type (GH-27139) (GH-27143)
int | TypeVar('T') returns now an instance of types.Union
instead of typing.Union.
(cherry picked from commit a158b20019)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-07-15 10:25:22 +03:00
Serhiy Storchaka
6dec525582
[3.10] bpo-44635: Convert None to NoneType in the union type constructor (GH-27136). (GH-27142)
(cherry picked from commit b81cac0560)
2021-07-15 10:15:14 +03:00
Miss Islington (bot)
b42eee78e7
bpo-44606: Fix __instancecheck__ and __subclasscheck__ for the union type. (GH-27120)
* Fix issubclass() for None.
  E.g. issubclass(type(None), int | None) returns now True.
* Fix issubclass() for virtual subclasses.
  E.g. issubclass(dict, int | collections.abc.Mapping) returns now True.
* Fix crash in isinstance() if the check for one of items raises exception.
(cherry picked from commit 81989058de)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-07-13 21:55:45 -07:00
Miss Islington (bot)
d17cc1ff9f
bpo-44562: Use PyType_GenericAlloc in Py_GenericAlias (GH-27021)
* bpo-44562: Use PyType_GenericAlloc in Py_GenericAlias

* Update Objects/genericaliasobject.c

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit b324c4c5f7)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-07-05 04:33:53 -07:00
Miss Islington (bot)
68330b681a
bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016)
(cherry picked from commit d33943a6c3)

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
2021-07-04 10:55:35 -07:00
Miss Islington (bot)
000b9e803a
bpo-44553: Correct failure in tp_new for the union object (GH-27008) (GH-27009)
(cherry picked from commit bc39614856)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-07-03 21:51:10 +01:00
Miss Islington (bot)
0856134542
bpo-44553 : Implement GC methods for types.Union (GH-26993)
(cherry picked from commit 1097384ce9)

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
2021-07-03 06:33:16 -07:00
Miss Islington (bot)
2df13e1211
bpo-44523: Remove the pass-through for hash() in weakref proxy objects (GH-26950)
(cherry picked from commit e2fea101fd)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-06-29 16:19:06 -07:00
Miss Islington (bot)
7e6cad7e30
bpo-44483: Fix crash in union object with bad `__module__` (GH-26848) (GH-26852)
(cherry picked from commit adfa1ba398)

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
2021-06-23 12:38:49 +03:00
Mark Dickinson
929dd817b3
[3.10] Fix a potential reference-counting bug in long_pow (GH-26690) (GH-26703)
(cherry picked from commit 5924243199)

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
2021-06-13 08:58:32 +01:00
Serhiy Storchaka
c43317d41e
[3.10] Add more const modifiers. (GH-26691). (GH-26692)
(cherry picked from commit be8b631b7a)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-06-12 18:44:32 +01:00
Miss Islington (bot)
7b3b6982a5
bpo-44232: Fix type_new() error reporting (GH-26359) (GH-26365)
Fix a regression in type() when a metaclass raises an exception. The
C function type_new() must properly report the exception when a
metaclass constructor raises an exception and the winner class is not
the metaclass.
(cherry picked from commit bd199e72fb)

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-05-26 11:30:55 +02:00
Miss Islington (bot)
50b0d148a6
bpo-44184: Fix subtype_dealloc() for freed type (GH-26274) (GH-26290)
Fix a crash at Python exit when a deallocator function removes the
last strong reference to a heap type.

Don't read type memory after calling basedealloc() since
basedealloc() can deallocate the type and free its memory.

_PyMem_IsPtrFreed() argument is now constant.
(cherry picked from commit 615069eb08)

Co-authored-by: Victor Stinner <vstinner@python.org>

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-05-22 00:50:11 +02:00
Miss Islington (bot)
c4c3beb5ad
bpo-44114: Remove redundant cast. (GH-26098)
(cherry picked from commit e0c614e5fd)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2021-05-13 14:07:31 -07:00
Miss Islington (bot)
7cbe6ca634
bpo-44114: Fix dictkeys_reversed and dictvalues_reversed function signatures (GH-26062)
These are passed and called as PyCFunction, however they are defined here without the (ignored) args parameter.

This works fine in some C compilers, but fails in webassembly or anything else that has strict function pointer call type checking.
(cherry picked from commit ab383eb6f0)

Co-authored-by: Joe Marshall <joe.marshall@nottingham.ac.uk>
2021-05-13 01:17:07 -07:00
Pablo Galindo
c2931d31f8
bpo-43916: Move the _PyStructSequence_InitType function to the internal API (GH-25854) 2021-05-03 15:50:24 +01:00
Mark Shannon
33ec88ac81
bpo-43977: Make sure that tp_flags for pattern matching are inherited correctly. (GH-25813) 2021-05-03 00:38:22 +01:00
Pablo Galindo
e374a40afa
bpo-43901: Fix refleaks in test_module (GH-25754) 2021-04-30 17:26:45 +01:00
Erlend Egeberg Aasland
64141382ec
bpo-43908: check_set_special_type_attr() checks Py_TPFLAGS_IMMUTABLETYPE (GH-25743)
check_set_special_type_attr() and type_set_annotations()
now check for immutable flag (Py_TPFLAGS_IMMUTABLETYPE).

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-30 15:25:43 +02:00
Jakub Kulík
9032cf5cb1
bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris (GH-25096) 2021-04-30 15:21:42 +02:00
Victor Stinner
4908fae3d5
bpo-43916: PyStdPrinter_Type uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25749)
The PyStdPrinter_Type type now uses the
Py_TPFLAGS_DISALLOW_INSTANTIATION flag to disallow instantiation,
rather than seting a tp_init method which always fail.

Write also unit tests for PyStdPrinter_Type.
2021-04-30 14:56:27 +02:00
Victor Stinner
3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
Erlend Egeberg Aasland
b73b5fb9ea
bpo-43973: object_set_class() checks Py_TPFLAGS_IMMUTABLETYPE (GH-25714)
Use Py_TPFLAGS_IMMUTABLETYPE to check for class assignments.
2021-04-30 12:07:02 +02:00
Mark Shannon
069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
larryhastings
175a54b2d8
Two minor fixes for accessing a module's name. (#25658)
While working on another issue, I noticed two minor nits in the C implementation of the module object.  Both are related to getting a module's name.

First, the C function module_dir() (module.__dir__) starts by ensuring the module dict is valid.  If the module dict is invalid, it wants to format an exception using the name of the module, which it gets from PyModule_GetName().  However, PyModule_GetName() gets the name of the module from the dict.  So getting the name in this circumstance will never succeed.

When module_dir() wants to format the error but can't get the name, it knows that PyModule_GetName() must have already raised an exception.  So it leaves that exception alone and returns an error.  The end result is that the exception raised here is kind of useless and misleading: dir(module) on a module with no __dict__ raises SystemError("nameless module").  I changed the code to actually raise the exception it wanted to raise, just without a real module name: TypeError("<module>.__dict__ is not a dictionary").  This seems more useful, and would do a better job putting the programmer who encountered this on the right track of figuring out what was going on.

Second, the C API function PyModule_GetNameObject() checks to see if the module has a dict.  If m->md_dict is not NULL, it calls _PyDict_GetItemIdWithError().  However, it's possible for m->md_dict to be None.  And if you call _PyDict_GetItemIdWithError(Py_None, ...) it will *crash*.

Unfortunately, this crash was due to my own bug in the other branch.  Fixing my code made the crash go away.  I assert that this is still possible at the API level.

The fix is easy: add a PyDict_Check() to PyModule_GetNameObject().

Unfortunately, I don't know how to add a unit test for this.  Having changed module_dir() above, I can't find any other interfaces callable from Python that eventually call PyModule_GetNameObject().  So I don't know how to trick the runtime into reproducing this error.

Since both these changes are minor--each entails only a small edit to only one line--I didn't bother with a news item.
2021-04-29 20:13:25 -07:00
larryhastings
2f2b69855d
bpo-43901: Lazy-create an empty annotations dict in all unannotated user classes and modules (#25623)
Change class and module objects to lazy-create empty annotations dicts on demand.  The annotations dicts are stored in the object's `__dict__` for backwards compatibility.
2021-04-29 20:09:08 -07:00
Steve Dower
87655e2cf5
bpo-42800: Rename AUDIT_READ to PY_AUDIT_READ (GH-25736) 2021-04-30 01:08:55 +01:00
Ryan Hileman
9a2c2a9ec3
bpo-42800: add audit hooks for f_code and tb_frame (GH-24182)
Accessing the following attributes will now fire PEP 578 style audit hooks as ("object.__getattr__", obj, name):
* PyTracebackObject: tb_frame
* PyFrameObject: f_code
* PyGenObject: gi_code, gi_frame
* PyCoroObject: cr_code, cr_frame
* PyAsyncGenObject: ag_code, ag_frame
Add an AUDIT_READ attribute flag aliased to READ_RESTRICTED.
Update obsolete flag documentation.
2021-04-30 00:15:55 +01:00
Mark Shannon
088a15c49d
bpo-43933: Show frame.f_lineno as None, rather than -1, if there is no line number. (GH-25717) 2021-04-29 19:28:50 +01:00
Mark Shannon
c76da79b37
bpo-42739: Don't use sentinels to mark end of line table. (GH-25657)
* Add length parameter to PyLineTable_InitAddressRange and doen't use sentinel values at end of table. Makes the line number table more robust.

* Update PyCodeAddressRange to match PEP 626.
2021-04-29 13:12:51 +01:00
Hristo Venev
8557edbfa8
bpo-24275: Don't downgrade unicode-only dicts to mixed on lookups (GH-25186) 2021-04-29 11:06:03 +09:00
Erlend Egeberg Aasland
3b52c8d66b
bpo-43908: Add Py_TPFLAGS_IMMUTABLETYPE flag (GH-25520)
Introduce Py_TPFLAGS_IMMUTABLETYPE flag for immutable type objects, and
modify PyType_Ready() to set it for static types.

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-28 19:02:42 +02:00
Ken Jin
859577c249
bpo-41559: Change PEP 612 implementation to pure Python (#25449) 2021-04-28 08:38:14 -07:00
Victor Stinner
32c5a17444
bpo-43962: Fix _PyInterpreterState_IDIncref() (GH-25683)
_PyInterpreterState_IDIncref() now calls
_PyInterpreterState_IDInitref() and always increments id_refcount.
2021-04-28 13:40:44 +02:00
Pablo Galindo
a77aac4fca
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525)
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
2021-04-23 14:27:05 +01:00
Raymond Hettinger
a07da09ad5
bpo-43475: Fix worst case collision behavior for NaN instances (GH-25493) 2021-04-22 08:34:57 -07:00
Victor Stinner
cdad2724e6
bpo-40137: Add pycore_moduleobject.h internal header (GH-25507)
Add pycore_moduleobject.h internal header file with static inline
functions to access module members:

* _PyModule_GetDict()
* _PyModule_GetDef()
* _PyModule_GetState()

These functions don't check at runtime if their argument has a valid
type and can be inlined even if Python is not built with LTO.

_PyType_GetModuleByDef() uses _PyModule_GetDef().

Replace PyModule_GetState() with _PyModule_GetState() in the
extension modules, considered as performance sensitive:

* _abc
* _functools
* _operator
* _pickle
* _queue
* _random
* _sre
* _struct
* _thread
* _winapi
* array
* posix

The following extensions are now built with the Py_BUILD_CORE_MODULE
macro defined, to be able to use the internal pycore_moduleobject.h
header: _abc, array, _operator, _queue, _sre, _struct.
2021-04-22 00:52:52 +02:00
Victor Stinner
760da626ff
bpo-40137: Optimize _PyType_GetModuleByDef() loop (GH-25505)
PyType_Ready() now ensures that a type MRO cannot be empty.

_PyType_GetModuleByDef() no longer checks "i < PyTuple_GET_SIZE(mro)"
at the first loop iteration to optimize the most common case, when
the argument is the defining class.
2021-04-22 00:10:16 +02:00
Victor Stinner
d4aaa34798
bpo-40137: _PyType_GetModuleByDef() doesn't check tp_flags (GH-25504)
_PyType_GetModuleByDef() no longer checks if types are heap types.

_PyType_GetModuleByDef() must only be called on a heap type created
by PyType_FromModuleAndSpec() or on its subclasses.
type_ready_mro() ensures that a static type cannot inherit from a
heap type.
2021-04-21 23:36:26 +02:00
Pablo Galindo
3b82cae774
bpo-38530: Properly extend UnboundLocalError from NameError (GH-25444) 2021-04-16 16:38:55 +01:00
Pablo Galindo
5bf8bf2267
bpo-38530: Offer suggestions on NameError (GH-25397)
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:

    >>> schwarzschild_black_hole = None
    >>> schwarschild_black_hole
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
2021-04-14 15:10:33 +01:00
Pablo Galindo
37494b441a
bpo-38530: Offer suggestions on AttributeError (#16856)
When printing AttributeError, PyErr_Display will offer suggestions of similar 
attribute names in the object that the exception was raised from:

>>> collections.namedtoplo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?
2021-04-14 02:36:07 +01:00
Victor Stinner
a328d73843
bpo-43770: Cleanup type_ready() (GH-25388)
* Rename functions
* Only pass type parameter to "add_xxx" functions.
* Clarify the role of the type_ready_inherit_as_structs() function.
* Move type_dict_set_doc() code to call it in type_ready_fill_dict().
2021-04-13 15:25:16 +02:00
Victor Stinner
65f058eb08
bpo-43770: Reorder type_ready() (GH-25373)
Add type_ready_create_dict() sub-function.

* Start with type_ready_create_dict().
* Call type_ready_mro() earlier, before type_ready_add_attrs().
* Call type_ready_inherit_special() earlier, in type_ready_inherit().
2021-04-13 13:33:31 +02:00
Victor Stinner
553ee2781a
bpo-43682: Make staticmethod objects callable (GH-25117)
Static methods (@staticmethod) are now callable as regular functions.
2021-04-12 00:21:22 +02:00
Victor Stinner
53114ffef1
bpo-43770: Refactor PyType_Ready() function (GH-25336)
* Split PyType_Ready() into sub-functions.
* type_ready_mro() now checks if bases are static types earlier.
* Check tp_name earlier, in type_ready_checks().
* Add _PyType_IsReady() macro to check if a type is ready.
2021-04-11 23:57:09 +02:00