mirrors/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-09-29 05:14:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

[ty] Add support for PEP 800 (#20084)

Browse source
This commit is contained in:
Alex Waygood 2025-08-25 19:39:05 +01:00 • committed by GitHub
parent 33c5f6f4f8
commit ecf3c4ca11
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 275 additions and 271 deletions
Download patch file Download diff file Expand all files Collapse all files

2
crates/ty/docs/rules.md generated
View file

@ -422,7 +422,7 @@ class D(A, B, C): ...
**Known problems** **Known problems**
Classes that have "dynamic" definitions of `__slots__` (definitions do not consist Classes that have "dynamic" definitions of `__slots__` (definitions do not consist
of string literals, or tuples of string literals) are not currently considered solid of string literals, or tuples of string literals) are not currently considered disjoint
bases by ty. bases by ty.
Additionally, this check is not exhaustive: many C extensions (including several in Additionally, this check is not exhaustive: many C extensions (including several in

30
crates/ty_python_semantic/resources/mdtest/instance_layout_conflict.md
View file

@ -103,7 +103,7 @@ class E( # error: [instance-layout-conflict]
): ... ): ...
``` ```
## A single "solid base" ## A single "disjoint base"
```py ```py
class A: class A:
@ -152,14 +152,15 @@ class Baz(Foo, Bar): ... # fine
<!-- snapshot-diagnostics --> <!-- snapshot-diagnostics -->
Certain classes implemented in C extensions also have an extended instance memory layout, in the Certain classes implemented in C extensions also have an extended instance memory layout, in the
same way as classes that define non-empty `__slots__`. (CPython internally calls all such classes same way as classes that define non-empty `__slots__`. CPython internally calls all such classes
with a unique instance memory layout "solid bases", and we also borrow this term.) There is with a unique instance memory layout "solid bases", but [PEP 800](https://peps.python.org/pep-0800/)
currently no generalized way for ty to detect such a C-extension class, as there is currently no way calls these classes "disjoint bases", and this is the term we generally use. The `@disjoint_base`
of expressing the fact that a class is a solid base in a stub file. However, ty special-cases decorator introduced by this PEP provides a generalised way for type checkers to identify such
certain builtin classes in order to detect that attempting to combine them in a single MRO would classes.
fail:
```py ```py
from typing_extensions import disjoint_base
# fmt: off # fmt: off
class A( # error: [instance-layout-conflict] class A( # error: [instance-layout-conflict]
@ -183,6 +184,17 @@ class E( # error: [instance-layout-conflict]
class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict] class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict]
@disjoint_base
class G: ...
@disjoint_base
class H: ...
class I( # error: [instance-layout-conflict]
G,
H
): ...
# fmt: on # fmt: on
``` ```
@ -193,9 +205,9 @@ We avoid emitting an `instance-layout-conflict` diagnostic for this class defini
class Foo(range, str): ... # error: [subclass-of-final-class] class Foo(range, str): ... # error: [subclass-of-final-class]
``` ```
## Multiple "solid bases" where one is a subclass of the other ## Multiple "disjoint bases" where one is a subclass of the other
A class is permitted to multiple-inherit from multiple solid bases if one is a subclass of the A class is permitted to multiple-inherit from multiple disjoint bases if one is a subclass of the
other: other:
```py ```py

202
crates/ty_python_semantic/resources/mdtest/snapshots/instance_layout_conf…_-_Tests_for_ty's_`inst…_-_Built-ins_with_impli…_(f5857d64ce69ca1d).snap
View file

@ -12,59 +12,72 @@ mdtest path: crates/ty_python_semantic/resources/mdtest/instance_layout_conflict
## mdtest_snippet.py ## mdtest_snippet.py
``` ```
1 | # fmt: off 1 | from typing_extensions import disjoint_base
2 | 2 |
3 | class A( # error: [instance-layout-conflict] 3 | # fmt: off
4 | int, 4 |
5 | str 5 | class A( # error: [instance-layout-conflict]
6 | ): ... 6 | int,
7 | 7 | str
8 | class B: 8 | ): ...
9 | __slots__ = ("b",) 9 |
10 | 10 | class B:
11 | class C( # error: [instance-layout-conflict] 11 | __slots__ = ("b",)
12 | int, 12 |
13 | B, 13 | class C( # error: [instance-layout-conflict]
14 | ): ... 14 | int,
15 | class D(int): ... 15 | B,
16 | 16 | ): ...
17 | class E( # error: [instance-layout-conflict] 17 | class D(int): ...
18 | D, 18 |
19 | str 19 | class E( # error: [instance-layout-conflict]
20 | ): ... 20 | D,
21 | 21 | str
22 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict] 22 | ): ...
23 | 23 |
24 | # fmt: on 24 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict]
25 | class Foo(range, str): ... # error: [subclass-of-final-class] 25 |
26 | @disjoint_base
27 | class G: ...
28 |
29 | @disjoint_base
30 | class H: ...
31 |
32 | class I( # error: [instance-layout-conflict]
33 | G,
34 | H
35 | ): ...
36 |
37 | # fmt: on
38 | class Foo(range, str): ... # error: [subclass-of-final-class]
``` ```
# Diagnostics # Diagnostics
``` ```
error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases
--> src/mdtest_snippet.py:3:7 --> src/mdtest_snippet.py:5:7
| |
1 | # fmt: off 3 | # fmt: off
2 | 4 |
3 | class A( # error: [instance-layout-conflict] 5 | class A( # error: [instance-layout-conflict]
| _______^ | _______^
4 | | int, 6 | | int,
5 | | str 7 | | str
6 | | ): ... 8 | | ): ...
| |_^ Bases `int` and `str` cannot be combined in multiple inheritance | |_^ Bases `int` and `str` cannot be combined in multiple inheritance
7 | 9 |
8 | class B: 10 | class B:
| |
info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts
--> src/mdtest_snippet.py:4:5 --> src/mdtest_snippet.py:6:5
| |
3 | class A( # error: [instance-layout-conflict] 5 | class A( # error: [instance-layout-conflict]
4 | int, 6 | int,
| --- `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension | --- `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension
5 | str 7 | str
| --- `str` instances have a distinct memory layout because of the way `str` is implemented in a C extension | --- `str` instances have a distinct memory layout because of the way `str` is implemented in a C extension
6 | ): ... 8 | ): ...
| |
info: rule `instance-layout-conflict` is enabled by default info: rule `instance-layout-conflict` is enabled by default
@ -72,28 +85,28 @@ info: rule `instance-layout-conflict` is enabled by default
``` ```
error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases
--> src/mdtest_snippet.py:11:7 --> src/mdtest_snippet.py:13:7
| |
9 | __slots__ = ("b",) 11 | __slots__ = ("b",)
10 | 12 |
11 | class C( # error: [instance-layout-conflict] 13 | class C( # error: [instance-layout-conflict]
| _______^ | _______^
12 | | int, 14 | | int,
13 | | B, 15 | | B,
14 | | ): ... 16 | | ): ...
| |_^ Bases `int` and `B` cannot be combined in multiple inheritance | |_^ Bases `int` and `B` cannot be combined in multiple inheritance
15 | class D(int): ... 17 | class D(int): ...
| |
info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts
--> src/mdtest_snippet.py:12:5 --> src/mdtest_snippet.py:14:5
| |
11 | class C( # error: [instance-layout-conflict] 13 | class C( # error: [instance-layout-conflict]
12 | int, 14 | int,
| --- `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension | --- `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension
13 | B, 15 | B,
| - `B` instances have a distinct memory layout because `B` defines non-empty `__slots__` | - `B` instances have a distinct memory layout because `B` defines non-empty `__slots__`
14 | ): ... 16 | ): ...
15 | class D(int): ... 17 | class D(int): ...
| |
info: rule `instance-layout-conflict` is enabled by default info: rule `instance-layout-conflict` is enabled by default
@ -101,31 +114,31 @@ info: rule `instance-layout-conflict` is enabled by default
``` ```
error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases
--> src/mdtest_snippet.py:17:7 --> src/mdtest_snippet.py:19:7
| |
15 | class D(int): ... 17 | class D(int): ...
16 | 18 |
17 | class E( # error: [instance-layout-conflict] 19 | class E( # error: [instance-layout-conflict]
| _______^ | _______^
18 | | D, 20 | | D,
19 | | str 21 | | str
20 | | ): ... 22 | | ): ...
| |_^ Bases `D` and `str` cannot be combined in multiple inheritance | |_^ Bases `D` and `str` cannot be combined in multiple inheritance
21 | 23 |
22 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict] 24 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict]
| |
info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts
--> src/mdtest_snippet.py:18:5 --> src/mdtest_snippet.py:20:5
| |
17 | class E( # error: [instance-layout-conflict] 19 | class E( # error: [instance-layout-conflict]
18 | D, 20 | D,
| - | -
| | | |
| `D` instances have a distinct memory layout because `D` inherits from `int` | `D` instances have a distinct memory layout because `D` inherits from `int`
| `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension | `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension
19 | str 21 | str
| --- `str` instances have a distinct memory layout because of the way `str` is implemented in a C extension | --- `str` instances have a distinct memory layout because of the way `str` is implemented in a C extension
20 | ): ... 22 | ): ...
| |
info: rule `instance-layout-conflict` is enabled by default info: rule `instance-layout-conflict` is enabled by default
@ -133,28 +146,57 @@ info: rule `instance-layout-conflict` is enabled by default
``` ```
error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases
--> src/mdtest_snippet.py:22:7 --> src/mdtest_snippet.py:24:7
| |
20 | ): ... 22 | ): ...
21 |
22 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Bases `int`, `str`, `bytes` and `bytearray` cannot be combined in multiple inheritance
23 | 23 |
24 | # fmt: on 24 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Bases `int`, `str`, `bytes` and `bytearray` cannot be combined in multiple inheritance
25 |
26 | @disjoint_base
| |
info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts
--> src/mdtest_snippet.py:22:9 --> src/mdtest_snippet.py:24:9
| |
20 | ): ... 22 | ): ...
21 | 23 |
22 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict] 24 | class F(int, str, bytes, bytearray): ... # error: [instance-layout-conflict]
| --- --- ----- --------- `bytearray` instances have a distinct memory layout because of the way `bytearray` is implemented in a C extension | --- --- ----- --------- `bytearray` instances have a distinct memory layout because of the way `bytearray` is implemented in a C extension
| | | | | | | |
| | | `bytes` instances have a distinct memory layout because of the way `bytes` is implemented in a C extension | | | `bytes` instances have a distinct memory layout because of the way `bytes` is implemented in a C extension
| | `str` instances have a distinct memory layout because of the way `str` is implemented in a C extension | | `str` instances have a distinct memory layout because of the way `str` is implemented in a C extension
| `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension | `int` instances have a distinct memory layout because of the way `int` is implemented in a C extension
23 | 25 |
24 | # fmt: on 26 | @disjoint_base
|
info: rule `instance-layout-conflict` is enabled by default
```
```
error[instance-layout-conflict]: Class will raise `TypeError` at runtime due to incompatible bases
--> src/mdtest_snippet.py:32:7
|
30 | class H: ...
31 |
32 | class I( # error: [instance-layout-conflict]
| _______^
33 | | G,
34 | | H
35 | | ): ...
| |_^ Bases `G` and `H` cannot be combined in multiple inheritance
36 |
37 | # fmt: on
|
info: Two classes cannot coexist in a class's MRO if their instances have incompatible memory layouts
--> src/mdtest_snippet.py:33:5
|
32 | class I( # error: [instance-layout-conflict]
33 | G,
| - `G` instances have a distinct memory layout because of the way `G` is implemented in a C extension
34 | H
| - `H` instances have a distinct memory layout because of the way `H` is implemented in a C extension
35 | ): ...
| |
info: rule `instance-layout-conflict` is enabled by default info: rule `instance-layout-conflict` is enabled by default
@ -162,10 +204,10 @@ info: rule `instance-layout-conflict` is enabled by default
``` ```
error[subclass-of-final-class]: Class `Foo` cannot inherit from final class `range` error[subclass-of-final-class]: Class `Foo` cannot inherit from final class `range`
--> src/mdtest_snippet.py:25:11 --> src/mdtest_snippet.py:38:11
| |
24 | # fmt: on 37 | # fmt: on
25 | class Foo(range, str): ... # error: [subclass-of-final-class] 38 | class Foo(range, str): ... # error: [subclass-of-final-class]
| ^^^^^ | ^^^^^
| |
info: rule `subclass-of-final-class` is enabled by default info: rule `subclass-of-final-class` is enabled by default

26
crates/ty_python_semantic/resources/mdtest/type_properties/is_disjoint_from.md
View file

@ -87,7 +87,7 @@ static_assert(is_disjoint_from(memoryview, Foo))
static_assert(is_disjoint_from(type[memoryview], type[Foo])) static_assert(is_disjoint_from(type[memoryview], type[Foo]))
``` ```
## "Solid base" builtin types ## "Disjoint base" builtin types
Most other builtins can be subclassed and can even be used in multiple inheritance. However, builtin Most other builtins can be subclassed and can even be used in multiple inheritance. However, builtin
classes *cannot* generally be used in multiple inheritance with other builtin types. This is because classes *cannot* generally be used in multiple inheritance with other builtin types. This is because
@ -95,11 +95,14 @@ the CPython interpreter considers these classes "solid bases": due to the way th
in C, they have atypical instance memory layouts. No class can ever have more than one "solid base" in C, they have atypical instance memory layouts. No class can ever have more than one "solid base"
in its MRO. in its MRO.
It's not currently possible for ty to detect in a generalized way whether a class is a "solid base" [PEP 800](https://peps.python.org/pep-0800/) provides a generalised way for type checkers to know
or not, but we special-case some commonly used builtin types: whether a class has an atypical instance memory layout via the `@disjoint_base` decorator; we
generally use the term "disjoint base" for these classes.
```py ```py
import asyncio
from typing import Any from typing import Any
from typing_extensions import disjoint_base
from ty_extensions import static_assert, is_disjoint_from from ty_extensions import static_assert, is_disjoint_from
class Foo: ... class Foo: ...
@ -114,12 +117,23 @@ static_assert(is_disjoint_from(list, dict[Any, Any]))
static_assert(is_disjoint_from(list[Foo], dict[Any, Any])) static_assert(is_disjoint_from(list[Foo], dict[Any, Any]))
static_assert(is_disjoint_from(list[Any], dict[Any, Any])) static_assert(is_disjoint_from(list[Any], dict[Any, Any]))
static_assert(is_disjoint_from(type[list], type[dict])) static_assert(is_disjoint_from(type[list], type[dict]))
static_assert(is_disjoint_from(asyncio.Task, dict))
@disjoint_base
class A: ...
@disjoint_base
class B: ...
static_assert(is_disjoint_from(A, B))
``` ```
## Other solid bases ## Other disjoint bases
As well as certain classes that are implemented in C extensions, any class that declares non-empty As well as certain classes that are implemented in C extensions, any class that declares non-empty
`__slots__` is also considered a "solid base"; these types are also considered to be disjoint by ty: `__slots__` is also considered a "disjoint base"; these types are also considered to be disjoint by
ty:
```py ```py
from ty_extensions import static_assert, is_disjoint_from from ty_extensions import static_assert, is_disjoint_from
@ -141,7 +155,7 @@ static_assert(not is_disjoint_from(B, C))
static_assert(not is_disjoint_from(type[B], type[C])) static_assert(not is_disjoint_from(type[B], type[C]))
``` ```
Two solid bases are not disjoint if one inherits from the other, however: Two disjoint bases are not disjoint if one inherits from the other, however:
```py ```py
class D(A): class D(A):

181
crates/ty_python_semantic/src/types/class.rs
View file

@ -465,9 +465,9 @@ impl<'db> ClassType<'db> {
class_literal.definition(db) class_literal.definition(db)
} }
/// Return `Some` if this class is known to be a [`SolidBase`], or `None` if it is not. /// Return `Some` if this class is known to be a [`DisjointBase`], or `None` if it is not.
pub(super) fn as_solid_base(self, db: &'db dyn Db) -> Option<SolidBase<'db>> { pub(super) fn as_disjoint_base(self, db: &'db dyn Db) -> Option<DisjointBase<'db>> {
self.class_literal(db).0.as_solid_base(db) self.class_literal(db).0.as_disjoint_base(db)
} }
/// Return `true` if this class represents `known_class` /// Return `true` if this class represents `known_class`
@ -633,13 +633,13 @@ impl<'db> ClassType<'db> {
.apply_optional_specialization(db, specialization) .apply_optional_specialization(db, specialization)
} }
/// Return the [`SolidBase`] that appears first in the MRO of this class. /// Return the [`DisjointBase`] that appears first in the MRO of this class.
/// ///
/// Returns `None` if this class does not have any solid bases in its MRO. /// Returns `None` if this class does not have any disjoint bases in its MRO.
pub(super) fn nearest_solid_base(self, db: &'db dyn Db) -> Option<SolidBase<'db>> { pub(super) fn nearest_disjoint_base(self, db: &'db dyn Db) -> Option<DisjointBase<'db>> {
self.iter_mro(db) self.iter_mro(db)
.filter_map(ClassBase::into_class) .filter_map(ClassBase::into_class)
.find_map(|base| base.as_solid_base(db)) .find_map(|base| base.as_disjoint_base(db))
} }
/// Return `true` if this class could coexist in an MRO with `other`. /// Return `true` if this class could coexist in an MRO with `other`.
@ -660,12 +660,17 @@ impl<'db> ClassType<'db> {
return other.is_subclass_of(db, self); return other.is_subclass_of(db, self);
} }
// Two solid bases can only coexist in an MRO if one is a subclass of the other. // Two disjoint bases can only coexist in an MRO if one is a subclass of the other.
if self.nearest_solid_base(db).is_some_and(|solid_base_1| { if self
other.nearest_solid_base(db).is_some_and(|solid_base_2| { .nearest_disjoint_base(db)
!solid_base_1.could_coexist_in_mro_with(db, &solid_base_2) .is_some_and(|disjoint_base_1| {
other
.nearest_disjoint_base(db)
.is_some_and(|disjoint_base_2| {
!disjoint_base_1.could_coexist_in_mro_with(db, &disjoint_base_2)
}) })
}) { })
{
return false; return false;
} }
@ -1519,14 +1524,19 @@ impl<'db> ClassLiteral<'db> {
} }
} }
/// Return `Some()` if this class is known to be a [`SolidBase`], or `None` if it is not. /// Return `Some()` if this class is known to be a [`DisjointBase`], or `None` if it is not.
pub(super) fn as_solid_base(self, db: &'db dyn Db) -> Option<SolidBase<'db>> { pub(super) fn as_disjoint_base(self, db: &'db dyn Db) -> Option<DisjointBase<'db>> {
if let Some(known_class) = self.known(db) { // TODO: Typeshed cannot add `@disjoint_base` to its `tuple` definition without breaking pyright.
known_class // See <https://github.com/microsoft/pyright/issues/10836>.
.is_solid_base() // This should be fixed soon; we can remove this workaround then.
.then_some(SolidBase::hard_coded(self)) if self.is_known(db, KnownClass::Tuple)
|| self
.known_function_decorators(db)
.contains(&KnownFunction::DisjointBase)
{
Some(DisjointBase::due_to_decorator(self))
} else if SlotsKind::from(db, self) == SlotsKind::NotEmpty { } else if SlotsKind::from(db, self) == SlotsKind::NotEmpty {
Some(SolidBase::due_to_dunder_slots(self)) Some(DisjointBase::due_to_dunder_slots(self))
} else { } else {
None None
} }
@ -3375,39 +3385,47 @@ impl InheritanceCycle {
/// CPython internally considers a class a "solid base" if it has an atypical instance memory layout, /// CPython internally considers a class a "solid base" if it has an atypical instance memory layout,
/// with additional memory "slots" for each instance, besides the default object metadata and an /// with additional memory "slots" for each instance, besides the default object metadata and an
/// attribute dictionary. A "solid base" can be a class defined in a C extension which defines C-level /// attribute dictionary. Per [PEP 800], however, we use the term "disjoint base" for this concept.
/// instance slots, or a Python class that defines non-empty `__slots__`.
/// ///
/// Two solid bases can only coexist in a class's MRO if one is a subclass of the other. Knowing if /// A "disjoint base" can be a class defined in a C extension which defines C-level instance slots,
/// a class is "solid base" or not is therefore valuable for inferring whether two instance types or /// or a Python class that defines non-empty `__slots__`. C-level instance slots are not generally
/// visible to Python code, but PEP 800 specifies that any class decorated with
/// `@typing_extensions.disjoint_base` should be treated by type checkers as a disjoint base; it is
/// assumed that classes with C-level instance slots will be decorated as such when they appear in
/// stub files.
///
/// Two disjoint bases can only coexist in a class's MRO if one is a subclass of the other. Knowing if
/// a class is "disjoint base" or not is therefore valuable for inferring whether two instance types or
/// two subclass-of types are disjoint from each other. It also allows us to detect possible /// two subclass-of types are disjoint from each other. It also allows us to detect possible
/// `TypeError`s resulting from class definitions. /// `TypeError`s resulting from class definitions.
///
/// [PEP 800]: https://peps.python.org/pep-0800/
#[derive(Debug, PartialEq, Eq, Hash, Copy, Clone)] #[derive(Debug, PartialEq, Eq, Hash, Copy, Clone)]
pub(super) struct SolidBase<'db> { pub(super) struct DisjointBase<'db> {
pub(super) class: ClassLiteral<'db>, pub(super) class: ClassLiteral<'db>,
pub(super) kind: SolidBaseKind, pub(super) kind: DisjointBaseKind,
} }
impl<'db> SolidBase<'db> { impl<'db> DisjointBase<'db> {
/// Creates a [`SolidBase`] instance where we know the class is a solid base /// Creates a [`DisjointBase`] instance where we know the class is a disjoint base
/// because it is special-cased by ty. /// because it has the `@disjoint_base` decorator on its definition
fn hard_coded(class: ClassLiteral<'db>) -> Self { fn due_to_decorator(class: ClassLiteral<'db>) -> Self {
Self { Self {
class, class,
kind: SolidBaseKind::HardCoded, kind: DisjointBaseKind::DisjointBaseDecorator,
} }
} }
/// Creates a [`SolidBase`] instance where we know the class is a solid base /// Creates a [`DisjointBase`] instance where we know the class is a disjoint base
/// because of its `__slots__` definition. /// because of its `__slots__` definition.
fn due_to_dunder_slots(class: ClassLiteral<'db>) -> Self { fn due_to_dunder_slots(class: ClassLiteral<'db>) -> Self {
Self { Self {
class, class,
kind: SolidBaseKind::DefinesSlots, kind: DisjointBaseKind::DefinesSlots,
} }
} }
/// Two solid bases can only coexist in a class's MRO if one is a subclass of the other /// Two disjoint bases can only coexist in a class's MRO if one is a subclass of the other
fn could_coexist_in_mro_with(&self, db: &'db dyn Db, other: &Self) -> bool { fn could_coexist_in_mro_with(&self, db: &'db dyn Db, other: &Self) -> bool {
self == other self == other
|| self || self
@ -3420,10 +3438,11 @@ impl<'db> SolidBase<'db> {
} }
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)] #[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub(super) enum SolidBaseKind { pub(super) enum DisjointBaseKind {
/// We know the class is a solid base because of some hardcoded knowledge in ty. /// We know the class is a disjoint base because it's either hardcoded in ty
HardCoded, /// or has the `@disjoint_base` decorator.
/// We know the class is a solid base because it has a non-empty `__slots__` definition. DisjointBaseDecorator,
/// We know the class is a disjoint base because it has a non-empty `__slots__` definition.
DefinesSlots, DefinesSlots,
} }
@ -3624,94 +3643,6 @@ impl KnownClass {
} }
} }
/// Return `true` if this class is a [`SolidBase`]
const fn is_solid_base(self) -> bool {
match self {
Self::Object => false,
// Most non-`@final` builtins (other than `object`) are solid bases.
Self::Set
| Self::FrozenSet
| Self::BaseException
| Self::Bytearray
| Self::Int
| Self::Float
| Self::Complex
| Self::Str
| Self::List
| Self::Tuple
| Self::Dict
| Self::Slice
| Self::Property
| Self::Staticmethod
| Self::Classmethod
| Self::Deprecated
| Self::Type
| Self::ModuleType
| Self::Super
| Self::GenericAlias
| Self::Deque
| Self::Bytes => true,
// It doesn't really make sense to ask the question for `@final` types,
// since these are "more than solid bases". But we'll anyway infer a `@final`
// class as being disjoint from a class that doesn't appear in its MRO,
// and we'll anyway complain if we see a class definition that includes a
// `@final` class in its bases. We therefore return `false` here to avoid
// unnecessary duplicate diagnostics elsewhere.
Self::TypeVarTuple
| Self::TypeAliasType
| Self::UnionType
| Self::NoDefaultType
| Self::MethodType
| Self::MethodWrapperType
| Self::FunctionType
| Self::GeneratorType
| Self::AsyncGeneratorType
| Self::StdlibAlias
| Self::SpecialForm
| Self::TypeVar
| Self::ParamSpec
| Self::ParamSpecArgs
| Self::ParamSpecKwargs
| Self::WrapperDescriptorType
| Self::EllipsisType
| Self::NotImplementedType
| Self::KwOnly
| Self::InitVar
| Self::VersionInfo
| Self::Bool
| Self::NoneType
| Self::CoroutineType => false,
// Anything with a *runtime* MRO (N.B. sometimes different from the MRO that typeshed gives!)
// with length >2, or anything that is implemented in pure Python, is not a solid base.
Self::ABCMeta
| Self::Awaitable
| Self::Generator
| Self::Enum
| Self::EnumType
| Self::Auto
| Self::Member
| Self::Nonmember
| Self::ChainMap
| Self::Exception
| Self::ExceptionGroup
| Self::Field
| Self::SupportsIndex
| Self::NamedTupleFallback
| Self::NamedTupleLike
| Self::TypedDictFallback
| Self::Counter
| Self::DefaultDict
| Self::OrderedDict
| Self::NewType
| Self::Iterable
| Self::Iterator
| Self::BaseExceptionGroup => false,
}
}
/// Return `true` if this class is a subclass of `enum.Enum` *and* has enum members, i.e. /// Return `true` if this class is a subclass of `enum.Enum` *and* has enum members, i.e.
/// if it is an "actual" enum, not `enum.Enum` itself or a similar custom enum class. /// if it is an "actual" enum, not `enum.Enum` itself or a similar custom enum class.
pub(crate) const fn is_enum_subclass_with_members(self) -> bool { pub(crate) const fn is_enum_subclass_with_members(self) -> bool {

78
crates/ty_python_semantic/src/types/diagnostic.rs
View file

@ -10,7 +10,7 @@ use crate::semantic_index::SemanticIndex;
use crate::semantic_index::definition::Definition; use crate::semantic_index::definition::Definition;
use crate::semantic_index::place::{PlaceTable, ScopedPlaceId}; use crate::semantic_index::place::{PlaceTable, ScopedPlaceId};
use crate::suppression::FileSuppressionId; use crate::suppression::FileSuppressionId;
use crate::types::class::{Field, SolidBase, SolidBaseKind}; use crate::types::class::{DisjointBase, DisjointBaseKind, Field};
use crate::types::function::KnownFunction; use crate::types::function::KnownFunction;
use crate::types::string_annotation::{ use crate::types::string_annotation::{
BYTE_STRING_TYPE_ANNOTATION, ESCAPE_CHARACTER_IN_FORWARD_ANNOTATION, FSTRING_TYPE_ANNOTATION, BYTE_STRING_TYPE_ANNOTATION, ESCAPE_CHARACTER_IN_FORWARD_ANNOTATION, FSTRING_TYPE_ANNOTATION,
@ -405,7 +405,7 @@ declare_lint! {
/// ///
/// ## Known problems /// ## Known problems
/// Classes that have "dynamic" definitions of `__slots__` (definitions do not consist /// Classes that have "dynamic" definitions of `__slots__` (definitions do not consist
/// of string literals, or tuples of string literals) are not currently considered solid /// of string literals, or tuples of string literals) are not currently considered disjoint
/// bases by ty. /// bases by ty.
/// ///
/// Additionally, this check is not exhaustive: many C extensions (including several in /// Additionally, this check is not exhaustive: many C extensions (including several in
@ -2170,9 +2170,9 @@ pub(crate) fn report_instance_layout_conflict(
context: &InferContext, context: &InferContext,
class: ClassLiteral, class: ClassLiteral,
node: &ast::StmtClassDef, node: &ast::StmtClassDef,
solid_bases: &IncompatibleBases, disjoint_bases: &IncompatibleBases,
) { ) {
debug_assert!(solid_bases.len() > 1); debug_assert!(disjoint_bases.len() > 1);
let db = context.db(); let db = context.db();
@ -2186,7 +2186,7 @@ pub(crate) fn report_instance_layout_conflict(
diagnostic.set_primary_message(format_args!( diagnostic.set_primary_message(format_args!(
"Bases {} cannot be combined in multiple inheritance", "Bases {} cannot be combined in multiple inheritance",
solid_bases.describe_problematic_class_bases(db) disjoint_bases.describe_problematic_class_bases(db)
)); ));
let mut subdiagnostic = SubDiagnostic::new( let mut subdiagnostic = SubDiagnostic::new(
@ -2195,23 +2195,23 @@ pub(crate) fn report_instance_layout_conflict(
have incompatible memory layouts", have incompatible memory layouts",
); );
for (solid_base, solid_base_info) in solid_bases { for (disjoint_base, disjoint_base_info) in disjoint_bases {
let IncompatibleBaseInfo { let IncompatibleBaseInfo {
node_index, node_index,
originating_base, originating_base,
} = solid_base_info; } = disjoint_base_info;
let span = context.span(&node.bases()[*node_index]); let span = context.span(&node.bases()[*node_index]);
let mut annotation = Annotation::secondary(span.clone()); let mut annotation = Annotation::secondary(span.clone());
if solid_base.class == *originating_base { if disjoint_base.class == *originating_base {
match solid_base.kind { match disjoint_base.kind {
SolidBaseKind::DefinesSlots => { DisjointBaseKind::DefinesSlots => {
annotation = annotation.message(format_args!( annotation = annotation.message(format_args!(
"`{base}` instances have a distinct memory layout because `{base}` defines non-empty `__slots__`", "`{base}` instances have a distinct memory layout because `{base}` defines non-empty `__slots__`",
base = originating_base.name(db) base = originating_base.name(db)
)); ));
} }
SolidBaseKind::HardCoded => { DisjointBaseKind::DisjointBaseDecorator => {
annotation = annotation.message(format_args!( annotation = annotation.message(format_args!(
"`{base}` instances have a distinct memory layout because of the way `{base}` \ "`{base}` instances have a distinct memory layout because of the way `{base}` \
is implemented in a C extension", is implemented in a C extension",
@ -2223,26 +2223,28 @@ pub(crate) fn report_instance_layout_conflict(
} else { } else {
annotation = annotation.message(format_args!( annotation = annotation.message(format_args!(
"`{base}` instances have a distinct memory layout \ "`{base}` instances have a distinct memory layout \
because `{base}` inherits from `{solid_base}`", because `{base}` inherits from `{disjoint_base}`",
base = originating_base.name(db), base = originating_base.name(db),
solid_base = solid_base.class.name(db) disjoint_base = disjoint_base.class.name(db)
)); ));
subdiagnostic.annotate(annotation); subdiagnostic.annotate(annotation);
let mut additional_annotation = Annotation::secondary(span); let mut additional_annotation = Annotation::secondary(span);
additional_annotation = match solid_base.kind { additional_annotation = match disjoint_base.kind {
SolidBaseKind::DefinesSlots => additional_annotation.message(format_args!( DisjointBaseKind::DefinesSlots => additional_annotation.message(format_args!(
"`{solid_base}` instances have a distinct memory layout because `{solid_base}` \ "`{disjoint_base}` instances have a distinct memory layout because `{disjoint_base}` \
defines non-empty `__slots__`", defines non-empty `__slots__`",
solid_base = solid_base.class.name(db), disjoint_base = disjoint_base.class.name(db),
)), )),
SolidBaseKind::HardCoded => additional_annotation.message(format_args!( DisjointBaseKind::DisjointBaseDecorator => {
"`{solid_base}` instances have a distinct memory layout \ additional_annotation.message(format_args!(
because of the way `{solid_base}` is implemented in a C extension", "`{disjoint_base}` instances have a distinct memory layout \
solid_base = solid_base.class.name(db), because of the way `{disjoint_base}` is implemented in a C extension",
)), disjoint_base = disjoint_base.class.name(db),
))
}
}; };
subdiagnostic.annotate(additional_annotation); subdiagnostic.annotate(additional_annotation);
@ -2252,20 +2254,20 @@ pub(crate) fn report_instance_layout_conflict(
diagnostic.sub(subdiagnostic); diagnostic.sub(subdiagnostic);
} }
/// Information regarding the conflicting solid bases a class is inferred to have in its MRO. /// Information regarding the conflicting disjoint bases a class is inferred to have in its MRO.
/// ///
/// For each solid base, we record information about which element in the class's bases list /// For each disjoint base, we record information about which element in the class's bases list
/// caused the solid base to be included in the class's MRO. /// caused the disjoint base to be included in the class's MRO.
/// ///
/// The inner data is an `IndexMap` to ensure that diagnostics regarding conflicting solid bases /// The inner data is an `IndexMap` to ensure that diagnostics regarding conflicting disjoint bases
/// are reported in a stable order. /// are reported in a stable order.
#[derive(Debug, Default)] #[derive(Debug, Default)]
pub(super) struct IncompatibleBases<'db>(FxIndexMap<SolidBase<'db>, IncompatibleBaseInfo<'db>>); pub(super) struct IncompatibleBases<'db>(FxIndexMap<DisjointBase<'db>, IncompatibleBaseInfo<'db>>);
impl<'db> IncompatibleBases<'db> { impl<'db> IncompatibleBases<'db> {
pub(super) fn insert( pub(super) fn insert(
&mut self, &mut self,
base: SolidBase<'db>, base: DisjointBase<'db>,
node_index: usize, node_index: usize,
class: ClassLiteral<'db>, class: ClassLiteral<'db>,
) { ) {
@ -2287,19 +2289,19 @@ impl<'db> IncompatibleBases<'db> {
self.0.len() self.0.len()
} }
/// Two solid bases are allowed to coexist in an MRO if one is a subclass of the other. /// Two disjoint bases are allowed to coexist in an MRO if one is a subclass of the other.
/// This method therefore removes any entry in `self` that is a subclass of one or more /// This method therefore removes any entry in `self` that is a subclass of one or more
/// other entries also contained in `self`. /// other entries also contained in `self`.
pub(super) fn remove_redundant_entries(&mut self, db: &'db dyn Db) { pub(super) fn remove_redundant_entries(&mut self, db: &'db dyn Db) {
self.0 = self self.0 = self
.0 .0
.iter() .iter()
.filter(|(solid_base, _)| { .filter(|(disjoint_base, _)| {
self.0 self.0
.keys() .keys()
.filter(|other_base| other_base != solid_base) .filter(|other_base| other_base != disjoint_base)
.all(|other_base| { .all(|other_base| {
!solid_base.class.is_subclass_of( !disjoint_base.class.is_subclass_of(
db, db,
None, None,
other_base.class.default_specialization(db), other_base.class.default_specialization(db),
@ -2312,25 +2314,25 @@ impl<'db> IncompatibleBases<'db> {
} }
impl<'a, 'db> IntoIterator for &'a IncompatibleBases<'db> { impl<'a, 'db> IntoIterator for &'a IncompatibleBases<'db> {
type Item = (&'a SolidBase<'db>, &'a IncompatibleBaseInfo<'db>); type Item = (&'a DisjointBase<'db>, &'a IncompatibleBaseInfo<'db>);
type IntoIter = indexmap::map::Iter<'a, SolidBase<'db>, IncompatibleBaseInfo<'db>>; type IntoIter = indexmap::map::Iter<'a, DisjointBase<'db>, IncompatibleBaseInfo<'db>>;
fn into_iter(self) -> Self::IntoIter { fn into_iter(self) -> Self::IntoIter {
self.0.iter() self.0.iter()
} }
} }
/// Information about which class base the "solid base" stems from /// Information about which class base the "disjoint base" stems from
#[derive(Debug, Copy, Clone)] #[derive(Debug, Copy, Clone)]
pub(super) struct IncompatibleBaseInfo<'db> { pub(super) struct IncompatibleBaseInfo<'db> {
/// The index of the problematic base in the [`ast::StmtClassDef`]'s bases list. /// The index of the problematic base in the [`ast::StmtClassDef`]'s bases list.
node_index: usize, node_index: usize,
/// The base class in the [`ast::StmtClassDef`]'s bases list that caused /// The base class in the [`ast::StmtClassDef`]'s bases list that caused
/// the solid base to be included in the class's MRO. /// the disjoint base to be included in the class's MRO.
/// ///
/// This won't necessarily be the same class as the `SolidBase`'s class, /// This won't necessarily be the same class as the `DisjointBase`'s class,
/// as the `SolidBase` may have found its way into the class's MRO by dint of it being a /// as the `DisjointBase` may have found its way into the class's MRO by dint of it being a
/// superclass of one of the classes in the class definition's bases list. /// superclass of one of the classes in the class definition's bases list.
originating_base: ClassLiteral<'db>, originating_base: ClassLiteral<'db>,
} }

5
crates/ty_python_semantic/src/types/function.rs
View file

@ -1109,7 +1109,8 @@ pub enum KnownFunction {
/// `typing(_extensions).final` /// `typing(_extensions).final`
Final, Final,
/// `typing(_extensions).disjoint_base`
DisjointBase,
/// [`typing(_extensions).no_type_check`](https://typing.python.org/en/latest/spec/directives.html#no-type-check) /// [`typing(_extensions).no_type_check`](https://typing.python.org/en/latest/spec/directives.html#no-type-check)
NoTypeCheck, NoTypeCheck,
@ -1212,6 +1213,7 @@ impl KnownFunction {
| Self::GetProtocolMembers | Self::GetProtocolMembers
| Self::RuntimeCheckable | Self::RuntimeCheckable
| Self::DataclassTransform | Self::DataclassTransform
| Self::DisjointBase
| Self::NoTypeCheck => { | Self::NoTypeCheck => {
matches!(module, KnownModule::Typing | KnownModule::TypingExtensions) matches!(module, KnownModule::Typing | KnownModule::TypingExtensions)
} }
@ -1574,6 +1576,7 @@ pub(crate) mod tests {
| KnownFunction::GetProtocolMembers | KnownFunction::GetProtocolMembers
| KnownFunction::RuntimeCheckable | KnownFunction::RuntimeCheckable
| KnownFunction::DataclassTransform | KnownFunction::DataclassTransform
| KnownFunction::DisjointBase
| KnownFunction::NoTypeCheck => KnownModule::TypingExtensions, | KnownFunction::NoTypeCheck => KnownModule::TypingExtensions,
KnownFunction::IsSingleton KnownFunction::IsSingleton

12
crates/ty_python_semantic/src/types/infer.rs
View file

@ -1147,7 +1147,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
let is_protocol = class.is_protocol(self.db()); let is_protocol = class.is_protocol(self.db());
let mut solid_bases = IncompatibleBases::default(); let mut disjoint_bases = IncompatibleBases::default();
// (3) Iterate through the class's explicit bases to check for various possible errors: // (3) Iterate through the class's explicit bases to check for various possible errors:
// - Check for inheritance from plain `Generic`, // - Check for inheritance from plain `Generic`,
@ -1209,8 +1209,8 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
_ => continue, _ => continue,
}; };
if let Some(solid_base) = base_class.nearest_solid_base(self.db()) { if let Some(disjoint_base) = base_class.nearest_disjoint_base(self.db()) {
solid_bases.insert(solid_base, i, base_class.class_literal(self.db()).0); disjoint_bases.insert(disjoint_base, i, base_class.class_literal(self.db()).0);
} }
if is_protocol if is_protocol
@ -1301,14 +1301,14 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
} }
}, },
Ok(_) => { Ok(_) => {
solid_bases.remove_redundant_entries(self.db()); disjoint_bases.remove_redundant_entries(self.db());
if solid_bases.len() > 1 { if disjoint_bases.len() > 1 {
report_instance_layout_conflict( report_instance_layout_conflict(
&self.context, &self.context,
class, class,
class_node, class_node,
&solid_bases, &disjoint_bases,
); );
} }
} }

2
ty.schema.json generated
View file

@ -433,7 +433,7 @@
}, },
"instance-layout-conflict": { "instance-layout-conflict": {
"title": "detects class definitions that raise `TypeError` due to instance layout conflict", "title": "detects class definitions that raise `TypeError` due to instance layout conflict",
"description": "## What it does\nChecks for classes definitions which will fail at runtime due to\n\"instance memory layout conflicts\".\n\nThis error is usually caused by attempting to combine multiple classes\nthat define non-empty `__slots__` in a class's [Method Resolution Order]\n(MRO), or by attempting to combine multiple builtin classes in a class's\nMRO.\n\n## Why is this bad?\nInheriting from bases with conflicting instance memory layouts\nwill lead to a `TypeError` at runtime.\n\nAn instance memory layout conflict occurs when CPython cannot determine\nthe memory layout instances of a class should have, because the instance\nmemory layout of one of its bases conflicts with the instance memory layout\nof one or more of its other bases.\n\nFor example, if a Python class defines non-empty `__slots__`, this will\nimpact the memory layout of instances of that class. Multiple inheritance\nfrom more than one different class defining non-empty `__slots__` is not\nallowed:\n\n```python\nclass A:\n __slots__ = (\"a\", \"b\")\n\nclass B:\n __slots__ = (\"a\", \"b\") # Even if the values are the same\n\n# TypeError: multiple bases have instance lay-out conflict\nclass C(A, B): ...\n```\n\nAn instance layout conflict can also be caused by attempting to use\nmultiple inheritance with two builtin classes, due to the way that these\nclasses are implemented in a CPython C extension:\n\n```python\nclass A(int, float): ... # TypeError: multiple bases have instance lay-out conflict\n```\n\nNote that pure-Python classes with no `__slots__`, or pure-Python classes\nwith empty `__slots__`, are always compatible:\n\n```python\nclass A: ...\nclass B:\n __slots__ = ()\nclass C:\n __slots__ = (\"a\", \"b\")\n\n# fine\nclass D(A, B, C): ...\n```\n\n## Known problems\nClasses that have \"dynamic\" definitions of `__slots__` (definitions do not consist\nof string literals, or tuples of string literals) are not currently considered solid\nbases by ty.\n\nAdditionally, this check is not exhaustive: many C extensions (including several in\nthe standard library) define classes that use extended memory layouts and thus cannot\ncoexist in a single MRO. Since it is currently not possible to represent this fact in\nstub files, having a full knowledge of these classes is also impossible. When it comes\nto classes that do not define `__slots__` at the Python level, therefore, ty, currently\nonly hard-codes a number of cases where it knows that a class will produce instances with\nan atypical memory layout.\n\n## Further reading\n- [CPython documentation: `__slots__`](https://docs.python.org/3/reference/datamodel.html#slots)\n- [CPython documentation: Method Resolution Order](https://docs.python.org/3/glossary.html#term-method-resolution-order)\n\n[Method Resolution Order]: https://docs.python.org/3/glossary.html#term-method-resolution-order", "description": "## What it does\nChecks for classes definitions which will fail at runtime due to\n\"instance memory layout conflicts\".\n\nThis error is usually caused by attempting to combine multiple classes\nthat define non-empty `__slots__` in a class's [Method Resolution Order]\n(MRO), or by attempting to combine multiple builtin classes in a class's\nMRO.\n\n## Why is this bad?\nInheriting from bases with conflicting instance memory layouts\nwill lead to a `TypeError` at runtime.\n\nAn instance memory layout conflict occurs when CPython cannot determine\nthe memory layout instances of a class should have, because the instance\nmemory layout of one of its bases conflicts with the instance memory layout\nof one or more of its other bases.\n\nFor example, if a Python class defines non-empty `__slots__`, this will\nimpact the memory layout of instances of that class. Multiple inheritance\nfrom more than one different class defining non-empty `__slots__` is not\nallowed:\n\n```python\nclass A:\n __slots__ = (\"a\", \"b\")\n\nclass B:\n __slots__ = (\"a\", \"b\") # Even if the values are the same\n\n# TypeError: multiple bases have instance lay-out conflict\nclass C(A, B): ...\n```\n\nAn instance layout conflict can also be caused by attempting to use\nmultiple inheritance with two builtin classes, due to the way that these\nclasses are implemented in a CPython C extension:\n\n```python\nclass A(int, float): ... # TypeError: multiple bases have instance lay-out conflict\n```\n\nNote that pure-Python classes with no `__slots__`, or pure-Python classes\nwith empty `__slots__`, are always compatible:\n\n```python\nclass A: ...\nclass B:\n __slots__ = ()\nclass C:\n __slots__ = (\"a\", \"b\")\n\n# fine\nclass D(A, B, C): ...\n```\n\n## Known problems\nClasses that have \"dynamic\" definitions of `__slots__` (definitions do not consist\nof string literals, or tuples of string literals) are not currently considered disjoint\nbases by ty.\n\nAdditionally, this check is not exhaustive: many C extensions (including several in\nthe standard library) define classes that use extended memory layouts and thus cannot\ncoexist in a single MRO. Since it is currently not possible to represent this fact in\nstub files, having a full knowledge of these classes is also impossible. When it comes\nto classes that do not define `__slots__` at the Python level, therefore, ty, currently\nonly hard-codes a number of cases where it knows that a class will produce instances with\nan atypical memory layout.\n\n## Further reading\n- [CPython documentation: `__slots__`](https://docs.python.org/3/reference/datamodel.html#slots)\n- [CPython documentation: Method Resolution Order](https://docs.python.org/3/glossary.html#term-method-resolution-order)\n\n[Method Resolution Order]: https://docs.python.org/3/glossary.html#term-method-resolution-order",
"default": "error", "default": "error",
"oneOf": [ "oneOf": [
{ {