language-servers/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-09-29 13:25:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

[red-knot] Understand Annotated (#14950)

Browse source 
## Summary

Resolves #14922.

## Test Plan

Markdown tests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
This commit is contained in:
InSync 2024-12-14 00:41:37 +07:00 • committed by GitHub
parent 3533d7f5b4
commit aa1938f6ba
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 199 additions and 90 deletions
Download patch file Download diff file Expand all files Collapse all files

80
crates/red_knot_python_semantic/resources/mdtest/annotations/annotated.md Normal file
View file

@ -0,0 +1,80 @@
# `Annotated`
`Annotated` attaches arbitrary metadata to a given type.
## Usages
`Annotated[T, ...]` is equivalent to `T`: All metadata arguments are simply ignored.
```py
from typing_extensions import Annotated
def _(x: Annotated[int, "foo"]):
reveal_type(x) # revealed: int
def _(x: Annotated[int, lambda: 0 + 1 * 2 // 3, _(4)]):
reveal_type(x) # revealed: int
def _(x: Annotated[int, "arbitrary", "metadata", "elements", "are", "fine"]):
reveal_type(x) # revealed: int
def _(x: Annotated[tuple[str, int], bytes]):
reveal_type(x) # revealed: tuple[str, int]
```
## Parameterization
It is invalid to parameterize `Annotated` with less than two arguments.
```py
from typing_extensions import Annotated
# TODO: This should be an error
def _(x: Annotated):
reveal_type(x) # revealed: Unknown
# error: [invalid-type-form]
def _(x: Annotated[()]):
reveal_type(x) # revealed: Unknown
# error: [invalid-type-form]
def _(x: Annotated[int]):
# `Annotated[T]` is invalid and will raise an error at runtime,
# but we treat it the same as `T` to provide better diagnostics later on.
# The subscription itself is still reported, regardless.
# Same for the `(int,)` form below.
reveal_type(x) # revealed: int
# error: [invalid-type-form]
def _(x: Annotated[(int,)]):
reveal_type(x) # revealed: int
```
## Inheritance
### Correctly parameterized
Inheriting from `Annotated[T, ...]` is equivalent to inheriting from `T` itself.
```py
from typing_extensions import Annotated
# TODO: False positive
# error: [invalid-base]
class C(Annotated[int, "foo"]): ...
# TODO: Should be `tuple[Literal[C], Literal[int], Literal[object]]`
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Unknown, Literal[object]]
```
### Not parameterized
```py
from typing_extensions import Annotated
# At runtime, this is an error.
# error: [invalid-base]
class C(Annotated): ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Unknown, Literal[object]]
```

2
crates/red_knot_python_semantic/resources/mdtest/annotations/any.md
View file

@ -77,7 +77,7 @@ def _(s: Subclass):
```py
from typing import Any
# error: [invalid-type-parameter] "Type `typing.Any` expected no type parameter"
# error: [invalid-type-form] "Type `typing.Any` expected no type parameter"
def f(x: Any[int]):
reveal_type(x) # revealed: Unknown
```

12
crates/red_knot_python_semantic/resources/mdtest/annotations/literal_string.md
View file

@ -27,19 +27,19 @@ def f():
```py
from typing_extensions import Literal, LiteralString
bad_union: Literal["hello", LiteralString] # error: [invalid-literal-parameter]
bad_nesting: Literal[LiteralString] # error: [invalid-literal-parameter]
bad_union: Literal["hello", LiteralString] # error: [invalid-type-form]
bad_nesting: Literal[LiteralString] # error: [invalid-type-form]
```
### Parametrized
### Parameterized
`LiteralString` cannot be parametrized.
`LiteralString` cannot be parameterized.
```py
from typing_extensions import LiteralString
a: LiteralString[str] # error: [invalid-type-parameter]
b: LiteralString["foo"] # error: [invalid-type-parameter]
a: LiteralString[str] # error: [invalid-type-form]
b: LiteralString["foo"] # error: [invalid-type-form]
```
### As a base class

2
crates/red_knot_python_semantic/resources/mdtest/annotations/never.md
View file

@ -21,7 +21,7 @@ reveal_type(stop())
```py
from typing_extensions import NoReturn, Never, Any
# error: [invalid-type-parameter] "Type `typing.Never` expected no type parameter"
# error: [invalid-type-form] "Type `typing.Never` expected no type parameter"
x: Never[int]
a1: NoReturn
a2: Never

6
crates/red_knot_python_semantic/resources/mdtest/annotations/unsupported_special_forms.md
View file

@ -61,11 +61,11 @@ Some of these are not subscriptable:
```py
from typing_extensions import Self, TypeAlias
X: TypeAlias[T] = int # error: [invalid-type-parameter]
X: TypeAlias[T] = int # error: [invalid-type-form]
class Foo[T]:
# error: [invalid-type-parameter] "Special form `typing.Self` expected no type parameter"
# error: [invalid-type-parameter] "Special form `typing.Self` expected no type parameter"
# error: [invalid-type-form] "Special form `typing.Self` expected no type parameter"
# error: [invalid-type-form] "Special form `typing.Self` expected no type parameter"
def method(self: Self[int]) -> Self[int]:
reveal_type(self) # revealed: Unknown
```

12
crates/red_knot_python_semantic/resources/mdtest/literal/literal.md
View file

@ -45,19 +45,19 @@ def f():
# TODO: This should be Color.RED
reveal_type(b1) # revealed: Literal[0]
# error: [invalid-literal-parameter]
# error: [invalid-type-form]
invalid1: Literal[3 + 4]
# error: [invalid-literal-parameter]
# error: [invalid-type-form]
invalid2: Literal[4 + 3j]
# error: [invalid-literal-parameter]
# error: [invalid-type-form]
invalid3: Literal[(3, 4)]
hello = "hello"
invalid4: Literal[
1 + 2, # error: [invalid-literal-parameter]
1 + 2, # error: [invalid-type-form]
"foo",
hello, # error: [invalid-literal-parameter]
(1, 2, 3), # error: [invalid-literal-parameter]
hello, # error: [invalid-type-form]
(1, 2, 3), # error: [invalid-type-form]
]
```

11
crates/red_knot_python_semantic/src/types.rs
View file

@ -1794,6 +1794,8 @@ impl<'db> Type<'db> {
}
Type::KnownInstance(KnownInstanceType::LiteralString) => Type::LiteralString,
Type::KnownInstance(KnownInstanceType::Any) => Type::Any,
// TODO: Should emit a diagnostic
Type::KnownInstance(KnownInstanceType::Annotated) => Type::Unknown,
Type::Todo(_) => *self,
_ => todo_type!("Unsupported or invalid type in a type expression"),
}
@ -2163,6 +2165,8 @@ impl<'db> KnownClass {
/// Enumeration of specific runtime that are special enough to be considered their own type.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, salsa::Update)]
pub enum KnownInstanceType<'db> {
/// The symbol `typing.Annotated` (which can also be found as `typing_extensions.Annotated`)
Annotated,
/// The symbol `typing.Literal` (which can also be found as `typing_extensions.Literal`)
Literal,
/// The symbol `typing.LiteralString` (which can also be found as `typing_extensions.LiteralString`)
@ -2215,6 +2219,7 @@ pub enum KnownInstanceType<'db> {
impl<'db> KnownInstanceType<'db> {
pub const fn as_str(self) -> &'static str {
match self {
Self::Annotated => "Annotated",
Self::Literal => "Literal",
Self::LiteralString => "LiteralString",
Self::Optional => "Optional",
@ -2253,7 +2258,8 @@ impl<'db> KnownInstanceType<'db> {
/// Evaluate the known instance in boolean context
pub const fn bool(self) -> Truthiness {
match self {
Self::Literal
Self::Annotated
| Self::Literal
| Self::LiteralString
| Self::Optional
| Self::TypeVar(_)
@ -2291,6 +2297,7 @@ impl<'db> KnownInstanceType<'db> {
/// Return the repr of the symbol at runtime
pub fn repr(self, db: &'db dyn Db) -> &'db str {
match self {
Self::Annotated => "typing.Annotated",
Self::Literal => "typing.Literal",
Self::LiteralString => "typing.LiteralString",
Self::Optional => "typing.Optional",
@ -2329,6 +2336,7 @@ impl<'db> KnownInstanceType<'db> {
/// Return the [`KnownClass`] which this symbol is an instance of
pub const fn class(self) -> KnownClass {
match self {
Self::Annotated => KnownClass::SpecialForm,
Self::Literal => KnownClass::SpecialForm,
Self::LiteralString => KnownClass::SpecialForm,
Self::Optional => KnownClass::SpecialForm,
@ -2395,6 +2403,7 @@ impl<'db> KnownInstanceType<'db> {
("typing", "Tuple") => Some(Self::Tuple),
("typing", "Type") => Some(Self::Type),
("typing", "Callable") => Some(Self::Callable),
("typing" | "typing_extensions", "Annotated") => Some(Self::Annotated),
("typing" | "typing_extensions", "Literal") => Some(Self::Literal),
("typing" | "typing_extensions", "LiteralString") => Some(Self::LiteralString),
("typing" | "typing_extensions", "Never") => Some(Self::Never),

1
crates/red_knot_python_semantic/src/types/class_base.rs
View file

@ -74,6 +74,7 @@ impl<'db> ClassBase<'db> {
Type::KnownInstance(known_instance) => match known_instance {
KnownInstanceType::TypeVar(_)
| KnownInstanceType::TypeAliasType(_)
| KnownInstanceType::Annotated
| KnownInstanceType::Literal
| KnownInstanceType::LiteralString
| KnownInstanceType::Union

24
crates/red_knot_python_semantic/src/types/diagnostic.rs
View file

@ -30,13 +30,11 @@ pub(crate) fn register_lints(registry: &mut LintRegistryBuilder) {
registry.register_lint(&CONFLICTING_DECLARATIONS);
registry.register_lint(&DIVISION_BY_ZERO);
registry.register_lint(&CALL_NON_CALLABLE);
registry.register_lint(&INVALID_TYPE_PARAMETER);
registry.register_lint(&INVALID_TYPE_VARIABLE_CONSTRAINTS);
registry.register_lint(&CYCLIC_CLASS_DEFINITION);
registry.register_lint(&DUPLICATE_BASE);
registry.register_lint(&INVALID_BASE);
registry.register_lint(&INCONSISTENT_MRO);
registry.register_lint(&INVALID_LITERAL_PARAMETER);
registry.register_lint(&CALL_POSSIBLY_UNBOUND_METHOD);
registry.register_lint(&POSSIBLY_UNBOUND_ATTRIBUTE);
registry.register_lint(&UNRESOLVED_ATTRIBUTE);
@ -249,16 +247,6 @@ declare_lint! {
}
}
declare_lint! {
/// ## What it does
/// TODO #14889
pub(crate) static INVALID_TYPE_PARAMETER = {
summary: "detects invalid type parameters",
status: LintStatus::preview("1.0.0"),
default_level: Level::Error,
}
}
declare_lint! {
/// TODO #14889
pub(crate) static INVALID_TYPE_VARIABLE_CONSTRAINTS = {
@ -308,18 +296,6 @@ declare_lint! {
}
}
declare_lint! {
/// ## What it does
/// Checks for invalid parameters to `typing.Literal`.
///
/// TODO #14889
pub(crate) static INVALID_LITERAL_PARAMETER = {
summary: "detects invalid literal parameters",
status: LintStatus::preview("1.0.0"),
default_level: Level::Error,
}
}
declare_lint! {
/// ## What it does
/// Checks for calls to possibly unbound methods.

139
crates/red_knot_python_semantic/src/types/infer.rs
View file

@ -54,8 +54,7 @@ use crate::types::diagnostic::{
TypeCheckDiagnostics, TypeCheckDiagnosticsBuilder, CALL_NON_CALLABLE,
CALL_POSSIBLY_UNBOUND_METHOD, CONFLICTING_DECLARATIONS, CONFLICTING_METACLASS,
CYCLIC_CLASS_DEFINITION, DIVISION_BY_ZERO, DUPLICATE_BASE, INCONSISTENT_MRO, INVALID_BASE,
INVALID_CONTEXT_MANAGER, INVALID_DECLARATION, INVALID_LITERAL_PARAMETER,
INVALID_PARAMETER_DEFAULT, INVALID_TYPE_FORM, INVALID_TYPE_PARAMETER,
INVALID_CONTEXT_MANAGER, INVALID_DECLARATION, INVALID_PARAMETER_DEFAULT, INVALID_TYPE_FORM,
INVALID_TYPE_VARIABLE_CONSTRAINTS, POSSIBLY_UNBOUND_ATTRIBUTE, POSSIBLY_UNBOUND_IMPORT,
UNDEFINED_REVEAL, UNRESOLVED_ATTRIBUTE, UNRESOLVED_IMPORT, UNSUPPORTED_OPERATOR,
};
@ -4810,126 +4809,170 @@ impl<'db> TypeInferenceBuilder<'db> {
subscript: &ast::ExprSubscript,
known_instance: KnownInstanceType,
) -> Type<'db> {
let parameters = &*subscript.slice;
let arguments_slice = &*subscript.slice;
match known_instance {
KnownInstanceType::Literal => match self.infer_literal_parameter_type(parameters) {
Ok(ty) => ty,
Err(nodes) => {
for node in nodes {
self.diagnostics.add_lint(
&INVALID_LITERAL_PARAMETER,
node.into(),
format_args!(
"Type arguments for `Literal` must be `None`, \
a literal value (int, bool, str, or bytes), or an enum value"
),
);
}
Type::Unknown
KnownInstanceType::Annotated => {
let mut report_invalid_arguments = || {
self.diagnostics.add_lint(
&INVALID_TYPE_FORM,
subscript.into(),
format_args!(
"Special form `{}` expected at least 2 arguments (one type and at least one metadata element)",
known_instance.repr(self.db)
),
);
};
let ast::Expr::Tuple(ast::ExprTuple {
elts: arguments, ..
}) = arguments_slice
else {
report_invalid_arguments();
// `Annotated[]` with less than two arguments is an error at runtime.
// However, we still treat `Annotated[T]` as `T` here for the purpose of
// giving better diagnostics later on.
// Pyright also does this. Mypy doesn't; it falls back to `Any` instead.
return self.infer_type_expression(arguments_slice);
};
if arguments.len() < 2 {
report_invalid_arguments();
}
},
let [type_expr, metadata @ ..] = &arguments[..] else {
self.infer_type_expression(arguments_slice);
return Type::Unknown;
};
for element in metadata {
self.infer_expression(element);
}
let ty = self.infer_type_expression(type_expr);
self.store_expression_type(arguments_slice, ty);
ty
}
KnownInstanceType::Literal => {
match self.infer_literal_parameter_type(arguments_slice) {
Ok(ty) => ty,
Err(nodes) => {
for node in nodes {
self.diagnostics.add_lint(
&INVALID_TYPE_FORM,
node.into(),
format_args!(
"Type arguments for `Literal` must be `None`, \
a literal value (int, bool, str, or bytes), or an enum value"
),
);
}
Type::Unknown
}
}
}
KnownInstanceType::Optional => {
let param_type = self.infer_type_expression(parameters);
let param_type = self.infer_type_expression(arguments_slice);
UnionType::from_elements(self.db, [param_type, Type::none(self.db)])
}
KnownInstanceType::Union => match parameters {
KnownInstanceType::Union => match arguments_slice {
ast::Expr::Tuple(t) => {
let union_ty = UnionType::from_elements(
self.db,
t.iter().map(|elt| self.infer_type_expression(elt)),
);
self.store_expression_type(parameters, union_ty);
self.store_expression_type(arguments_slice, union_ty);
union_ty
}
_ => self.infer_type_expression(parameters),
_ => self.infer_type_expression(arguments_slice),
},
KnownInstanceType::TypeVar(_) => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("TypeVar annotations")
}
KnownInstanceType::TypeAliasType(_) => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Generic PEP-695 type alias")
}
KnownInstanceType::Callable => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Callable types")
}
KnownInstanceType::ChainMap => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.ChainMap alias")
}
KnownInstanceType::OrderedDict => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.OrderedDict alias")
}
KnownInstanceType::Dict => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.Dict alias")
}
KnownInstanceType::List => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.List alias")
}
KnownInstanceType::DefaultDict => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.DefaultDict[] alias")
}
KnownInstanceType::Counter => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.Counter[] alias")
}
KnownInstanceType::Set => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.Set alias")
}
KnownInstanceType::FrozenSet => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.FrozenSet alias")
}
KnownInstanceType::Deque => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("typing.Deque alias")
}
KnownInstanceType::ReadOnly => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Required[] type qualifier")
}
KnownInstanceType::NotRequired => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("NotRequired[] type qualifier")
}
KnownInstanceType::ClassVar => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("ClassVar[] type qualifier")
}
KnownInstanceType::Final => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Final[] type qualifier")
}
KnownInstanceType::Required => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Required[] type qualifier")
}
KnownInstanceType::TypeIs => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("TypeIs[] special form")
}
KnownInstanceType::TypeGuard => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("TypeGuard[] special form")
}
KnownInstanceType::Concatenate => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Concatenate[] special form")
}
KnownInstanceType::Unpack => {
self.infer_type_expression(parameters);
self.infer_type_expression(arguments_slice);
todo_type!("Unpack[] special form")
}
KnownInstanceType::NoReturn | KnownInstanceType::Never | KnownInstanceType::Any => {
self.diagnostics.add_lint(
&INVALID_TYPE_PARAMETER,
&INVALID_TYPE_FORM,
subscript.into(),
format_args!(
"Type `{}` expected no type parameter",
@ -4940,7 +4983,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
KnownInstanceType::TypingSelf | KnownInstanceType::TypeAlias => {
self.diagnostics.add_lint(
&INVALID_TYPE_PARAMETER,
&INVALID_TYPE_FORM,
subscript.into(),
format_args!(
"Special form `{}` expected no type parameter",
@ -4951,7 +4994,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
KnownInstanceType::LiteralString => {
self.diagnostics.add_lint(
&INVALID_TYPE_PARAMETER,
&INVALID_TYPE_FORM,
subscript.into(),
format_args!(
"Type `{}` expected no type parameter. Did you mean to use `Literal[...]` instead?",
@ -4960,8 +5003,8 @@ impl<'db> TypeInferenceBuilder<'db> {
);
Type::Unknown
}
KnownInstanceType::Type => self.infer_subclass_of_type_expression(parameters),
KnownInstanceType::Tuple => self.infer_tuple_type_expression(parameters),
KnownInstanceType::Type => self.infer_subclass_of_type_expression(arguments_slice),
KnownInstanceType::Tuple => self.infer_tuple_type_expression(arguments_slice),
}
}