[ty] validate constructor call of TypedDict (#19810)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run

## Summary
Implement validation for `TypedDict` constructor calls and dictionary
literal assignments, including support for `total=False` and proper
field management.
Also add support for `Required` and `NotRequired` type qualifiers in
`TypedDict` classes, along with proper inheritance behavior and the
`total=` parameter.
Support both constructor calls and dict literal syntax

part of https://github.com/astral-sh/ty/issues/154

### Basic Required Field Validation
```py
class Person(TypedDict):
    name: str
    age: int | None

# Error: Missing required field 'name' in TypedDict `Person` constructor
incomplete = Person(age=25)

# Error: Invalid argument to key "name" with declared type `str` on TypedDict `Person`
wrong_type = Person(name=123, age=25)

# Error: Invalid key access on TypedDict `Person`: Unknown key "extra"
extra_field = Person(name="Bob", age=25, extra=True)
```
<img width="773" height="191" alt="Screenshot 2025-08-07 at 17 59 22"
src="https://github.com/user-attachments/assets/79076d98-e85f-4495-93d6-a731aa72a5c9"
/>

### Support for `total=False`
```py
class OptionalPerson(TypedDict, total=False):
    name: str
    age: int | None

# All valid - all fields are optional with total=False
charlie = OptionalPerson()
david = OptionalPerson(name="David")
emily = OptionalPerson(age=30)
frank = OptionalPerson(name="Frank", age=25)

# But type validation and extra fields still apply
invalid_type = OptionalPerson(name=123)  # Error: Invalid argument type
invalid_extra = OptionalPerson(extra=True)  # Error: Invalid key access
```

### Dictionary Literal Validation
```py
# Type checking works for both constructors and dict literals
person: Person = {"name": "Alice", "age": 30}

reveal_type(person["name"])  # revealed: str
reveal_type(person["age"])   # revealed: int | None

# Error: Invalid key access on TypedDict `Person`: Unknown key "non_existing"
reveal_type(person["non_existing"])  # revealed: Unknown
```

### `Required`, `NotRequired`, `total`
```python
from typing import TypedDict
from typing_extensions import Required, NotRequired

class PartialUser(TypedDict, total=False):
    name: Required[str]      # Required despite total=False
    age: int                 # Optional due to total=False
    email: NotRequired[str]  # Explicitly optional (redundant)

class User(TypedDict):
    name: Required[str]      # Explicitly required (redundant)
    age: int                 # Required due to total=True
    bio: NotRequired[str]    # Optional despite total=True

# Valid constructions
partial = PartialUser(name="Alice")  # name required, age optional
full = User(name="Bob", age=25)      # name and age required, bio optional

# Inheritance maintains original field requirements
class Employee(PartialUser):
    department: str                  # Required (new field)
    # name: still Required (inherited)
    # age: still optional (inherited)

emp = Employee(name="Charlie", department="Engineering")  # 
Employee(department="Engineering")  # 
e: Employee = {"age": 1}  # 
```

<img width="898" height="683" alt="Screenshot 2025-08-11 at 22 02 57"
src="https://github.com/user-attachments/assets/4c1b18cd-cb2e-493a-a948-51589d121738"
/>

## Implementation
The implementation reuses existing validation logic done in
https://github.com/astral-sh/ruff/pull/19782

### ℹ️ Why I did NOT synthesize an `__init__` for `TypedDict`:

`TypedDict` inherits `dict.__init__(self, *args, **kwargs)` that accepts
all arguments.
The type resolution system finds this inherited signature **before**
looking for synthesized members.
So `own_synthesized_member()` is never called because a signature
already exists.

To force synthesis, you'd have to override Python’s inheritance
mechanism, which would break compatibility with the existing ecosystem.

This is why I went with ad-hoc validation. IMO it's the only viable
approach that respects Python’s
inheritance semantics while providing the required validation.

### Refacto of `Field`

**Before:**
```rust
struct Field<'db> {
    declared_ty: Type<'db>,
    default_ty: Option<Type<'db>>,     // NamedTuple and dataclass only
    init_only: bool,                   // dataclass only  
    init: bool,                        // dataclass only
    is_required: Option<bool>,         // TypedDict only
}
```

**After:**
```rust
struct Field<'db> {
    declared_ty: Type<'db>,
    kind: FieldKind<'db>,
}

enum FieldKind<'db> {
    NamedTuple { default_ty: Option<Type<'db>> },
    Dataclass { default_ty: Option<Type<'db>>, init_only: bool, init: bool },
    TypedDict { is_required: bool },
}
```

## Test Plan
Updated Markdown tests

---------

Co-authored-by: David Peter <mail@david-peter.de>
This commit is contained in:
Eric Jolibois 2025-08-25 14:45:52 +02:00 committed by GitHub
parent 376e3ff395
commit f9bbee33f6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
14 changed files with 1113 additions and 248 deletions

View file

@ -574,6 +574,16 @@ impl<'db> PlaceAndQualifiers<'db> {
self.qualifiers.contains(TypeQualifiers::INIT_VAR)
}
/// Returns `true` if the place has a `Required` type qualifier.
pub(crate) fn is_required(&self) -> bool {
self.qualifiers.contains(TypeQualifiers::REQUIRED)
}
/// Returns `true` if the place has a `NotRequired` type qualifier.
pub(crate) fn is_not_required(&self) -> bool {
self.qualifiers.contains(TypeQualifiers::NOT_REQUIRED)
}
/// Returns `Some(…)` if the place is qualified with `typing.Final` without a specified type.
pub(crate) fn is_bare_final(&self) -> Option<TypeQualifiers> {
match self {

View file

@ -37,7 +37,6 @@ use crate::semantic_index::scope::ScopeId;
use crate::semantic_index::{imported_modules, place_table, semantic_index};
use crate::suppression::check_suppressions;
use crate::types::call::{Binding, Bindings, CallArguments, CallableBinding};
use crate::types::class::{CodeGeneratorKind, Field};
pub(crate) use crate::types::class_base::ClassBase;
use crate::types::constraints::{
Constraints, IteratorConstraintsExtension, OptionConstraintsExtension,
@ -63,10 +62,11 @@ use crate::types::mro::{Mro, MroError, MroIterator};
pub(crate) use crate::types::narrow::infer_narrowing_constraint;
use crate::types::signatures::{Parameter, ParameterForm, Parameters, walk_signature};
use crate::types::tuple::TupleSpec;
pub(crate) use crate::types::typed_dict::{TypedDictParams, TypedDictType, walk_typed_dict_type};
use crate::types::variance::{TypeVarVariance, VarianceInferable};
use crate::unpack::EvaluationMode;
pub use crate::util::diagnostics::add_inferred_python_version_hint_to_diagnostic;
use crate::{Db, FxOrderMap, FxOrderSet, Module, Program};
use crate::{Db, FxOrderSet, Module, Program};
pub(crate) use class::{ClassLiteral, ClassType, GenericAlias, KnownClass};
use instance::Protocol;
pub use instance::{NominalInstanceType, ProtocolInstanceType};
@ -96,6 +96,7 @@ mod string_annotation;
mod subclass_of;
mod tuple;
mod type_ordering;
mod typed_dict;
mod unpacker;
mod variance;
mod visitor;
@ -1039,9 +1040,7 @@ impl<'db> Type<'db> {
}
pub(crate) fn typed_dict(defining_class: impl Into<ClassType<'db>>) -> Self {
Self::TypedDict(TypedDictType {
defining_class: defining_class.into(),
})
Self::TypedDict(TypedDictType::new(defining_class.into()))
}
#[must_use]
@ -5899,7 +5898,7 @@ impl<'db> Type<'db> {
Type::AlwaysTruthy | Type::AlwaysFalsy => KnownClass::Type.to_instance(db),
Type::BoundSuper(_) => KnownClass::Super.to_class_literal(db),
Type::ProtocolInstance(protocol) => protocol.to_meta_type(db),
Type::TypedDict(typed_dict) => SubclassOfType::from(db, typed_dict.defining_class),
Type::TypedDict(typed_dict) => SubclassOfType::from(db, typed_dict.defining_class()),
Type::TypeAlias(alias) => alias.value_type(db).to_meta_type(db),
}
}
@ -6366,7 +6365,7 @@ impl<'db> Type<'db> {
},
Type::TypedDict(typed_dict) => {
Some(TypeDefinition::Class(typed_dict.defining_class.definition(db)))
Some(TypeDefinition::Class(typed_dict.defining_class().definition(db)))
}
Self::Union(_) | Self::Intersection(_) => None,
@ -6879,6 +6878,12 @@ bitflags! {
const FINAL = 1 << 1;
/// `dataclasses.InitVar`
const INIT_VAR = 1 << 2;
/// `typing_extensions.Required`
const REQUIRED = 1 << 3;
/// `typing_extensions.NotRequired`
const NOT_REQUIRED = 1 << 4;
/// `typing_extensions.ReadOnly`
const READ_ONLY = 1 << 5;
}
}
@ -6894,6 +6899,8 @@ impl TypeQualifiers {
Self::CLASS_VAR => "ClassVar",
Self::FINAL => "Final",
Self::INIT_VAR => "InitVar",
Self::REQUIRED => "Required",
Self::NOT_REQUIRED => "NotRequired",
_ => {
unreachable!("Only a single bit should be set when calling `TypeQualifiers::name`")
}
@ -9849,43 +9856,6 @@ impl<'db> EnumLiteralType<'db> {
}
}
/// Type that represents the set of all inhabitants (`dict` instances) that conform to
/// a given `TypedDict` schema.
#[derive(Debug, Copy, Clone, PartialEq, Eq, salsa::Update, Hash, get_size2::GetSize)]
pub struct TypedDictType<'db> {
/// A reference to the class (inheriting from `typing.TypedDict`) that specifies the
/// schema of this `TypedDict`.
defining_class: ClassType<'db>,
}
impl<'db> TypedDictType<'db> {
pub(crate) fn items(self, db: &'db dyn Db) -> FxOrderMap<Name, Field<'db>> {
let (class_literal, specialization) = self.defining_class.class_literal(db);
class_literal.fields(db, specialization, CodeGeneratorKind::TypedDict)
}
pub(crate) fn apply_type_mapping_impl<'a>(
self,
db: &'db dyn Db,
type_mapping: &TypeMapping<'a, 'db>,
visitor: &ApplyTypeMappingVisitor<'db>,
) -> Self {
Self {
defining_class: self
.defining_class
.apply_type_mapping_impl(db, type_mapping, visitor),
}
}
}
fn walk_typed_dict_type<'db, V: visitor::TypeVisitor<'db> + ?Sized>(
db: &'db dyn Db,
typed_dict: TypedDictType<'db>,
visitor: &V,
) {
visitor.visit_type(db, typed_dict.defining_class.into());
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum BoundSuperError<'db> {
InvalidPivotClassType {

View file

@ -27,13 +27,14 @@ use crate::types::generics::{GenericContext, Specialization, walk_specialization
use crate::types::infer::nearest_enclosing_class;
use crate::types::signatures::{CallableSignature, Parameter, Parameters, Signature};
use crate::types::tuple::{TupleSpec, TupleType};
use crate::types::typed_dict::typed_dict_params_from_class_def;
use crate::types::{
ApplyTypeMappingVisitor, Binding, BoundSuperError, BoundSuperType, CallableType,
DataclassParams, DeprecatedInstance, HasRelationToVisitor, IsEquivalentVisitor,
KnownInstanceType, ManualPEP695TypeAliasType, NormalizedVisitor, PropertyInstanceType,
StringLiteralType, TypeAliasType, TypeMapping, TypeRelation, TypeVarBoundOrConstraints,
TypeVarInstance, TypeVarKind, VarianceInferable, declaration_type, infer_definition_types,
todo_type,
TypeVarInstance, TypeVarKind, TypedDictParams, VarianceInferable, declaration_type,
infer_definition_types, todo_type,
};
use crate::{
Db, FxIndexMap, FxOrderSet, Program,
@ -1241,24 +1242,51 @@ impl MethodDecorator {
}
}
/// Kind-specific metadata for different types of fields
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum FieldKind<'db> {
/// `NamedTuple` field metadata
NamedTuple { default_ty: Option<Type<'db>> },
/// dataclass field metadata
Dataclass {
/// The type of the default value for this field
default_ty: Option<Type<'db>>,
/// Whether or not this field is "init-only". If this is true, it only appears in the
/// `__init__` signature, but is not accessible as a real field
init_only: bool,
/// Whether or not this field should appear in the signature of `__init__`.
init: bool,
/// Whether or not this field can only be passed as a keyword argument to `__init__`.
kw_only: Option<bool>,
},
/// `TypedDict` field metadata
TypedDict {
/// Whether this field is required
is_required: bool,
},
}
/// Metadata regarding a dataclass field/attribute or a `TypedDict` "item" / key-value pair.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct Field<'db> {
/// The declared type of the field
pub(crate) declared_ty: Type<'db>,
/// Kind-specific metadata for this field
pub(crate) kind: FieldKind<'db>,
}
/// The type of the default value for this field
pub(crate) default_ty: Option<Type<'db>>,
/// Whether or not this field is "init-only". If this is true, it only appears in the
/// `__init__` signature, but is not accessible as a real field
pub(crate) init_only: bool,
/// Whether or not this field should appear in the signature of `__init__`.
pub(crate) init: bool,
/// Whether or not this field can only be passed as a keyword argument to `__init__`.
pub(crate) kw_only: Option<bool>,
impl Field<'_> {
pub(crate) const fn is_required(&self) -> bool {
match &self.kind {
FieldKind::NamedTuple { default_ty } => default_ty.is_none(),
// A dataclass field is NOT required if `default` (or `default_factory`) is set
// or if `init` has been set to `False`.
FieldKind::Dataclass {
init, default_ty, ..
} => default_ty.is_none() && *init,
FieldKind::TypedDict { is_required } => *is_required,
}
}
}
impl<'db> Field<'db> {
@ -1664,6 +1692,17 @@ impl<'db> ClassLiteral<'db> {
.any(|base| matches!(base, ClassBase::TypedDict))
}
/// Compute `TypedDict` parameters dynamically based on MRO detection and AST parsing.
fn typed_dict_params(self, db: &'db dyn Db) -> Option<TypedDictParams> {
if !self.is_typed_dict(db) {
return None;
}
let module = parsed_module(db, self.file(db)).load(db);
let class_stmt = self.node(db, &module);
Some(typed_dict_params_from_class_def(class_stmt))
}
/// Return the explicit `metaclass` of this class, if one is defined.
///
/// ## Note
@ -1967,7 +2006,10 @@ impl<'db> ClassLiteral<'db> {
}
if CodeGeneratorKind::NamedTuple.matches(db, self) {
if let Some(field) = self.own_fields(db, specialization).get(name) {
if let Some(field) = self
.own_fields(db, specialization, CodeGeneratorKind::NamedTuple)
.get(name)
{
let property_getter_signature = Signature::new(
Parameters::new([Parameter::positional_only(Some(Name::new_static("self")))]),
Some(field.declared_ty),
@ -2033,17 +2075,19 @@ impl<'db> ClassLiteral<'db> {
Type::instance(db, self.apply_optional_specialization(db, specialization));
let signature_from_fields = |mut parameters: Vec<_>, return_ty: Option<Type<'db>>| {
for (
field_name,
field @ Field {
declared_ty: mut field_ty,
mut default_ty,
init_only: _,
init,
kw_only,
},
) in self.fields(db, specialization, field_policy)
{
for (field_name, field) in self.fields(db, specialization, field_policy) {
let (init, mut default_ty, kw_only) = match field.kind {
FieldKind::NamedTuple { default_ty } => (true, default_ty, None),
FieldKind::Dataclass {
init,
default_ty,
kw_only,
..
} => (init, default_ty, kw_only),
FieldKind::TypedDict { .. } => continue,
};
let mut field_ty = field.declared_ty;
if name == "__init__" && !init {
// Skip fields with `init=False`
continue;
@ -2351,7 +2395,7 @@ impl<'db> ClassLiteral<'db> {
if field_policy == CodeGeneratorKind::NamedTuple {
// NamedTuples do not allow multiple inheritance, so it is sufficient to enumerate the
// fields of this class only.
return self.own_fields(db, specialization);
return self.own_fields(db, specialization, field_policy);
}
let matching_classes_in_mro: Vec<_> = self
@ -2374,7 +2418,7 @@ impl<'db> ClassLiteral<'db> {
matching_classes_in_mro
.into_iter()
.rev()
.flat_map(|(class, specialization)| class.own_fields(db, specialization))
.flat_map(|(class, specialization)| class.own_fields(db, specialization, field_policy))
// We collect into a FxOrderMap here to deduplicate attributes
.collect()
}
@ -2394,6 +2438,7 @@ impl<'db> ClassLiteral<'db> {
self,
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
field_policy: CodeGeneratorKind,
) -> FxOrderMap<Name, Field<'db>> {
let mut attributes = FxOrderMap::default();
@ -2401,7 +2446,10 @@ impl<'db> ClassLiteral<'db> {
let table = place_table(db, class_body_scope);
let use_def = use_def_map(db, class_body_scope);
let typed_dict_params = self.typed_dict_params(db);
let mut kw_only_sentinel_field_seen = false;
for (symbol_id, declarations) in use_def.all_end_of_scope_symbol_declarations() {
// Here, we exclude all declarations that are not annotated assignments. We need this because
// things like function definitions and nested classes would otherwise be considered dataclass
@ -2456,12 +2504,34 @@ impl<'db> ClassLiteral<'db> {
}
}
let kind = match field_policy {
CodeGeneratorKind::NamedTuple => FieldKind::NamedTuple { default_ty },
CodeGeneratorKind::DataclassLike => FieldKind::Dataclass {
default_ty,
init_only: attr.is_init_var(),
init,
kw_only,
},
CodeGeneratorKind::TypedDict => {
let is_required = if attr.is_required() {
// Explicit Required[T] annotation - always required
true
} else if attr.is_not_required() {
// Explicit NotRequired[T] annotation - never required
false
} else {
// No explicit qualifier - use class default (`total` parameter)
typed_dict_params
.expect("TypedDictParams should be available for CodeGeneratorKind::TypedDict")
.contains(TypedDictParams::TOTAL)
};
FieldKind::TypedDict { is_required }
}
};
let mut field = Field {
declared_ty: attr_ty.apply_optional_specialization(db, specialization),
default_ty,
init_only: attr.is_init_var(),
init,
kw_only,
kind,
};
// Check if this is a KW_ONLY sentinel and mark subsequent fields as keyword-only
@ -2470,8 +2540,14 @@ impl<'db> ClassLiteral<'db> {
}
// If no explicit kw_only setting and we've seen KW_ONLY sentinel, mark as keyword-only
if field.kw_only.is_none() && kw_only_sentinel_field_seen {
field.kw_only = Some(true);
if kw_only_sentinel_field_seen {
if let FieldKind::Dataclass {
kw_only: ref mut kw @ None,
..
} = field.kind
{
*kw = Some(true);
}
}
attributes.insert(symbol.name().clone(), field);

View file

@ -1,4 +1,5 @@
use crate::Db;
use crate::types::class::CodeGeneratorKind;
use crate::types::generics::Specialization;
use crate::types::tuple::TupleType;
use crate::types::{
@ -206,7 +207,7 @@ impl<'db> ClassBase<'db> {
SpecialFormType::Generic => Some(Self::Generic),
SpecialFormType::NamedTuple => {
let fields = subclass.own_fields(db, None);
let fields = subclass.own_fields(db, None, CodeGeneratorKind::NamedTuple);
Self::try_from_type(
db,
TupleType::heterogeneous(

View file

@ -96,6 +96,7 @@ pub(crate) fn register_lints(registry: &mut LintRegistryBuilder) {
registry.register_lint(&INVALID_ATTRIBUTE_ACCESS);
registry.register_lint(&REDUNDANT_CAST);
registry.register_lint(&UNRESOLVED_GLOBAL);
registry.register_lint(&MISSING_TYPED_DICT_KEY);
// String annotations
registry.register_lint(&BYTE_STRING_TYPE_ANNOTATION);
@ -1758,6 +1759,33 @@ declare_lint! {
}
}
declare_lint! {
/// ## What it does
/// Detects missing required keys in `TypedDict` constructor calls.
///
/// ## Why is this bad?
/// `TypedDict` requires all non-optional keys to be provided during construction.
/// Missing items can lead to a `KeyError` at runtime.
///
/// ## Example
/// ```python
/// from typing import TypedDict
///
/// class Person(TypedDict):
/// name: str
/// age: int
///
/// alice: Person = {"name": "Alice"} # missing required key 'age'
///
/// alice["age"] # KeyError
/// ```
pub(crate) static MISSING_TYPED_DICT_KEY = {
summary: "detects missing required keys in `TypedDict` constructors",
status: LintStatus::preview("1.0.0"),
default_level: Level::Error,
}
}
/// A collection of type check diagnostics.
#[derive(Default, Eq, PartialEq, get_size2::GetSize)]
pub struct TypeCheckDiagnostics {
@ -2761,18 +2789,18 @@ fn report_invalid_base<'ctx, 'db>(
pub(crate) fn report_invalid_key_on_typed_dict<'db>(
context: &InferContext<'db, '_>,
value_node: AnyNodeRef,
slice_node: AnyNodeRef,
value_ty: Type<'db>,
slice_ty: Type<'db>,
typed_dict_node: AnyNodeRef,
key_node: AnyNodeRef,
typed_dict_ty: Type<'db>,
key_ty: Type<'db>,
items: &FxOrderMap<Name, Field<'db>>,
) {
let db = context.db();
if let Some(builder) = context.report_lint(&INVALID_KEY, slice_node) {
match slice_ty {
if let Some(builder) = context.report_lint(&INVALID_KEY, key_node) {
match key_ty {
Type::StringLiteral(key) => {
let key = key.value(db);
let typed_dict_name = value_ty.display(db);
let typed_dict_name = typed_dict_ty.display(db);
let mut diagnostic = builder.into_diagnostic(format_args!(
"Invalid key access on TypedDict `{typed_dict_name}`",
@ -2780,7 +2808,7 @@ pub(crate) fn report_invalid_key_on_typed_dict<'db>(
diagnostic.annotate(
context
.secondary(value_node)
.secondary(typed_dict_node)
.message(format_args!("TypedDict `{typed_dict_name}`")),
);
@ -2799,8 +2827,8 @@ pub(crate) fn report_invalid_key_on_typed_dict<'db>(
}
_ => builder.into_diagnostic(format_args!(
"TypedDict `{}` cannot be indexed with a key of type `{}`",
value_ty.display(db),
slice_ty.display(db),
typed_dict_ty.display(db),
key_ty.display(db),
)),
};
}
@ -2860,6 +2888,21 @@ pub(super) fn report_namedtuple_field_without_default_after_field_with_default<'
}
}
pub(crate) fn report_missing_typed_dict_key<'db>(
context: &InferContext<'db, '_>,
constructor_node: AnyNodeRef,
typed_dict_ty: Type<'db>,
missing_field: &str,
) {
let db = context.db();
if let Some(builder) = context.report_lint(&MISSING_TYPED_DICT_KEY, constructor_node) {
let typed_dict_name = typed_dict_ty.display(db);
builder.into_diagnostic(format_args!(
"Missing required key '{missing_field}' in TypedDict `{typed_dict_name}` constructor",
));
}
}
/// This function receives an unresolved `from foo import bar` import,
/// where `foo` can be resolved to a module but that module does not
/// have a `bar` member or submodule.

View file

@ -315,7 +315,7 @@ impl Display for DisplayRepresentation<'_> {
}
f.write_str("]")
}
Type::TypedDict(typed_dict) => f.write_str(typed_dict.defining_class.name(self.db)),
Type::TypedDict(typed_dict) => f.write_str(typed_dict.defining_class().name(self.db)),
Type::TypeAlias(alias) => f.write_str(alias.name(self.db)),
}
}

View file

@ -90,7 +90,7 @@ use crate::semantic_index::{
ApplicableConstraints, EnclosingSnapshotResult, SemanticIndex, place_table, semantic_index,
};
use crate::types::call::{Binding, Bindings, CallArguments, CallError, CallErrorKind};
use crate::types::class::{CodeGeneratorKind, MetaclassErrorKind};
use crate::types::class::{CodeGeneratorKind, FieldKind, MetaclassErrorKind};
use crate::types::diagnostic::{
self, CALL_NON_CALLABLE, CONFLICTING_DECLARATIONS, CONFLICTING_METACLASS,
CYCLIC_CLASS_DEFINITION, DIVISION_BY_ZERO, DUPLICATE_KW_ONLY, INCONSISTENT_MRO,
@ -117,6 +117,10 @@ use crate::types::instance::SliceLiteral;
use crate::types::mro::MroErrorKind;
use crate::types::signatures::{CallableSignature, Signature};
use crate::types::tuple::{Tuple, TupleSpec, TupleSpecBuilder, TupleType};
use crate::types::typed_dict::{
TypedDictAssignmentKind, validate_typed_dict_constructor, validate_typed_dict_dict_literal,
validate_typed_dict_key_assignment,
};
use crate::types::unpacker::{UnpackResult, Unpacker};
use crate::types::{
CallDunderError, CallableType, ClassLiteral, ClassType, DataclassParams, DynamicType,
@ -1118,8 +1122,15 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
if is_named_tuple {
let mut field_with_default_encountered = None;
for (field_name, field) in class.own_fields(self.db(), None) {
if field.default_ty.is_some() {
for (field_name, field) in
class.own_fields(self.db(), None, CodeGeneratorKind::NamedTuple)
{
if matches!(
field.kind,
FieldKind::NamedTuple {
default_ty: Some(_)
}
) {
field_with_default_encountered = Some(field_name);
} else if let Some(field_with_default) = field_with_default_encountered.as_ref()
{
@ -3804,47 +3815,16 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
if let Some(typed_dict) = value_ty.into_typed_dict() {
if let Some(key) = slice_ty.into_string_literal() {
let key = key.value(self.db());
let items = typed_dict.items(self.db());
if let Some((_, item)) =
items.iter().find(|(name, _)| *name == key)
{
if let Some(builder) =
context.report_lint(&INVALID_ASSIGNMENT, rhs)
{
let mut diagnostic = builder.into_diagnostic(format_args!(
"Invalid assignment to key \"{key}\" with declared type `{}` on TypedDict `{value_d}`",
item.declared_ty.display(db),
));
diagnostic.set_primary_message(format_args!(
"value of type `{assigned_d}`"
));
diagnostic.annotate(
self.context
.secondary(value.as_ref())
.message(format_args!("TypedDict `{value_d}`")),
);
diagnostic.annotate(
self.context.secondary(slice.as_ref()).message(
format_args!(
"key has declared type `{}`",
item.declared_ty.display(db),
),
),
);
}
} else {
report_invalid_key_on_typed_dict(
&self.context,
value.as_ref().into(),
slice.as_ref().into(),
value_ty,
slice_ty,
&items,
);
}
validate_typed_dict_key_assignment(
&self.context,
typed_dict,
key,
assigned_ty,
value.as_ref(),
slice.as_ref(),
rhs,
TypedDictAssignmentKind::Subscript,
);
} else {
// Check if the key has a valid type. We only allow string literals, a union of string literals,
// or a dynamic type like `Any`. We can do this by checking assignability to `LiteralString`,
@ -4695,7 +4675,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
if let Some(value) = value {
let inferred_ty = self.infer_maybe_standalone_expression(value);
let inferred_ty = if target
let mut inferred_ty = if target
.as_name_expr()
.is_some_and(|name| &name.id == "TYPE_CHECKING")
{
@ -4705,6 +4685,25 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
} else {
inferred_ty
};
// Validate `TypedDict` dictionary literal assignments
if let Some(typed_dict) = declared.inner_type().into_typed_dict() {
if let Some(dict_expr) = value.as_dict_expr() {
validate_typed_dict_dict_literal(
&self.context,
typed_dict,
dict_expr,
target.into(),
|expr| self.expression_type(expr),
);
// Override the inferred type of the dict literal to be the `TypedDict` type
// This ensures that the dict literal gets the correct type for key access
let typed_dict_type = Type::TypedDict(typed_dict);
inferred_ty = typed_dict_type;
}
}
self.add_declaration_with_binding(
target.into(),
definition,
@ -6269,6 +6268,22 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
.match_parameters(&call_arguments);
self.infer_argument_types(arguments, &mut call_arguments, &bindings.argument_forms);
// Validate `TypedDict` constructor calls after argument type inference
if let Some(class_literal) = callable_type.into_class_literal() {
if class_literal.is_typed_dict(self.db()) {
let typed_dict_type = Type::typed_dict(ClassType::NonGeneric(class_literal));
if let Some(typed_dict) = typed_dict_type.into_typed_dict() {
validate_typed_dict_constructor(
&self.context,
typed_dict,
arguments,
func.as_ref().into(),
|expr| self.expression_type(expr),
);
}
}
}
let mut bindings = match bindings.check_types(self.db(), &call_arguments) {
Ok(bindings) => bindings,
Err(CallError(_, bindings)) => {
@ -9422,6 +9437,15 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
Type::SpecialForm(SpecialFormType::Final) => {
TypeAndQualifiers::new(Type::unknown(), TypeQualifiers::FINAL)
}
Type::SpecialForm(SpecialFormType::Required) => {
TypeAndQualifiers::new(Type::unknown(), TypeQualifiers::REQUIRED)
}
Type::SpecialForm(SpecialFormType::NotRequired) => {
TypeAndQualifiers::new(Type::unknown(), TypeQualifiers::NOT_REQUIRED)
}
Type::SpecialForm(SpecialFormType::ReadOnly) => {
TypeAndQualifiers::new(Type::unknown(), TypeQualifiers::READ_ONLY)
}
Type::ClassLiteral(class)
if class.is_known(self.db(), KnownClass::InitVar) =>
{
@ -9497,7 +9521,11 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
}
}
Type::SpecialForm(
type_qualifier @ (SpecialFormType::ClassVar | SpecialFormType::Final),
type_qualifier @ (SpecialFormType::ClassVar
| SpecialFormType::Final
| SpecialFormType::Required
| SpecialFormType::NotRequired
| SpecialFormType::ReadOnly),
) => {
let arguments = if let ast::Expr::Tuple(tuple) = slice {
&*tuple.elts
@ -9516,6 +9544,15 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
SpecialFormType::Final => {
type_and_qualifiers.add_qualifier(TypeQualifiers::FINAL);
}
SpecialFormType::Required => {
type_and_qualifiers.add_qualifier(TypeQualifiers::REQUIRED);
}
SpecialFormType::NotRequired => {
type_and_qualifiers.add_qualifier(TypeQualifiers::NOT_REQUIRED);
}
SpecialFormType::ReadOnly => {
type_and_qualifiers.add_qualifier(TypeQualifiers::READ_ONLY);
}
_ => unreachable!(),
}
type_and_qualifiers
@ -10802,15 +10839,11 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
KnownClass::Deque,
),
SpecialFormType::ReadOnly => {
self.infer_type_expression(arguments_slice);
todo_type!("`ReadOnly[]` type qualifier")
}
SpecialFormType::NotRequired => {
self.infer_type_expression(arguments_slice);
todo_type!("`NotRequired[]` type qualifier")
}
SpecialFormType::ClassVar | SpecialFormType::Final => {
SpecialFormType::ClassVar
| SpecialFormType::Final
| SpecialFormType::Required
| SpecialFormType::NotRequired
| SpecialFormType::ReadOnly => {
if let Some(builder) = self.context.report_lint(&INVALID_TYPE_FORM, subscript) {
let diag = builder.into_diagnostic(format_args!(
"Type qualifier `{special_form}` is not allowed in type expressions \
@ -10820,10 +10853,6 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
}
self.infer_type_expression(arguments_slice)
}
SpecialFormType::Required => {
self.infer_type_expression(arguments_slice);
todo_type!("`Required[]` type qualifier")
}
SpecialFormType::TypeIs => match arguments_slice {
ast::Expr::Tuple(_) => {
self.infer_type_expression(arguments_slice);

View file

@ -245,7 +245,7 @@ pub(super) fn union_or_intersection_elements_ordering<'db>(
}
(Type::TypedDict(left), Type::TypedDict(right)) => {
left.defining_class.cmp(&right.defining_class)
left.defining_class().cmp(&right.defining_class())
}
(Type::TypedDict(_), _) => Ordering::Less,
(_, Type::TypedDict(_)) => Ordering::Greater,

View file

@ -0,0 +1,361 @@
use bitflags::bitflags;
use ruff_python_ast::Arguments;
use ruff_python_ast::{self as ast, AnyNodeRef, StmtClassDef, name::Name};
use super::class::{ClassType, CodeGeneratorKind, Field};
use super::context::InferContext;
use super::diagnostic::{
INVALID_ARGUMENT_TYPE, INVALID_ASSIGNMENT, report_invalid_key_on_typed_dict,
report_missing_typed_dict_key,
};
use super::{ApplyTypeMappingVisitor, Type, TypeMapping, visitor};
use crate::{Db, FxOrderMap};
use ordermap::OrderSet;
bitflags! {
/// Used for `TypedDict` class parameters.
/// Keeps track of the keyword arguments that were passed-in during class definition.
/// (see https://typing.python.org/en/latest/spec/typeddict.html)
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct TypedDictParams: u8 {
/// Whether keys are required by default (`total=True`)
const TOTAL = 1 << 0;
}
}
impl get_size2::GetSize for TypedDictParams {}
impl Default for TypedDictParams {
fn default() -> Self {
Self::TOTAL
}
}
/// Type that represents the set of all inhabitants (`dict` instances) that conform to
/// a given `TypedDict` schema.
#[derive(Debug, Copy, Clone, PartialEq, Eq, salsa::Update, Hash, get_size2::GetSize)]
pub struct TypedDictType<'db> {
/// A reference to the class (inheriting from `typing.TypedDict`) that specifies the
/// schema of this `TypedDict`.
defining_class: ClassType<'db>,
}
impl<'db> TypedDictType<'db> {
pub(crate) fn new(defining_class: ClassType<'db>) -> Self {
Self { defining_class }
}
pub(crate) fn defining_class(self) -> ClassType<'db> {
self.defining_class
}
pub(crate) fn items(self, db: &'db dyn Db) -> FxOrderMap<Name, Field<'db>> {
let (class_literal, specialization) = self.defining_class.class_literal(db);
class_literal.fields(db, specialization, CodeGeneratorKind::TypedDict)
}
pub(crate) fn apply_type_mapping_impl<'a>(
self,
db: &'db dyn Db,
type_mapping: &TypeMapping<'a, 'db>,
visitor: &ApplyTypeMappingVisitor<'db>,
) -> Self {
Self {
defining_class: self
.defining_class
.apply_type_mapping_impl(db, type_mapping, visitor),
}
}
}
pub(crate) fn walk_typed_dict_type<'db, V: visitor::TypeVisitor<'db> + ?Sized>(
db: &'db dyn Db,
typed_dict: TypedDictType<'db>,
visitor: &V,
) {
visitor.visit_type(db, typed_dict.defining_class.into());
}
pub(super) fn typed_dict_params_from_class_def(class_stmt: &StmtClassDef) -> TypedDictParams {
let mut typed_dict_params = TypedDictParams::default();
// Check for `total` keyword argument in the class definition
// Note that it is fine to only check for Boolean literals here
// (https://typing.python.org/en/latest/spec/typeddict.html#totality)
if let Some(arguments) = &class_stmt.arguments {
for keyword in &arguments.keywords {
if keyword.arg.as_deref() == Some("total")
&& matches!(
&keyword.value,
ast::Expr::BooleanLiteral(ast::ExprBooleanLiteral { value: false, .. })
)
{
typed_dict_params.remove(TypedDictParams::TOTAL);
}
}
}
typed_dict_params
}
#[derive(Debug, Clone, Copy)]
pub(super) enum TypedDictAssignmentKind {
/// For subscript assignments like `d["key"] = value`
Subscript,
/// For constructor arguments like `MyTypedDict(key=value)`
Constructor,
}
impl TypedDictAssignmentKind {
fn diagnostic_name(self) -> &'static str {
match self {
Self::Subscript => "assignment",
Self::Constructor => "argument",
}
}
fn diagnostic_type(self) -> &'static crate::lint::LintMetadata {
match self {
Self::Subscript => &INVALID_ASSIGNMENT,
Self::Constructor => &INVALID_ARGUMENT_TYPE,
}
}
}
/// Validates assignment of a value to a specific key on a `TypedDict`.
/// Returns true if the assignment is valid, false otherwise.
#[allow(clippy::too_many_arguments)]
pub(super) fn validate_typed_dict_key_assignment<'db, 'ast>(
context: &InferContext<'db, 'ast>,
typed_dict: TypedDictType<'db>,
key: &str,
value_ty: Type<'db>,
typed_dict_node: impl Into<AnyNodeRef<'ast>>,
key_node: impl Into<AnyNodeRef<'ast>>,
value_node: impl Into<AnyNodeRef<'ast>>,
assignment_kind: TypedDictAssignmentKind,
) -> bool {
let db = context.db();
let items = typed_dict.items(db);
// Check if key exists in `TypedDict`
let Some((_, item)) = items.iter().find(|(name, _)| *name == key) else {
report_invalid_key_on_typed_dict(
context,
typed_dict_node.into(),
key_node.into(),
Type::TypedDict(typed_dict),
Type::string_literal(db, key),
&items,
);
return false;
};
// Key exists, check if value type is assignable to declared type
if value_ty.is_assignable_to(db, item.declared_ty) {
return true;
}
// Invalid assignment - emit diagnostic
if let Some(builder) = context.report_lint(assignment_kind.diagnostic_type(), value_node.into())
{
let typed_dict_ty = Type::TypedDict(typed_dict);
let typed_dict_d = typed_dict_ty.display(db);
let value_d = value_ty.display(db);
let item_type_d = item.declared_ty.display(db);
let mut diagnostic = builder.into_diagnostic(format_args!(
"Invalid {} to key \"{key}\" with declared type `{item_type_d}` on TypedDict `{typed_dict_d}`",
assignment_kind.diagnostic_name(),
));
diagnostic.set_primary_message(format_args!("value of type `{value_d}`"));
diagnostic.annotate(
context
.secondary(typed_dict_node.into())
.message(format_args!("TypedDict `{typed_dict_d}`")),
);
diagnostic.annotate(
context
.secondary(key_node.into())
.message(format_args!("key has declared type `{item_type_d}`")),
);
}
false
}
/// Validates that all required keys are provided in a `TypedDict` construction.
/// Reports errors for any keys that are required but not provided.
pub(super) fn validate_typed_dict_required_keys<'db, 'ast>(
context: &InferContext<'db, 'ast>,
typed_dict: TypedDictType<'db>,
provided_keys: &OrderSet<&str>,
error_node: AnyNodeRef<'ast>,
) {
let db = context.db();
let items = typed_dict.items(db);
let required_keys: OrderSet<&str> = items
.iter()
.filter_map(|(key_name, field)| field.is_required().then_some(key_name.as_str()))
.collect();
for missing_key in required_keys.difference(provided_keys) {
report_missing_typed_dict_key(
context,
error_node,
Type::TypedDict(typed_dict),
missing_key,
);
}
}
pub(super) fn validate_typed_dict_constructor<'db, 'ast>(
context: &InferContext<'db, 'ast>,
typed_dict: TypedDictType<'db>,
arguments: &'ast Arguments,
error_node: AnyNodeRef<'ast>,
expression_type_fn: impl Fn(&ast::Expr) -> Type<'db>,
) {
let has_positional_dict = arguments.args.len() == 1 && arguments.args[0].is_dict_expr();
let provided_keys = if has_positional_dict {
validate_from_dict_literal(
context,
typed_dict,
arguments,
error_node,
&expression_type_fn,
)
} else {
validate_from_keywords(
context,
typed_dict,
arguments,
error_node,
&expression_type_fn,
)
};
validate_typed_dict_required_keys(context, typed_dict, &provided_keys, error_node);
}
/// Validates a `TypedDict` constructor call with a single positional dictionary argument
/// e.g. `Person({"name": "Alice", "age": 30})`
fn validate_from_dict_literal<'db, 'ast>(
context: &InferContext<'db, 'ast>,
typed_dict: TypedDictType<'db>,
arguments: &'ast Arguments,
error_node: AnyNodeRef<'ast>,
expression_type_fn: &impl Fn(&ast::Expr) -> Type<'db>,
) -> OrderSet<&'ast str> {
let mut provided_keys = OrderSet::new();
if let ast::Expr::Dict(dict_expr) = &arguments.args[0] {
// Validate dict entries
for dict_item in &dict_expr.items {
if let Some(ref key_expr) = dict_item.key {
if let ast::Expr::StringLiteral(ast::ExprStringLiteral {
value: key_value, ..
}) = key_expr
{
let key_str = key_value.to_str();
provided_keys.insert(key_str);
// Get the already-inferred argument type
let value_type = expression_type_fn(&dict_item.value);
validate_typed_dict_key_assignment(
context,
typed_dict,
key_str,
value_type,
error_node,
key_expr,
&dict_item.value,
TypedDictAssignmentKind::Constructor,
);
}
}
}
}
provided_keys
}
/// Validates a `TypedDict` constructor call with keywords
/// e.g. `Person(name="Alice", age=30)`
fn validate_from_keywords<'db, 'ast>(
context: &InferContext<'db, 'ast>,
typed_dict: TypedDictType<'db>,
arguments: &'ast Arguments,
error_node: AnyNodeRef<'ast>,
expression_type_fn: &impl Fn(&ast::Expr) -> Type<'db>,
) -> OrderSet<&'ast str> {
let provided_keys: OrderSet<&str> = arguments
.keywords
.iter()
.filter_map(|kw| kw.arg.as_ref().map(|arg| arg.id.as_str()))
.collect();
// Validate that each key is assigned a type that is compatible with the keys's value type
for keyword in &arguments.keywords {
if let Some(arg_name) = &keyword.arg {
// Get the already-inferred argument type
let arg_type = expression_type_fn(&keyword.value);
validate_typed_dict_key_assignment(
context,
typed_dict,
arg_name.as_str(),
arg_type,
error_node,
keyword,
&keyword.value,
TypedDictAssignmentKind::Constructor,
);
}
}
provided_keys
}
/// Validates a `TypedDict` dictionary literal assignment
/// e.g. `person: Person = {"name": "Alice", "age": 30}`
pub(super) fn validate_typed_dict_dict_literal<'db, 'ast>(
context: &InferContext<'db, 'ast>,
typed_dict: TypedDictType<'db>,
dict_expr: &'ast ast::ExprDict,
error_node: AnyNodeRef<'ast>,
expression_type_fn: impl Fn(&ast::Expr) -> Type<'db>,
) -> OrderSet<&'ast str> {
let mut provided_keys = OrderSet::new();
// Validate each key-value pair in the dictionary literal
for item in &dict_expr.items {
if let Some(key_expr) = &item.key {
if let ast::Expr::StringLiteral(key_literal) = key_expr {
let key_str = key_literal.value.to_str();
provided_keys.insert(key_str);
let value_type = expression_type_fn(&item.value);
validate_typed_dict_key_assignment(
context,
typed_dict,
key_str,
value_type,
error_node,
key_expr,
&item.value,
TypedDictAssignmentKind::Constructor,
);
}
}
}
validate_typed_dict_required_keys(context, typed_dict, &provided_keys, error_node);
provided_keys
}