[ty] validate constructor call of TypedDict (#19810)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run

## Summary
Implement validation for `TypedDict` constructor calls and dictionary
literal assignments, including support for `total=False` and proper
field management.
Also add support for `Required` and `NotRequired` type qualifiers in
`TypedDict` classes, along with proper inheritance behavior and the
`total=` parameter.
Support both constructor calls and dict literal syntax

part of https://github.com/astral-sh/ty/issues/154

### Basic Required Field Validation
```py
class Person(TypedDict):
    name: str
    age: int | None

# Error: Missing required field 'name' in TypedDict `Person` constructor
incomplete = Person(age=25)

# Error: Invalid argument to key "name" with declared type `str` on TypedDict `Person`
wrong_type = Person(name=123, age=25)

# Error: Invalid key access on TypedDict `Person`: Unknown key "extra"
extra_field = Person(name="Bob", age=25, extra=True)
```
<img width="773" height="191" alt="Screenshot 2025-08-07 at 17 59 22"
src="https://github.com/user-attachments/assets/79076d98-e85f-4495-93d6-a731aa72a5c9"
/>

### Support for `total=False`
```py
class OptionalPerson(TypedDict, total=False):
    name: str
    age: int | None

# All valid - all fields are optional with total=False
charlie = OptionalPerson()
david = OptionalPerson(name="David")
emily = OptionalPerson(age=30)
frank = OptionalPerson(name="Frank", age=25)

# But type validation and extra fields still apply
invalid_type = OptionalPerson(name=123)  # Error: Invalid argument type
invalid_extra = OptionalPerson(extra=True)  # Error: Invalid key access
```

### Dictionary Literal Validation
```py
# Type checking works for both constructors and dict literals
person: Person = {"name": "Alice", "age": 30}

reveal_type(person["name"])  # revealed: str
reveal_type(person["age"])   # revealed: int | None

# Error: Invalid key access on TypedDict `Person`: Unknown key "non_existing"
reveal_type(person["non_existing"])  # revealed: Unknown
```

### `Required`, `NotRequired`, `total`
```python
from typing import TypedDict
from typing_extensions import Required, NotRequired

class PartialUser(TypedDict, total=False):
    name: Required[str]      # Required despite total=False
    age: int                 # Optional due to total=False
    email: NotRequired[str]  # Explicitly optional (redundant)

class User(TypedDict):
    name: Required[str]      # Explicitly required (redundant)
    age: int                 # Required due to total=True
    bio: NotRequired[str]    # Optional despite total=True

# Valid constructions
partial = PartialUser(name="Alice")  # name required, age optional
full = User(name="Bob", age=25)      # name and age required, bio optional

# Inheritance maintains original field requirements
class Employee(PartialUser):
    department: str                  # Required (new field)
    # name: still Required (inherited)
    # age: still optional (inherited)

emp = Employee(name="Charlie", department="Engineering")  # 
Employee(department="Engineering")  # 
e: Employee = {"age": 1}  # 
```

<img width="898" height="683" alt="Screenshot 2025-08-11 at 22 02 57"
src="https://github.com/user-attachments/assets/4c1b18cd-cb2e-493a-a948-51589d121738"
/>

## Implementation
The implementation reuses existing validation logic done in
https://github.com/astral-sh/ruff/pull/19782

### ℹ️ Why I did NOT synthesize an `__init__` for `TypedDict`:

`TypedDict` inherits `dict.__init__(self, *args, **kwargs)` that accepts
all arguments.
The type resolution system finds this inherited signature **before**
looking for synthesized members.
So `own_synthesized_member()` is never called because a signature
already exists.

To force synthesis, you'd have to override Python’s inheritance
mechanism, which would break compatibility with the existing ecosystem.

This is why I went with ad-hoc validation. IMO it's the only viable
approach that respects Python’s
inheritance semantics while providing the required validation.

### Refacto of `Field`

**Before:**
```rust
struct Field<'db> {
    declared_ty: Type<'db>,
    default_ty: Option<Type<'db>>,     // NamedTuple and dataclass only
    init_only: bool,                   // dataclass only  
    init: bool,                        // dataclass only
    is_required: Option<bool>,         // TypedDict only
}
```

**After:**
```rust
struct Field<'db> {
    declared_ty: Type<'db>,
    kind: FieldKind<'db>,
}

enum FieldKind<'db> {
    NamedTuple { default_ty: Option<Type<'db>> },
    Dataclass { default_ty: Option<Type<'db>>, init_only: bool, init: bool },
    TypedDict { is_required: bool },
}
```

## Test Plan
Updated Markdown tests

---------

Co-authored-by: David Peter <mail@david-peter.de>
This commit is contained in:
Eric Jolibois 2025-08-25 14:45:52 +02:00 committed by GitHub
parent 376e3ff395
commit f9bbee33f6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
14 changed files with 1113 additions and 248 deletions

View file

@ -6,14 +6,12 @@ Several type qualifiers are unsupported by ty currently. However, we also don't
errors if you use one in an annotation:
```py
from typing_extensions import Final, Required, NotRequired, ReadOnly, TypedDict
from typing_extensions import Final, ReadOnly, TypedDict
X: Final = 42
Y: Final[int] = 42
class Bar(TypedDict):
x: Required[int]
y: NotRequired[str]
z: ReadOnly[bytes]
```
@ -25,23 +23,16 @@ One thing that is supported is error messages for using type qualifiers in type
from typing_extensions import Final, ClassVar, Required, NotRequired, ReadOnly
def _(
a: (
Final # error: [invalid-type-form] "Type qualifier `typing.Final` is not allowed in type expressions (only in annotation expressions)"
| int
),
b: (
ClassVar # error: [invalid-type-form] "Type qualifier `typing.ClassVar` is not allowed in type expressions (only in annotation expressions)"
| int
),
c: Required, # error: [invalid-type-form] "Type qualifier `typing.Required` is not allowed in type expressions (only in annotation expressions, and only with exactly one argument)"
d: NotRequired, # error: [invalid-type-form] "Type qualifier `typing.NotRequired` is not allowed in type expressions (only in annotation expressions, and only with exactly one argument)"
e: ReadOnly, # error: [invalid-type-form] "Type qualifier `typing.ReadOnly` is not allowed in type expressions (only in annotation expressions, and only with exactly one argument)"
# error: [invalid-type-form] "Type qualifier `typing.Final` is not allowed in type expressions (only in annotation expressions)"
a: Final | int,
# error: [invalid-type-form] "Type qualifier `typing.ClassVar` is not allowed in type expressions (only in annotation expressions)"
b: ClassVar | int,
# error: [invalid-type-form] "Type qualifier `typing.ReadOnly` is not allowed in type expressions (only in annotation expressions, and only with exactly one argument)"
c: ReadOnly | int,
) -> None:
reveal_type(a) # revealed: Unknown | int
reveal_type(b) # revealed: Unknown | int
reveal_type(c) # revealed: Unknown
reveal_type(d) # revealed: Unknown
reveal_type(e) # revealed: Unknown
reveal_type(c) # revealed: Unknown | int
```
## Inheritance
@ -53,7 +44,5 @@ from typing_extensions import Final, ClassVar, Required, NotRequired, ReadOnly
class A(Final): ... # error: [invalid-base]
class B(ClassVar): ... # error: [invalid-base]
class C(Required): ... # error: [invalid-base]
class D(NotRequired): ... # error: [invalid-base]
class E(ReadOnly): ... # error: [invalid-base]
class C(ReadOnly): ... # error: [invalid-base]
```

View file

@ -21,12 +21,10 @@ inferred based on the `TypedDict` definition:
```py
alice: Person = {"name": "Alice", "age": 30}
# TODO: this should be `str`
reveal_type(alice["name"]) # revealed: Unknown
# TODO: this should be `int | None`
reveal_type(alice["age"]) # revealed: Unknown
reveal_type(alice["name"]) # revealed: str
reveal_type(alice["age"]) # revealed: int | None
# TODO: this should reveal `Unknown`, and it should emit an error
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "non_existing""
reveal_type(alice["non_existing"]) # revealed: Unknown
```
@ -51,23 +49,26 @@ bob.update(age=26)
The construction of a `TypedDict` is checked for type correctness:
```py
# TODO: these should be errors (invalid argument type)
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `Person`"
eve1a: Person = {"name": b"Eve", "age": None}
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `Person`"
eve1b = Person(name=b"Eve", age=None)
# TODO: these should be errors (missing required key)
# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor"
eve2a: Person = {"age": 22}
# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor"
eve2b = Person(age=22)
# TODO: these should be errors (additional key)
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
eve3a: Person = {"name": "Eve", "age": 25, "extra": True}
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
eve3b = Person(name="Eve", age=25, extra=True)
```
Assignments to keys are also validated:
```py
# TODO: this should be an error
# error: [invalid-assignment] "Invalid assignment to key "name" with declared type `str` on TypedDict `Person`: value of type `None`"
alice["name"] = None
# error: [invalid-assignment] "Invalid assignment to key "name" with declared type `str` on TypedDict `Person`: value of type `None`"
@ -77,13 +78,221 @@ bob["name"] = None
Assignments to non-existing keys are disallowed:
```py
# TODO: this should be an error
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
alice["extra"] = True
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
bob["extra"] = True
```
## Validation of `TypedDict` construction
```py
from typing import TypedDict
class Person(TypedDict):
name: str
age: int | None
class House:
owner: Person
house = House()
def accepts_person(p: Person) -> None:
pass
```
The following constructions of `Person` are all valid:
```py
alice1: Person = {"name": "Alice", "age": 30}
Person(name="Alice", age=30)
Person({"name": "Alice", "age": 30})
accepts_person({"name": "Alice", "age": 30})
house.owner = {"name": "Alice", "age": 30}
```
All of these are missing the required `age` field:
```py
# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor"
alice2: Person = {"name": "Alice"}
# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor"
Person(name="Alice")
# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor"
Person({"name": "Alice"})
# TODO: this should be an error, similar to the above
accepts_person({"name": "Alice"})
# TODO: this should be an error, similar to the above
house.owner = {"name": "Alice"}
```
All of these have an invalid type for the `name` field:
```py
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `Person`: value of type `None`"
alice3: Person = {"name": None, "age": 30}
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `Person`: value of type `None`"
Person(name=None, age=30)
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `Person`: value of type `None`"
Person({"name": None, "age": 30})
# TODO: this should be an error, similar to the above
accepts_person({"name": None, "age": 30})
# TODO: this should be an error, similar to the above
house.owner = {"name": None, "age": 30}
```
All of these have an extra field that is not defined in the `TypedDict`:
```py
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
alice4: Person = {"name": "Alice", "age": 30, "extra": True}
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
Person(name="Alice", age=30, extra=True)
# error: [invalid-key] "Invalid key access on TypedDict `Person`: Unknown key "extra""
Person({"name": "Alice", "age": 30, "extra": True})
# TODO: this should be an error
accepts_person({"name": "Alice", "age": 30, "extra": True})
# TODO: this should be an error
house.owner = {"name": "Alice", "age": 30, "extra": True}
```
## Type ignore compatibility issues
Users should be able to ignore TypedDict validation errors with `# type: ignore`
```py
from typing import TypedDict
class Person(TypedDict):
name: str
age: int
alice_bad: Person = {"name": None} # type: ignore
Person(name=None, age=30) # type: ignore
Person(name="Alice", age=30, extra=True) # type: ignore
```
## Positional dictionary constructor pattern
The positional dictionary constructor pattern (used by libraries like strawberry) should work
correctly:
```py
from typing import TypedDict
class User(TypedDict):
name: str
age: int
# Valid usage - all required fields provided
user1 = User({"name": "Alice", "age": 30})
# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `User` constructor"
user2 = User({"name": "Bob"})
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `User`: value of type `None`"
user3 = User({"name": None, "age": 25})
# error: [invalid-key] "Invalid key access on TypedDict `User`: Unknown key "extra""
user4 = User({"name": "Charlie", "age": 30, "extra": True})
```
## Optional fields with `total=False`
By default, all fields in a `TypedDict` are required (`total=True`). You can make all fields
optional by setting `total=False`:
```py
from typing import TypedDict
class OptionalPerson(TypedDict, total=False):
name: str
age: int | None
# All fields are optional with total=False
charlie = OptionalPerson()
david = OptionalPerson(name="David")
emily = OptionalPerson(age=30)
frank = OptionalPerson(name="Frank", age=25)
# TODO: we could emit an error here, because these fields are not guaranteed to exist
reveal_type(charlie["name"]) # revealed: str
reveal_type(david["age"]) # revealed: int | None
```
Type validation still applies to provided fields:
```py
# error: [invalid-argument-type] "Invalid argument to key "name" with declared type `str` on TypedDict `OptionalPerson`"
invalid = OptionalPerson(name=123)
```
Extra fields are still not allowed, even with `total=False`:
```py
# error: [invalid-key] "Invalid key access on TypedDict `OptionalPerson`: Unknown key "extra""
invalid_extra = OptionalPerson(name="George", extra=True)
```
## `Required` and `NotRequired`
You can have fine-grained control over field requirements using `Required` and `NotRequired`
qualifiers, which override the class-level `total=` setting:
```py
from typing_extensions import TypedDict, Required, NotRequired
# total=False by default, but id is explicitly Required
class Message(TypedDict, total=False):
id: Required[int] # Always required, even though total=False
content: str # Optional due to total=False
timestamp: NotRequired[str] # Explicitly optional (redundant here)
# total=True by default, but content is explicitly NotRequired
class User(TypedDict):
name: str # Required due to total=True (default)
email: Required[str] # Explicitly required (redundant here)
bio: NotRequired[str] # Optional despite total=True
# Valid Message constructions
msg1 = Message(id=1) # id required, content optional
msg2 = Message(id=2, content="Hello") # both provided
msg3 = Message(id=3, timestamp="2024-01-01") # id required, timestamp optional
# Valid User constructions
user1 = User(name="Alice", email="alice@example.com") # required fields
user2 = User(name="Bob", email="bob@example.com", bio="Developer") # with optional bio
reveal_type(msg1["id"]) # revealed: int
reveal_type(msg1["content"]) # revealed: str
reveal_type(user1["name"]) # revealed: str
reveal_type(user1["bio"]) # revealed: str
```
Constructor validation respects `Required`/`NotRequired` overrides:
```py
# error: [missing-typed-dict-key] "Missing required key 'id' in TypedDict `Message` constructor"
invalid_msg = Message(content="Hello") # Missing required id
# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `User` constructor"
# error: [missing-typed-dict-key] "Missing required key 'email' in TypedDict `User` constructor"
invalid_user = User(bio="No name provided") # Missing required name and email
```
Type validation still applies to all fields when provided:
```py
# error: [invalid-argument-type] "Invalid argument to key "id" with declared type `int` on TypedDict `Message`"
invalid_type = Message(id="not-an-int", content="Hello")
```
## Structural assignability
Assignability between `TypedDict` types is structural, that is, it is based on the presence of keys
@ -134,8 +343,7 @@ alice: Person = {"name": "Alice"}
# TODO: this should be an invalid-assignment error
dangerous(alice)
# TODO: this should be `str`
reveal_type(alice["name"]) # revealed: Unknown
reveal_type(alice["name"]) # revealed: str
```
## Key-based access
@ -224,6 +432,20 @@ def _(person: Person, unknown_key: Any):
person[unknown_key] = "Eve"
```
## `ReadOnly`
`ReadOnly` is not supported yet, but this test makes sure that we do not emit any false positive
diagnostics:
```py
from typing_extensions import TypedDict, ReadOnly, Required
class Person(TypedDict, total=False):
id: ReadOnly[Required[int]]
name: str
age: int | None
```
## Methods on `TypedDict`
```py
@ -340,10 +562,79 @@ class Employee(Person):
alice: Employee = {"name": "Alice", "employee_id": 1}
# TODO: this should be an error (missing required key)
# error: [missing-typed-dict-key] "Missing required key 'employee_id' in TypedDict `Employee` constructor"
eve: Employee = {"name": "Eve"}
```
When inheriting from a `TypedDict` with a different `total` setting, inherited fields maintain their
original requirement status, while new fields follow the child class's `total` setting:
```py
from typing import TypedDict
# Case 1: total=True parent, total=False child
class PersonBase(TypedDict):
id: int # required (from total=True)
name: str # required (from total=True)
class PersonOptional(PersonBase, total=False):
age: int # optional (from total=False)
email: str # optional (from total=False)
# Inherited fields keep their original requirement status
person1 = PersonOptional(id=1, name="Alice") # Valid - id/name still required
person2 = PersonOptional(id=1, name="Alice", age=25) # Valid - age optional
person3 = PersonOptional(id=1, name="Alice", email="alice@test.com") # Valid
# These should be errors - missing required inherited fields
# error: [missing-typed-dict-key] "Missing required key 'id' in TypedDict `PersonOptional` constructor"
person_invalid1 = PersonOptional(name="Bob")
# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `PersonOptional` constructor"
person_invalid2 = PersonOptional(id=2)
# Case 2: total=False parent, total=True child
class PersonBaseOptional(TypedDict, total=False):
id: int # optional (from total=False)
name: str # optional (from total=False)
class PersonRequired(PersonBaseOptional): # total=True by default
age: int # required (from total=True)
# New fields in child are required, inherited fields stay optional
person4 = PersonRequired(age=30) # Valid - only age required, id/name optional
person5 = PersonRequired(id=1, name="Charlie", age=35) # Valid - all provided
# This should be an error - missing required new field
# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `PersonRequired` constructor"
person_invalid3 = PersonRequired(id=3, name="David")
```
This also works with `Required` and `NotRequired`:
```py
from typing_extensions import TypedDict, Required, NotRequired
# Case 3: Mixed inheritance with Required/NotRequired
class PersonMixed(TypedDict, total=False):
id: Required[int] # required despite total=False
name: str # optional due to total=False
class Employee(PersonMixed): # total=True by default
department: str # required due to total=True
# id stays required (Required override), name stays optional, department is required
emp1 = Employee(id=1, department="Engineering") # Valid
emp2 = Employee(id=2, name="Eve", department="Sales") # Valid
# Errors for missing required keys
# error: [missing-typed-dict-key] "Missing required key 'id' in TypedDict `Employee` constructor"
emp_invalid1 = Employee(department="HR")
# error: [missing-typed-dict-key] "Missing required key 'department' in TypedDict `Employee` constructor"
emp_invalid2 = Employee(id=3)
```
## Generic `TypedDict`
`TypedDict`s can also be generic.
@ -362,7 +653,7 @@ class TaggedData(TypedDict, Generic[T]):
p1: TaggedData[int] = {"data": 42, "tag": "number"}
p2: TaggedData[str] = {"data": "Hello", "tag": "text"}
# TODO: this should be an error (type mismatch)
# error: [invalid-argument-type] "Invalid argument to key "data" with declared type `int` on TypedDict `TaggedData`: value of type `Literal["not a number"]`"
p3: TaggedData[int] = {"data": "not a number", "tag": "number"}
```
@ -383,7 +674,7 @@ class TaggedData[T](TypedDict):
p1: TaggedData[int] = {"data": 42, "tag": "number"}
p2: TaggedData[str] = {"data": "Hello", "tag": "text"}
# TODO: this should be an error (type mismatch)
# error: [invalid-argument-type] "Invalid argument to key "data" with declared type `int` on TypedDict `TaggedData`: value of type `Literal["not a number"]`"
p3: TaggedData[int] = {"data": "not a number", "tag": "number"}
```
@ -475,4 +766,54 @@ def write_to_non_literal_string_key(person: Person, str_key: str):
person[str_key] = "Alice" # error: [invalid-key]
```
## Import aliases
`TypedDict` can be imported with aliases and should work correctly:
```py
from typing import TypedDict as TD
from typing_extensions import Required
class UserWithAlias(TD, total=False):
name: Required[str]
age: int
user_empty = UserWithAlias(name="Alice") # name is required
user_partial = UserWithAlias(name="Alice", age=30)
# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `UserWithAlias` constructor"
user_invalid = UserWithAlias(age=30)
reveal_type(user_empty["name"]) # revealed: str
reveal_type(user_partial["age"]) # revealed: int
```
## Shadowing behavior
When a local class shadows the `TypedDict` import, only the actual `TypedDict` import should be
treated as a `TypedDict`:
```py
from typing import TypedDict as TD
class TypedDict:
def __init__(self):
pass
class NotActualTypedDict(TypedDict, total=True):
name: str
class ActualTypedDict(TD, total=True):
name: str
not_td = NotActualTypedDict()
reveal_type(not_td) # revealed: NotActualTypedDict
# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `ActualTypedDict` constructor"
actual_td = ActualTypedDict()
actual_td = ActualTypedDict(name="Alice")
reveal_type(actual_td) # revealed: ActualTypedDict
reveal_type(actual_td["name"]) # revealed: str
```
[`typeddict`]: https://typing.python.org/en/latest/spec/typeddict.html