## Summary
As per our naming scheme (at least for callable types) this should
return a `BoundMethodType`, or be renamed, but it makes more sense to
change the return type.
I also ensure `ClassType.into_callable` returns a `Type::Callable` in
the changed branch.
Ideally we could return a `CallableType` from these `into_callable`
functions (and rename to `into_callable_type` but because of unions we
cannot do this.
## Summary
Validates writes to `TypedDict` keys, for example:
```py
class Person(TypedDict):
name: str
age: int | None
def f(person: Person):
person["naem"] = "Alice" # error: [invalid-key]
person["age"] = "42" # error: [invalid-assignment]
```
The new specialized `invalid-assignment` diagnostic looks like this:
<img width="1160" height="279" alt="image"
src="https://github.com/user-attachments/assets/51259455-3501-4829-a84e-df26ff90bd89"
/>
## Ecosystem analysis
As far as I can tell, all true positives!
There are some extremely long diagnostic messages. We should truncate
our display of overload sets somehow.
## Test Plan
New Markdown tests
## Summary
When seeing a failed test like
```bash
is_subtype_of.md - Subtype relation - Callable - Class literals - Classes with `__new_… (1e9782853227c019)
crates/ty_python_semantic/resources/mdtest/type_properties/is_subtype_of.md:1810 unexpected error: [unresolved-reference] "Name `Aa` used when not defined"
To rerun this specific test, set the environment variable: MDTEST_TEST_FILTER='is_subtype_of.md - Subtype relation - Callable - Class literals - Classes with `__new_… (1e9782853227c019)'
MDTEST_TEST_FILTER='is_subtype_of.md - Subtype relation - Callable - Class literals - Classes with `__new_… (1e9782853227c019)' cargo test -p ty_python_semantic --test mdtest -- mdtest__type_properties_is_subtype_of
```
running the following now works
```bash
MDTEST_TEST_FILTER='is_subtype_of.md - Subtype relation - Callable - Class literals - Classes with `__new_… (1e9782853227c019)' cargo test -p ty_python_semantic --test mdtest -- mdtest__type_properties_is_subtype_of
```
## Test Plan
Do we have tests for the test runner? :)
This fixes our logic for binding a legacy typevar with its binding
context. (To recap, a legacy typevar starts out "unbound" when it is
first created, and each time it's used in a generic class or function,
we "bind" it with the corresponding `Definition`.)
We treat `typing.Self` the same as a legacy typevar, and so we apply
this binding logic to it too. Before, we were using the enclosing class
as its binding context. But that's not correct — it's the method where
`typing.Self` is used that binds the typevar. (Each invocation of the
method will find a new specialization of `Self` based on the specific
instance type containing the invoked method.)
This required plumbing through some additional state to the
`in_type_expression` method.
This also revealed that we weren't handling `Self`-typed instance
attributes correctly (but were coincidentally not getting the expected
false positive diagnostics).
## Summary
Disallow `typing.TypedDict` in type expressions.
Related reference: https://github.com/python/mypy/issues/11030
## Test Plan
New Markdown tests, checked ecosystem and conformance test impact.
## Summary
This PR updates the client settings handling to recognize unknown
options provided by the user and show a warning popup along with a
warning log message.
## Test Plan
Add E2E tests.
## Summary
This PR implements support for providing LSP client settings.
The complementary PR in the ty VS Code extension:
astral-sh/ty-vscode#106.
Notes for the previous iteration of this PR is in
https://github.com/astral-sh/ruff/pull/19614#issuecomment-3136477864
(click on "Details").
Specifically, this PR splits the client settings into 3 distinct groups.
Keep in mind that these groups are not visible to the user, they're
merely an implementation detail. The groups are:
1. `GlobalOptions` - these are the options that are global to the
language server and will be the same for all the workspaces that are
handled by the server
2. `WorkspaceOptions` - these are the options that are specific to a
workspace and will be applied only when running any logic for that
workspace
3. `InitializationOptions` - these are the options that can be specified
during initialization
The initialization options are a superset that contains both the global
and workspace options flattened into a 1-dimensional structure. This
means that the user can specify any and all fields present in
`GlobalOptions` and `WorkspaceOptions` in the initialization options in
addition to the fields that are _specific_ to initialization options.
From the current set of available settings, following are only available
during initialization because they are required at that time, are static
during the runtime of the server and changing their values require a
restart to take effect:
- `logLevel`
- `logFile`
And, following are available under `GlobalOptions`:
- `diagnosticMode`
And, following under `WorkspaceOptions`:
- `disableLanguageServices`
- `pythonExtension` (Python environment information that is populated by
the ty VS Code extension)
### `workspace/configuration`
This request allows server to ask the client for configuration to a
specific workspace. But, this is only supported by the client that has
the `workspace.configuration` client capability set to `true`. What to
do for clients that don't support pulling configurations?
In that case, the settings needs to be provided in the initialization
options and updating the values of those settings can only be done by
restarting the server. With the way this is implemented, this means that
if the client does not support pulling workspace configuration then
there's no way to specify settings specific to a workspace. Earlier,
this would've been possible by providing an array of client options with
an additional field which specifies which workspace the options belong
to but that adds complexity and clients that actually do not support
`workspace/configuration` would usually not support multiple workspaces
either.
Now, for the clients that do support this, the server will initiate the
request to get the configuration for all the workspaces at the start of
the server. Once the server receives these options, it will resolve them
for each workspace as follows:
1. Combine the client options sent during initialization with the
options specific to the workspace creating the final client options
that's specific to this workspace
2. Create a global options by combining the global options from (1) for
all workspaces which in turn will also combine the global options sent
during initialization
The global options are resolved into the global settings and are
available on the `Session` which is initialized with the default global
settings. The workspace options are resolved into the workspace settings
and are available on the respective `Workspace`.
The `SessionSnapshot` contains the global settings while the document
snapshot contains the workspace settings. We could add the global
settings to the document snapshot but that's currently not needed.
### Document diagnostic dynamic registration
Currently, the document diagnostic server capability is created based on
the `diagnosticMode` sent during initialization. But, that wouldn't
provide us with the complete picture. This means the server needs to
defer registering the document diagnostic capability at a later point
once the settings have been resolved.
This is done using dynamic registration for clients that support it. For
clients that do not support dynamic registration for document diagnostic
capability, the server advertises itself as always supporting workspace
diagnostics and work done progress token.
This dynamic registration now allows us to change the server capability
for workspace diagnostics based on the resolved `diagnosticMode` value.
In the future, once `workspace/didChangeConfiguration` is supported, we
can avoid the server restart when users have changed any client
settings.
## Test Plan
Add integration tests and recorded videos on the user experience in
various editors:
### VS Code
For VS Code users, the settings experience is unchanged because the
extension defines it's own interface on how the user can specify the
server setting. This means everything is under the `ty.*` namespace as
usual.
https://github.com/user-attachments/assets/c2e5ba5c-7617-406e-a09d-e397ce9c3b93
### Zed
For Zed, the settings experience has changed. Users can specify settings
during initialization:
```json
{
"lsp": {
"ty": {
"initialization_options": {
"logLevel": "debug",
"logFile": "~/.cache/ty.log",
"diagnosticMode": "workspace",
"disableLanguageServices": true
}
},
}
}
```
Or, can specify the options under the `settings` key:
```json
{
"lsp": {
"ty": {
"settings": {
"ty": {
"diagnosticMode": "openFilesOnly",
"disableLanguageServices": true
}
},
"initialization_options": {
"logLevel": "debug",
"logFile": "~/.cache/ty.log"
}
},
}
}
```
The `logLevel` and `logFile` setting still needs to go under the
initialization options because they're required by the server during
initialization.
We can remove the nesting of the settings under the "ty" namespace by
updating the return type of
db9ea0cdfd/src/tychecker.rs (L45-L49)
to be wrapped inside `ty` directly so that users can avoid doing the
double nesting.
There's one issue here which is that if the `diagnosticMode` is
specified in both the initialization option and settings key, then the
resolution is a bit different - if either of them is set to be
`workspace`, then it wins which means that in the following
configuration, the diagnostic mode is `workspace`:
```json
{
"lsp": {
"ty": {
"settings": {
"ty": {
"diagnosticMode": "openFilesOnly"
}
},
"initialization_options": {
"diagnosticMode": "workspace"
}
},
}
}
```
This behavior is mainly a result of combining global options from
various workspace configuration results. Users should not be able to
provide global options in multiple workspaces but that restriction
cannot be done on the server side. The ty VS Code extension restricts
these global settings to only be set in the user settings and not in
workspace settings but we do not control extensions in other editors.
https://github.com/user-attachments/assets/8e2d6c09-18e6-49e5-ab78-6cf942fe1255
### Neovim
Same as in Zed.
### Other
Other editors that do not support `workspace/configuration`, the users
would need to provide the server settings during initialization.
## Summary
This PR improves the `is_safe_mutable_class` function in `infer.rs` in
several ways:
- It uses `KnownClass::to_instance()` for all "safe mutable classes".
Previously, we were using `SpecialFormType::instance_fallback()` for
some variants -- I'm not totally sure why. Switching to
`KnownClass::to_instance()` for all "safe mutable classes" fixes a
number of TODOs in the `assignment.md` mdtest suite
- Rather than eagerly calling `.to_instance(db)` on all "safe mutable
classes" every time `is_safe_mutable_class` is called, we now only call
it lazily on each element, allowing us to short-circuit more
effectively.
- I removed the entry entirely for `TypedDict` from the list of "safe
mutable classes", as it's not correct.
`SpecialFormType::TypedDict.instance_fallback(db)` just returns an
instance type representing "any instance of `typing._SpecialForm`",
which I don't think was the intent of this code. No tests fail as a
result of removing this entry, as we already check separately whether an
object is an inhabitant of a `TypedDict` type (and consider that object
safe-mutable if so!).
## Test Plan
mdtests updated
## Summary
This PR adds type inference for key-based access on `TypedDict`s and a
new diagnostic for invalid subscript accesses:
```py
class Person(TypedDict):
name: str
age: int | None
alice = Person(name="Alice", age=25)
reveal_type(alice["name"]) # revealed: str
reveal_type(alice["age"]) # revealed: int | None
alice["naem"] # Unknown key "naem" - did you mean "name"?
```
## Test Plan
Updated Markdown tests
## Summary
This PR remaps ranges in Jupyter notebooks from simple `row:column`
indices in the concatenated source code to `cell:row:col` to match
Ruff's output. This is probably not a likely change to land upstream in
`annotate-snippets`, but I didn't see a good way around it.
The remapping logic is taken nearly verbatim from here:
cd6bf1457d/crates/ruff_linter/src/message/text.rs (L212-L222)
## Test Plan
New `full` rendering test for a notebook
I was mainly focused on Ruff, but in local tests this also works for ty:
```
error[invalid-assignment]: Object of type `Literal[1]` is not assignable to `str`
--> Untitled.ipynb:cell 1:3:1
|
1 | import math
2 |
3 | x: str = 1
| ^
|
info: rule `invalid-assignment` is enabled by default
error[invalid-assignment]: Object of type `Literal[1]` is not assignable to `str`
--> Untitled.ipynb:cell 2:3:1
|
1 | import math
2 |
3 | x: str = 1
| ^
|
info: rule `invalid-assignment` is enabled by default
```
This isn't a duplicate diagnostic, just an unimaginative example:
```py
# cell 1
import math
x: str = 1
# cell 2
import math
x: str = 1
```
Summary
--
This is the other commit I wanted to spin off from #19415, currently
stacked on #19644.
This PR suppresses blank snippets for empty ranges at the very beginning
of a file, and for empty ranges in non-existent files. Ruff includes
empty ranges for IO errors, for example.
f4e93b6335/crates/ruff_linter/src/message/text.rs (L100-L110)
The diagnostics now look like this (new snapshot test):
```
error[test-diagnostic]: main diagnostic message
--> example.py:1:1
```
Instead of [^*]
```
error[test-diagnostic]: main diagnostic message
--> example.py:1:1
|
|
```
Test Plan
--
A new `ruff_db` test showing the expected output format
[^*]: This doesn't correspond precisely to the example in the PR because
of some details of the diagnostic builder helper methods in `ruff_db`,
but you can see another example in the current version of the summary in
#19415.
Summary
--
Fixes a snapshot test failure I saw in #19653 locally and in Windows CI
by
padding the hex ID to 16 digits to match the regex in
`filter_result_id`.
78e5fe0a51/crates/ty_server/tests/e2e/pull_diagnostics.rs (L380-L384)
Test Plan
--
I applied this to the branch from #19653 locally and saw that the tests
now
pass. I couldn't reproduce this failure directly on `main` or this
branch,
though.
## Summary
This PR is a spin-off from https://github.com/astral-sh/ruff/pull/19415.
It enables replacing the severity and lint name in a ty-style
diagnostic:
```
error[unused-import]: `os` imported but unused
```
with the noqa code and optional fix availability icon for a Ruff
diagnostic:
```
F401 [*] `os` imported but unused
F821 Undefined name `a`
```
or nothing at all for a Ruff syntax error:
```
SyntaxError: Expected one or more symbol names after import
```
Ruff adds the `SyntaxError` prefix to these messages manually.
Initially (d912458), I just passed a `hide_severity` flag through a
bunch of calls to get it into `annotate-snippets`, but after looking at
it again today, I think reusing the `None` severity/level gave a nicer
result. As I note in a lengthy code comment, I think all of this code
should be temporary and reverted when Ruff gets real severities, so
hopefully it's okay if it feels a little hacky.
I think the main visible downside of this approach is that we can't
style the asterisk in the fix availabilty icon in cyan, as in Ruff's
current output. It's part of the message in this PR and any styling gets
overwritten in `annotate-snippets`.
<img width="400" height="342" alt="image"
src="https://github.com/user-attachments/assets/57542ec9-a81c-4a01-91c7-bd6d7ec99f99"
/>
Hmm, I guess reusing `Level::None` also means the `F401` isn't red
anymore. Maybe my initial approach was better after all. In any case,
the rest of the PR should be basically the same, it just depends how we
want to toggle the severity.
## Test Plan
New `ruff_db` tests. These snapshots should be compared to the two tests
just above them (`hide_severity_output` vs `output` and
`hide_severity_syntax_errors` against `syntax_errors`).
## Summary
This PR fixes a few inaccuracies in attribute access on `TypedDict`s. It
also changes the return type of `type(person)` to `type[dict[str,
object]]` if `person: Person` is an inhabitant of a `TypedDict`
`Person`. We still use `type[Person]` as the *meta type* of Person,
however (see reasoning
[here](https://github.com/astral-sh/ruff/pull/19733#discussion_r2253297926)).
## Test Plan
Updated Markdown tests.
## Summary
This PR adds a new `Type::TypedDict` variant. Before this PR, we treated
`TypedDict`-based types as dynamic Todo-types, and I originally planned
to make this change a no-op. And we do in fact still treat that new
variant similar to a dynamic type when it comes to type properties such
as assignability and subtyping. But then I somehow tricked myself into
implementing some of the things correctly, so here we are. The two main
behavioral changes are: (1) we now also detect generic `TypedDict`s,
which removes a few false positives in the ecosystem, and (2) we now
support *attribute* access (not key-based indexing!) on these types,
i.e. we infer proper types for something like
`MyTypedDict.__required_keys__`. Nothing exciting yet, but gets the
infrastructure into place.
Note that with this PR, the type of (the type) `MyTypedDict` itself is
still represented as a `Type::ClassLiteral` or `Type::GenericAlias` (in
case `MyTypedDict` is generic). Only inhabitants of `MyTypedDict`
(instances of `dict` at runtime) are represented by `Type::TypedDict`.
We may want to revisit this decision in the future, if this turns out to
be too error-prone. Right now, we need to use `.is_typed_dict(db)` in
all the right places to distinguish between actual (generic) classes and
`TypedDict`s. But so far, it seemed unnecessary to add additional `Type`
variants for these as well.
part of https://github.com/astral-sh/ty/issues/154
## Ecosystem impact
The new diagnostics on `cloud-init` look like true positives to me.
## Test Plan
Updated and new Markdown tests
## Summary
This is a follow-up to #19321.
Narrowing constraints introduced in a class scope were not applied even
when they can be applied in lazy nested scopes. This PR fixes so that
they are now applied.
Conversely, there were cases where narrowing constraints were being
applied in places where they should not, so it is also fixed.
## Test Plan
Some TODOs in `narrow/conditionals/nested.md` are now work correctly.
## Summary
This is a follow-up to #19321.
If we try to access a class variable before it is defined, the variable
is looked up in the global scope, rather than in any enclosing scopes.
Closes https://github.com/astral-sh/ty/issues/875.
## Test Plan
New tests in `narrow/conditionals/nested.md`.
## Summary
This PR enhances the `BLE001` rule to correctly detect blind exception
handling in tuple exceptions. Previously, the rule only checked single
exception types, but Python allows catching multiple exceptions using
tuples like `except (Exception, ValueError):`.
## Test Plan
It fails the following (whereas the main branch does not):
```bash
cargo run -p ruff -- check somefile.py --no-cache --select=BLE001
```
```python
# somefile.py
try:
1/0
except (ValueError, Exception) as e:
print(e)
```
```
somefile.py:3:21: BLE001 Do not catch blind exception: `Exception`
|
1 | try:
2 | 1/0
3 | except (ValueError, Exception) as e:
| ^^^^^^^^^ BLE001
4 | print(e)
|
Found 1 error.
```
## Summary
Support `as` patterns in reachability analysis:
```py
from typing import assert_never
def f(subject: str | int):
match subject:
case int() as x:
pass
case str():
pass
case _:
assert_never(subject) # would previously emit an error
```
Note that we still don't support inferring correct types for the bound
name (`x`).
Closes https://github.com/astral-sh/ty/issues/928
## Test Plan
New Markdown tests
## Summary
This PR reduces the virality of some of the `Todo` types in
`infer_tuple_type_expression`. Rather than inferring `Todo`, we instead
infer `tuple[Todo, ...]`. This reflects the fact that whatever the
contents of the slice in a `tuple[]` type expression, we would always
infer some kind of tuple type as the result of the type expression. Any
tuple type should be assignable to `tuple[Todo, ...]`, so this shouldn't
introduce any new false positives; this can be seen in the ecosystem
report.
As a result of the change, we are now able to enforce in the signature
of `Type::infer_tuple_type_expression` that it returns an
`Option<TupleType<'db>>`, which is more strongly typed and expresses
clearly the invariant that a tuple type expression should always be
inferred as a `tuple` type. To enable this, it was necessary to refactor
several `TupleType` constructors in `tuple.rs` so that they return
`Option<TupleType>` rather than `Type`; this means that callers of these
constructor functions are now free to either propagate the
`Option<TupleType<'db>>` or convert it to a `Type<'db>`.
## Test Plan
Mdtests updated.
## Summary
When splitting triple-quoted, raw strings one has to take care before attempting to make each item have single-quotes.
Fixes#19577
---------
Co-authored-by: dylwil3 <dylwil3@gmail.com>
This is subtle, and the root cause became more apparent with #19604,
since we now have many more cases of superclasses and subclasses using
different typevars. The issue is easiest to see in the following:
```py
class C[T]:
def __init__(self, t: T) -> None: ...
class D[U](C[T]):
pass
reveal_type(C(1)) # revealed: C[int]
reveal_type(D(1)) # should be: D[int]
```
When instantiating a generic class, the `__init__` method inherits the
generic context of that class. This lets our call binding machinery
infer a specialization for that context.
Prior to this PR, the instantiation of `C` worked just fine. Its
`__init__` method would inherit the `[T]` generic context, and we would
infer `{T = int}` as the specialization based on the argument
parameters.
It didn't work for `D`. The issue is that the `__init__` method was
inheriting the generic context of the class where `__init__` was defined
(here, `C` and `[T]`). At the call site, we would then infer `{T = int}`
as the specialization — but that wouldn't help us specialize `D[U]`,
since `D` does not have `T` in its generic context!
Instead, the `__init__` method should inherit the generic context of the
class that we are performing the lookup on (here, `D` and `[U]`). That
lets us correctly infer `{U = int}` as the specialization, which we can
successfully apply to `D[U]`.
(Note that `__init__` refers to `C`'s typevars in its signature, but
that's okay; our member lookup logic already applies the `T = U`
specialization when returning a member of `C` while performing a lookup
on `D`, transforming its signature from `(Self, T) -> None` to `(Self,
U) -> None`.)
Closes https://github.com/astral-sh/ty/issues/588
This PR introduces a few related changes:
- We now keep track of each time a legacy typevar is bound in a
different generic context (e.g. class, function), and internally create
a new `TypeVarInstance` for each usage. This means the rest of the code
can now assume that salsa-equivalent `TypeVarInstance`s refer to the
same typevar, even taking into account that legacy typevars can be used
more than once.
- We also go ahead and track the binding context of PEP 695 typevars.
That's _much_ easier to track since we have the binding context right
there during type inference.
- With that in place, we can now include the name of the binding context
when rendering typevars (e.g. `T@f` instead of `T`)
## Summary
Adds an initial set of tests based on the highest-priority items in
https://github.com/astral-sh/ty/issues/154. This is certainly not yet
exhaustive (required/non-required, `total`, and other things are
missing), but will be useful to measure progress on this feature.
## Test Plan
Checked intended behavior against runtime and other type checkers.
