Commit graph

48 commits

Author SHA1 Message Date
Micha Reiser
6aaf1d9446
[red-knot] Remove lint-phase (#13922)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-25 18:40:52 +00:00
David Peter
f335fe4d4a
[red-knot] rename {Class,Module,Function} => {Class,Module,Function}Literal (#13873)
## Summary

* Rename `Type::Class` => `Type::ClassLiteral`
* Rename `Type::Function` => `Type::FunctionLiteral`
* Do not rename `Type::Module`
* Remove `*Literal` suffixes in `display::LiteralTypeKind` variants, as
per clippy suggestion
* Get rid of `Type::is_class()` in favor of `is_subtype_of(…, 'type')`;
modifiy `is_subtype_of` to support this.
* Add new `Type::is_xyz()` methods and use them instead of matching on
`Type` variants.

closes #13863 

## Test Plan

New `is_subtype_of_class_literals` unit test.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-22 22:10:53 +02:00
Dhruv Manilawala
b16f665a81
[red-knot] Infer target types for unpacked tuple assignment (#13316)
## Summary

This PR adds support for unpacking tuple expression in an assignment
statement where the target expression can be a tuple or a list (the
allowed sequence targets).

The implementation introduces a new `infer_assignment_target` which can
then be used for other targets like the ones in for loops as well. This
delegates it to the `infer_definition`. The final implementation uses a
recursive function that visits the target expression in source order and
compares the variable node that corresponds to the definition. At the
same time, it keeps track of where it is on the assignment value type.

The logic also accounts for the number of elements on both sides such
that it matches even if there's a gap in between. For example, if
there's a starred expression like `(a, *b, c) = (1, 2, 3)`, then the
type of `a` will be `Literal[1]` and the type of `b` will be
`Literal[2]`.

There are a couple of follow-ups that can be done:
* Use this logic for other target positions like `for` loop
* Add diagnostics for mis-match length between LHS and RHS

## Test Plan

Add various test cases using the new markdown test framework.
Validate that existing test cases pass.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-15 19:07:11 +00:00
Micha Reiser
5f65e842e8
Upgrade salsa (#13757) 2024-10-15 11:06:32 +00:00
Charlie Marsh
c3b40da0d2
Use backticks for code in red-knot messages (#13599)
## Summary

...and remove periods from messages that don't span more than a single
sentence.

This is more consistent with how we present user-facing messages in uv
(which has a defined style guide).
2024-10-02 03:14:28 +00:00
Alex Waygood
82324678cf
Rename the ruff_vendored crate to red_knot_vendored (#13586) 2024-10-01 16:16:59 +01:00
Micha Reiser
653c09001a
Use an empty vendored file system in Ruff (#13436)
## Summary

This PR changes removes the typeshed stubs from the vendored file system
shipped with ruff
and instead ships an empty "typeshed".

Making the typeshed files optional required extracting the typshed files
into a new `ruff_vendored` crate. I do like this even if all our builds
always include typeshed because it means `red_knot_python_semantic`
contains less code that needs compiling.

This also allows us to use deflate because the compression algorithm
doesn't matter for an archive containing a single, empty file.

## Test Plan

`cargo test`

I verified with ` cargo tree -f "{p} {f}" -p <package> ` that:

* red_knot_wasm: enables `deflate` compression
* red_knot: enables `zstd` compression
* `ruff`: uses stored


I'm not quiet sure how to build the binary that maturin builds but
comparing the release artifact size with `strip = true` shows a `1.5MB`
size reduction

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-09-21 16:31:42 +00:00
Carl Meyer
149fb2090e
[red-knot] more efficient UnionBuilder::add (#13411)
Avoid quadratic time in subsumed elements when adding a super-type of
existing union elements.

Reserve space in advance when adding multiple elements (from another
union) to a union.

Make union elements a `Box<[Type]>` instead of an `FxOrderSet`; the set
doesn't buy much since the rules of union uniqueness are defined in
terms of supertype/subtype, not in terms of simple type identity.

Move sealed-boolean handling out of a separate `UnionBuilder::simplify`
method and into `UnionBuilder::add`; now that `add` is iterating
existing elements anyway, this is more efficient.

Remove `UnionType::contains`, since it's now `O(n)` and we shouldn't
really need it, generally we care about subtype/supertype, not type
identity. (Right now it's used for `Type::Unbound`, which shouldn't even
be a type.)

Add support for `is_subtype_of` for the `object` type.

Addresses comments on https://github.com/astral-sh/ruff/pull/13401
2024-09-20 10:49:45 -07:00
Carl Meyer
260c2ecd15
[red-knot] visit with-item vars even if not a Name (#13409)
This fixes the last panic on checking pandas.

(Match statement became an `if let` because clippy decided it wanted
that once I added the additional line in the else case?)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-09-19 10:37:49 -07:00
Carl Meyer
29c36a56b2
[red-knot] fix scope inference with deferred types (#13204)
Test coverage for #13131 wasn't as good as I thought it was, because
although we infer a lot of types in stubs in typeshed, we don't check
typeshed, and therefore we don't do scope-level inference and pull all
types for a scope. So we didn't really have good test coverage for
scope-level inference in a stub. And because of this, I got the code for
supporting that wrong, meaning that if we did scope-level inference with
deferred types, we'd end up never populating the deferred types in the
scope's `TypeInference`, which causes panics like #13160.

Here I both add test coverage by running the corpus tests both as `.py`
and as `.pyi` (which reveals the panic), and I fix the code to support
deferred types in scope inference.

This also revealed a problem with deferred types in generic functions,
which effectively span two scopes. That problem will require a bit more
thought, and I don't want to block this PR on it, so for now I just
don't defer annotations on generic functions.

Fixes #13160.
2024-09-03 11:20:43 -07:00
Micha Reiser
599103c933
Add a few missing #[return_ref] attributes (#13223) 2024-09-03 09:15:43 +02:00
Micha Reiser
ecab04e338
Basic concurrent checking (#13049) 2024-08-24 09:53:27 +01:00
Dylan
8c09496b07
[red-knot] Resolve function annotations before adding function symbol (#13084)
This PR has the `SemanticIndexBuilder` visit function definition
annotations before adding the function symbol/name to the builder.

For example, the following snippet no longer causes a panic:

```python
def bool(x) -> bool:
    Return True
```

Note: This fix changes the ordering of the global symbol table.

Closes #13069
2024-08-23 19:31:36 -07:00
Dhruv Manilawala
cfe25ab465
[red-knot] Support untitled files in the server (#13044)
## Summary

This PR adds support for untitled files in the red knot server.

## Test Plan

https://github.com/user-attachments/assets/57fa5db6-e1ad-4694-ae5f-c47a21eaa82b
2024-08-23 12:47:35 +05:30
Dhruv Manilawala
21c5606793
[red-knot] Support textDocument/didChange notification (#13042)
## Summary

This PR adds support for `textDocument/didChange` notification.

There seems to be a bug (probably in Salsa) where it panics with:
```
2024-08-22 15:33:38.802 [info] panicked at /Users/dhruv/.cargo/git/checkouts/salsa-61760caba2b17ca5/f608ff8/src/tracked_struct.rs:377:9:
two concurrent writers to Id(4800), should not be possible
```

## Test Plan


https://github.com/user-attachments/assets/81055feb-ba8e-4acf-ad2f-94084a3efead
2024-08-23 06:58:54 +00:00
Dhruv Manilawala
c73a7bb929
[red-knot] Support files outside of any workspace (#13041)
## Summary

This PR adds basic support for files outside of any workspace in the red
knot server.

This also limits the red knot server to only work in a single workspace.
The server will not start if there are multiple workspaces.

## Test Plan

https://github.com/user-attachments/assets/de601387-0ad5-433c-9d2c-7b6ae5137654
2024-08-23 06:51:48 +00:00
Micha Reiser
1ca14e4335
Move collection of parse errors to check_file (#13059) 2024-08-23 08:22:12 +02:00
Dylan
2edd32aa31
[red-knot] SemanticIndexBuilder visits value before target in named expressions (#13053)
The `SemanticIndexBuilder` was causing a cycle in a salsa query by
attempting to resolve the target before the value in a named expression
(e.g. `x := x+1`). This PR swaps the order, avoiding a panic.

Closes #13012.
2024-08-22 07:59:13 -07:00
Micha Reiser
dce87c21fd
Eagerly validate typeshed versions (#12786) 2024-08-21 15:49:53 +00:00
Micha Reiser
5c5dfc11f0
Upgrade to Salsa with tables (#13016) 2024-08-21 06:58:53 +00:00
Alex Waygood
37a60460ed
[red-knot] Improve various tracing logs (#13015) 2024-08-20 18:34:51 +00:00
Micha Reiser
c65e3310d5
Add API to emit type-checking diagnostics (#12988)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-08-20 07:22:30 +00:00
Micha Reiser
38c19fb96e
Fix re-entrance deadlock in Package::files (#12948) 2024-08-20 06:51:08 +00:00
Alex Waygood
e6d5a7af37
Add the testing feature of ruff_db as a dev-dependency for ruff_workspace (#12985) 2024-08-19 10:22:01 +00:00
Micha Reiser
dd0a7ec73e
Pull all types in corpus tests (#12919) 2024-08-17 11:59:55 +00:00
Alex Waygood
a9847af6e8
[red-knot] Use Unknown rather than Unbound for unresolved imports (#12932) 2024-08-16 20:10:33 +01:00
Dhruv Manilawala
4b9ddc4a06
[red-knot] Use Windows specific path separator in tests (#12847) 2024-08-12 22:26:59 +05:30
Dhruv Manilawala
99dc208b00
[red-knot] Add filename and source location for diagnostics (#12842)
## Summary

I'm not sure if this is useful but this is a hacky implementation to add
the filename and row / column numbers to the current Red Knot
diagnostics.
2024-08-12 15:56:30 +00:00
Micha Reiser
a99a45868c
Eagerly validate search paths (#12783)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-08-12 07:46:59 +00:00
Micha Reiser
fabf19fdc9
Skip checking a file if it failed to read (#12755) 2024-08-12 07:26:37 +00:00
Alex Waygood
cf1a57df5a
Remove red_knot_python_semantic::python_version::TargetVersion (#12790) 2024-08-10 14:28:31 +01:00
Alex Waygood
37b9bac403
[red-knot] Add support for --system-site-packages virtual environments (#12759) 2024-08-09 21:02:16 +01:00
Micha Reiser
2abfab0f9b
Move Program and related structs to red_knot_python_semantic (#12777) 2024-08-09 11:50:45 +02:00
Micha Reiser
ffaa35eafe
Add test helper to setup tracing (#12741) 2024-08-09 07:04:04 +00:00
Alex Waygood
f1de08c2a0
[red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
Micha Reiser
2daa914334
Gracefully handle errors in CLI (#12747) 2024-08-08 11:02:47 +00:00
Micha Reiser
dc6aafecc2
Setup tracing and document tracing usage (#12730) 2024-08-08 06:28:40 +00:00
Alex Waygood
f34b9a77f0
[red-knot] Cleanups to logic resolving site-packages from a venv path (#12731) 2024-08-07 15:48:15 +01:00
Alex Waygood
aae9619d3d
[red-knot] Fix build on Windows (#12719)
## Summary

Tests are failing on `main` because automerge landed
https://github.com/astral-sh/ruff/pull/12716 despite the Windows tests
failing.
2024-08-06 20:21:25 +01:00
Alex Waygood
7fa76a2b2b
[red-knot] Derive site-packages from a venv path (#12716) 2024-08-06 18:34:37 +00:00
Micha Reiser
846f57fd15
Update salsa (#12711) 2024-08-06 13:17:39 +00:00
Dhruv Manilawala
e91a0fe94a
[red-knot] Implement basic LSP server (#12624)
## Summary

This PR adds basic LSP implementation for the Red Knot project.

This is basically a fork of the existing `ruff_server` crate into a
`red_knot_server` crate. The following are the main differences:
1. The `Session` stores a map from workspace root to the corresponding
Red Knot database (`RootDatabase`).
2. The database is initialized with the newly implemented `LSPSystem`
(implementation of `System` trait)
3. The `LSPSystem` contains the server index corresponding to each
workspace and an underlying OS system implementation. For certain
methods, the system first checks if there's an open document in LSP
system and returns the information from that. Otherwise, it falls back
to the OS system to get that information. These methods are
`path_metadata`, `read_to_string` and `read_to_notebook`
4. Add `as_any_mut` method for `System`

**Why fork?**

Forking allows us to experiment with the functionalities that are
specific to Red Knot. The architecture is completely different and so
the requirements for an LSP implementation are different as well. For
example, Red Knot only supports a single workspace, so the LSP system
needs to map the multi-workspace support to each Red Knot instance. In
the end, the server code isn't too big, it will be easier to implement
Red Knot specific functionality without worrying about existing server
limitations and it shouldn't be difficult to port the existing server.

## Review

Most of the server files hasn't been changed. I'm going to list down the
files that have been changed along with highlight the specific part of
the file that's changed from the existing server code.

Changed files:
* Red Knot CLI implementation:
https://github.com/astral-sh/ruff/pull/12624/files#diff-579596339a29d3212a641232e674778c339b446de33b890c7fdad905b5eb50e1
* In
https://github.com/astral-sh/ruff/pull/12624/files#diff-b9a9041a8a2bace014bf3687c3ef0512f25e0541f112fad6131b14242f408db6,
server capabilities have been updated, dynamic capability registration
is removed
* In
https://github.com/astral-sh/ruff/pull/12624/files#diff-b9a9041a8a2bace014bf3687c3ef0512f25e0541f112fad6131b14242f408db6,
the API for `clear_diagnostics` now take in a `Url` instead of
`DocumentQuery` as the document version doesn't matter when clearing
diagnostics after a document is closed
*
[`did_close`](https://github.com/astral-sh/ruff/pull/12624/files#diff-9271370102a6f3be8defaca40c82485b0048731942520b491a3bdd2ee0e25493),
[`did_close_notebook`](https://github.com/astral-sh/ruff/pull/12624/files#diff-96fb53ffb12c1694356e17313e4bb37b3f0931e887878b5d7c896c19ff60283b),
[`did_open`](https://github.com/astral-sh/ruff/pull/12624/files#diff-60e852cf1aa771e993131cabf98eb4c467963a8328f10eccdb43b3e8f0f1fb12),
[`did_open_notebook`](https://github.com/astral-sh/ruff/pull/12624/files#diff-ac356eb5e36c3b2c1c135eda9dfbcab5c12574d1cb77c71f7da8dbcfcfb2d2f1)
are updated to open / close file from the corresponding Red Knot
workspace
* The [diagnostic
handler](https://github.com/astral-sh/ruff/pull/12624/files#diff-4475f318fd0290d0292834569a7df5699debdcc0a453b411b8c3d329f1b879d9)
is updated to request diagnostics from Red Knot
* The [`Session::new`] method in
https://github.com/astral-sh/ruff/pull/12624/files#diff-55c96201296200c1cab37c8b0407b6c733381374b94be7ae50563bfe95264e4d
is updated to construct the Red Knot databases for each workspace. It
also contains the `index_mut` and `MutIndexGuard` implementation
* And, `LSPSystem` implementation is in
https://github.com/astral-sh/ruff/pull/12624/files#diff-4ed62bd359c43b0bf1a13f04349dcd954966934bb8d544de7813f974182b489e

## Test Plan

First, configure VS Code to use the `red_knot` binary

1. Build the `red_knot` binary by `cargo build`
2. Update the VS Code extension to specify the path to this binary
```json
{
	"ruff.path": ["/path/to/ruff/target/debug/red_knot"]
}
```
3. Restart VS Code

Now, open a file containing red-knot specific diagnostics, close the
file and validate that diagnostics disappear.
2024-08-06 11:27:30 +00:00
Alex Waygood
5499821c67
[red-knot] Rename workspace_root variables in the module resolver to src_root (#12697)
Fixes #12337
2024-08-05 23:07:18 +01:00
Micha Reiser
341a25eec1
Fix file watching on macOS if a module-search path is a symlink (#12634) 2024-08-03 07:24:07 +00:00
Micha Reiser
12177a42e3
Set durabilities for low-durability fields on high-durability inputs (#12627) 2024-08-02 19:42:34 +02:00
Micha Reiser
dfb08856eb
Fix file watcher stop data race (#12626) 2024-08-02 19:02:49 +02:00
Alex Waygood
fbab04fbe1
[red-knot] Allow multiple site-packages search paths (#12609) 2024-08-02 13:33:19 +00:00
Dhruv Manilawala
9aa43d5f91
Separate red_knot into CLI and red_knot_workspace crates (#12623)
## Summary

This PR separates the current `red_knot` crate into two crates:
1. `red_knot` - This will be similar to the `ruff` crate, it'll act as
the CLI crate
2. `red_knot_workspace` - This includes everything except for the CLI
functionality from the existing `red_knot` crate

Note that the code related to the file watcher is in
`red_knot_workspace` for now but might be required to extract it out in
the future.

The main motivation for this change is so that we can have a `red_knot
server` command. This makes it easier to test the server out without
making any changes in the VS Code extension. All we need is to specify
the `red_knot` executable path in `ruff.path` extension setting.

## Test Plan

- `cargo build`
- `cargo clippy --workspace --all-targets --all-features`
- `cargo shear --fix`
2024-08-02 11:24:36 +00:00