Commit graph

182 commits

Author SHA1 Message Date
renovate[bot]
5a67d3269b
Update pre-commit dependencies (#14719) 2024-12-02 06:02:56 +00:00
Micha Reiser
e53ac7985d
Enable logging for directory-renamed test (#14533) 2024-11-22 16:41:46 +00:00
Micha Reiser
1f07880d5c
Add tests for python version compatibility (#14430) 2024-11-18 12:26:55 +00:00
Micha Reiser
81e5830585
Workspace discovery (#14308) 2024-11-15 19:20:15 +01:00
Micha Reiser
59c0dacea0
Introduce Diagnostic trait (#14130) 2024-11-07 13:26:21 +01:00
Micha Reiser
32b57b2ee4
Enable nursery rules: 'redundant_clone', 'debug_assert_with_mut_call', and 'unused_peekable' (#13920) 2024-10-25 09:46:30 +02:00
Micha Reiser
5f65e842e8
Upgrade salsa (#13757) 2024-10-15 11:06:32 +00:00
Charlie Marsh
c3b40da0d2
Use backticks for code in red-knot messages (#13599)
## Summary

...and remove periods from messages that don't span more than a single
sentence.

This is more consistent with how we present user-facing messages in uv
(which has a defined style guide).
2024-10-02 03:14:28 +00:00
Micha Reiser
653c09001a
Use an empty vendored file system in Ruff (#13436)
## Summary

This PR changes removes the typeshed stubs from the vendored file system
shipped with ruff
and instead ships an empty "typeshed".

Making the typeshed files optional required extracting the typshed files
into a new `ruff_vendored` crate. I do like this even if all our builds
always include typeshed because it means `red_knot_python_semantic`
contains less code that needs compiling.

This also allows us to use deflate because the compression algorithm
doesn't matter for an archive containing a single, empty file.

## Test Plan

`cargo test`

I verified with ` cargo tree -f "{p} {f}" -p <package> ` that:

* red_knot_wasm: enables `deflate` compression
* red_knot: enables `zstd` compression
* `ruff`: uses stored


I'm not quiet sure how to build the binary that maturin builds but
comparing the release artifact size with `strip = true` shows a `1.5MB`
size reduction

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-09-21 16:31:42 +00:00
Alex Waygood
c6023c03a2
[red-knot] Add docs on using RAYON_NUM_THREADS for better logging (#13140)
Followup to #13049. We check files concurrently now; to get readable
logs, you probably want to switch that off
2024-08-28 17:14:56 +01:00
Micha Reiser
dce87c21fd
Eagerly validate typeshed versions (#12786) 2024-08-21 15:49:53 +00:00
Micha Reiser
e5f37a8254
Remove linter dependency from red_knot_server (#13028) 2024-08-21 10:02:42 +00:00
Micha Reiser
38c19fb96e
Fix re-entrance deadlock in Package::files (#12948) 2024-08-20 06:51:08 +00:00
Micha Reiser
a99a45868c
Eagerly validate search paths (#12783)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-08-12 07:46:59 +00:00
Alex Waygood
cf1a57df5a
Remove red_knot_python_semantic::python_version::TargetVersion (#12790) 2024-08-10 14:28:31 +01:00
Alex Waygood
37b9bac403
[red-knot] Add support for --system-site-packages virtual environments (#12759) 2024-08-09 21:02:16 +01:00
Micha Reiser
2abfab0f9b
Move Program and related structs to red_knot_python_semantic (#12777) 2024-08-09 11:50:45 +02:00
Micha Reiser
ffaa35eafe
Add test helper to setup tracing (#12741) 2024-08-09 07:04:04 +00:00
Alex Waygood
d28c5afd14
[red-knot] Remove mentions of Ruff from the CLI help (#12752)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-08-08 15:35:10 +01:00
Alex Waygood
f1de08c2a0
[red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
Micha Reiser
2daa914334
Gracefully handle errors in CLI (#12747) 2024-08-08 11:02:47 +00:00
Micha Reiser
df7345e118
Exit with an error if there are check failures (#12735) 2024-08-08 07:10:18 +00:00
Micha Reiser
dc6aafecc2
Setup tracing and document tracing usage (#12730) 2024-08-08 06:28:40 +00:00
Alex Waygood
7fa76a2b2b
[red-knot] Derive site-packages from a venv path (#12716) 2024-08-06 18:34:37 +00:00
Dhruv Manilawala
14dd6d980e
[red-knot] Keep subcommands optional for the binary (#12715)
## Summary

This PR updates the `red_knot` CLI to make the subcommand optional.

## Test Plan

Run the following commands:
* `cargo run --bin red_knot --
--current-directory=~/playground/ruff/type_inference` (no subcommand
requirement)
* `cargo run --bin red_knot -- server` (should start the server)
2024-08-06 20:24:49 +05:30
Micha Reiser
846f57fd15
Update salsa (#12711) 2024-08-06 13:17:39 +00:00
Micha Reiser
8e6aa78796
Remove 'cli' module from red_knot (#12714) 2024-08-06 12:10:36 +00:00
Dhruv Manilawala
e91a0fe94a
[red-knot] Implement basic LSP server (#12624)
## Summary

This PR adds basic LSP implementation for the Red Knot project.

This is basically a fork of the existing `ruff_server` crate into a
`red_knot_server` crate. The following are the main differences:
1. The `Session` stores a map from workspace root to the corresponding
Red Knot database (`RootDatabase`).
2. The database is initialized with the newly implemented `LSPSystem`
(implementation of `System` trait)
3. The `LSPSystem` contains the server index corresponding to each
workspace and an underlying OS system implementation. For certain
methods, the system first checks if there's an open document in LSP
system and returns the information from that. Otherwise, it falls back
to the OS system to get that information. These methods are
`path_metadata`, `read_to_string` and `read_to_notebook`
4. Add `as_any_mut` method for `System`

**Why fork?**

Forking allows us to experiment with the functionalities that are
specific to Red Knot. The architecture is completely different and so
the requirements for an LSP implementation are different as well. For
example, Red Knot only supports a single workspace, so the LSP system
needs to map the multi-workspace support to each Red Knot instance. In
the end, the server code isn't too big, it will be easier to implement
Red Knot specific functionality without worrying about existing server
limitations and it shouldn't be difficult to port the existing server.

## Review

Most of the server files hasn't been changed. I'm going to list down the
files that have been changed along with highlight the specific part of
the file that's changed from the existing server code.

Changed files:
* Red Knot CLI implementation:
https://github.com/astral-sh/ruff/pull/12624/files#diff-579596339a29d3212a641232e674778c339b446de33b890c7fdad905b5eb50e1
* In
https://github.com/astral-sh/ruff/pull/12624/files#diff-b9a9041a8a2bace014bf3687c3ef0512f25e0541f112fad6131b14242f408db6,
server capabilities have been updated, dynamic capability registration
is removed
* In
https://github.com/astral-sh/ruff/pull/12624/files#diff-b9a9041a8a2bace014bf3687c3ef0512f25e0541f112fad6131b14242f408db6,
the API for `clear_diagnostics` now take in a `Url` instead of
`DocumentQuery` as the document version doesn't matter when clearing
diagnostics after a document is closed
*
[`did_close`](https://github.com/astral-sh/ruff/pull/12624/files#diff-9271370102a6f3be8defaca40c82485b0048731942520b491a3bdd2ee0e25493),
[`did_close_notebook`](https://github.com/astral-sh/ruff/pull/12624/files#diff-96fb53ffb12c1694356e17313e4bb37b3f0931e887878b5d7c896c19ff60283b),
[`did_open`](https://github.com/astral-sh/ruff/pull/12624/files#diff-60e852cf1aa771e993131cabf98eb4c467963a8328f10eccdb43b3e8f0f1fb12),
[`did_open_notebook`](https://github.com/astral-sh/ruff/pull/12624/files#diff-ac356eb5e36c3b2c1c135eda9dfbcab5c12574d1cb77c71f7da8dbcfcfb2d2f1)
are updated to open / close file from the corresponding Red Knot
workspace
* The [diagnostic
handler](https://github.com/astral-sh/ruff/pull/12624/files#diff-4475f318fd0290d0292834569a7df5699debdcc0a453b411b8c3d329f1b879d9)
is updated to request diagnostics from Red Knot
* The [`Session::new`] method in
https://github.com/astral-sh/ruff/pull/12624/files#diff-55c96201296200c1cab37c8b0407b6c733381374b94be7ae50563bfe95264e4d
is updated to construct the Red Knot databases for each workspace. It
also contains the `index_mut` and `MutIndexGuard` implementation
* And, `LSPSystem` implementation is in
https://github.com/astral-sh/ruff/pull/12624/files#diff-4ed62bd359c43b0bf1a13f04349dcd954966934bb8d544de7813f974182b489e

## Test Plan

First, configure VS Code to use the `red_knot` binary

1. Build the `red_knot` binary by `cargo build`
2. Update the VS Code extension to specify the path to this binary
```json
{
	"ruff.path": ["/path/to/ruff/target/debug/red_knot"]
}
```
3. Restart VS Code

Now, open a file containing red-knot specific diagnostics, close the
file and validate that diagnostics disappear.
2024-08-06 11:27:30 +00:00
Alex Waygood
5499821c67
[red-knot] Rename workspace_root variables in the module resolver to src_root (#12697)
Fixes #12337
2024-08-05 23:07:18 +01:00
Micha Reiser
341a25eec1
Fix file watching on macOS if a module-search path is a symlink (#12634) 2024-08-03 07:24:07 +00:00
Alex Waygood
fbab04fbe1
[red-knot] Allow multiple site-packages search paths (#12609) 2024-08-02 13:33:19 +00:00
Dhruv Manilawala
9aa43d5f91
Separate red_knot into CLI and red_knot_workspace crates (#12623)
## Summary

This PR separates the current `red_knot` crate into two crates:
1. `red_knot` - This will be similar to the `ruff` crate, it'll act as
the CLI crate
2. `red_knot_workspace` - This includes everything except for the CLI
functionality from the existing `red_knot` crate

Note that the code related to the file watcher is in
`red_knot_workspace` for now but might be required to extract it out in
the future.

The main motivation for this change is so that we can have a `red_knot
server` command. This makes it easier to test the server out without
making any changes in the VS Code extension. All we need is to specify
the `red_knot` executable path in `ruff.path` extension setting.

## Test Plan

- `cargo build`
- `cargo clippy --workspace --all-targets --all-features`
- `cargo shear --fix`
2024-08-02 11:24:36 +00:00
Micha Reiser
966563c79b
Add tests for hard and soft links (#12590) 2024-08-02 10:14:28 +00:00
Micha Reiser
18f87b9497
Flaky file watching tests, add debug assertions (#12587) 2024-07-30 18:09:55 +00:00
Micha Reiser
adc8d4e1e7
File watch events: Add dynamic wait period before writing new changes (#12585) 2024-07-30 19:18:43 +02:00
Micha Reiser
264cd750e9
Add delay between updating a file (#12576) 2024-07-30 18:31:29 +02:00
Micha Reiser
a2286c8e47
Set Durability to 'HIGH' for most inputs and third-party libraries (#12566) 2024-07-30 09:03:59 +00:00
Micha Reiser
e18b4e42d3
[red-knot] Upgrade to the *new* *new* salsa (#12406) 2024-07-29 07:21:24 +00:00
Micha Reiser
bf23d38a21
Remove unnecessary clone in workspace API (#12529) 2024-07-26 17:19:05 +02:00
Carl Meyer
10c993e21a
[red-knot] remove wrong __init__.py from file-watching tests (#12519) 2024-07-26 07:14:01 +01:00
Carl Meyer
2d3914296d
[red-knot] handle all syntax without panic (#12499)
Extend red-knot type inference to cover all syntax, so that inferring
types for a scope gives all expressions a type. This means we can run
the red-knot semantic lint on all Python code without panics. It also
means we can infer types for `builtins.pyi` without panics.

To keep things simple, this PR intentionally doesn't add any new type
inference capabilities: the expanded coverage is all achieved with
`Type::Unknown`. But this puts the skeleton in place for adding better
inference of all these language features.

I also had to add basic Salsa cycle recovery (with just `Type::Unknown`
for now), because some `builtins.pyi` definitions are cyclic.

To test this, I added a comprehensive corpus of test snippets sourced
from Cinder under [MIT
license](https://github.com/facebookincubator/cinder/blob/cinder/3.10/cinderx/LICENSE),
which matches Ruff's license. I also added to this corpus some
additional snippets for newer language features: all the
`27_func_generic_*` and `73_class_generic_*` files, as well as
`20_lambda_default_arg.py`, and added a test which runs semantic-lint
over all these files. (The test doesn't assert the test-corpus files are
lint-free; just that they are able to lint without a panic.)
2024-07-25 17:38:08 -07:00
Micha Reiser
2ce3e3ae60
Fix the search path tests on MacOS (#12503) 2024-07-25 08:21:38 +02:00
Micha Reiser
eac965ecaf
[red-knot] Watch search paths (#12407) 2024-07-24 07:38:50 +00:00
Micha Reiser
f0fc6a95fe
[red-knot] Lazy package file discovery (#12452)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-07-23 08:47:15 +00:00
Micha Reiser
b9b7deff17
Implement upcast_mut for new TestDb (#12470) 2024-07-23 07:11:00 +00:00
Micha Reiser
40d9324f5a
[red-knot] Improved file watching (#12382) 2024-07-23 08:18:59 +02:00
Carl Meyer
f22c8ab811
[red-knot] add maybe-undefined lint rule (#12414)
Add a lint rule to detect if a name is definitely or possibly undefined
at a given usage.

If I create the file `undef/main.py` with contents:

```python
x = int
def foo():
    z
    return x
if flag:
    y = x
y
```

And then run `cargo run --bin red_knot -- --current-directory
../ruff-examples/undef`, I get the output:

```
Name 'z' used when not defined.
Name 'flag' used when not defined.
Name 'y' used when possibly not defined.
```

If I modify the file to add `y = 0` at the top, red-knot re-checks it
and I get the new output:

```
Name 'z' used when not defined.
Name 'flag' used when not defined.
```

Note that `int` is not flagged, since it's a builtin, and `return x` in
the function scope is not flagged, since it refers to the global `x`.
2024-07-22 13:53:59 -07:00
Micha Reiser
ad19b3fd0e
[red-knot] Add verbosity argument to CLI (#12404) 2024-07-19 11:38:24 +00:00
Carl Meyer
181e7b3c0d
[red-knot] rename module_global to global (#12385)
Per comments in https://github.com/astral-sh/ruff/pull/12269, "module
global" is kind of long, and arguably redundant.

I tried just using "module" but there were too many cases where I felt
this was ambiguous. I like the way "global" works out better, though it
does require an understanding that in Python "global" generally means
"module global" not "globally global" (though in a sense module globals
are also globally global since modules are singletons).
2024-07-18 13:05:30 -07:00
Micha Reiser
79b535587b
[red-knot] Reload notebook on file change (#12361) 2024-07-17 12:23:48 +00:00