Commit graph

8927 commits

Author SHA1 Message Date
renovate[bot]
aa575da1e7
Update Rust crate fern to v0.7.1 (#15002) 2024-12-15 20:25:52 -05:00
renovate[bot]
921eb2acb3
Update Rust crate chrono to v0.4.39 (#15001) 2024-12-15 20:25:46 -05:00
renovate[bot]
8665d2dc95
Update Rust crate bstr to v1.11.1 (#15000) 2024-12-15 20:25:39 -05:00
renovate[bot]
1cc27c995c
Update NPM Development dependencies (#14999) 2024-12-15 20:25:10 -05:00
renovate[bot]
a93bc2af6b
Update dependency ruff to v0.8.3 (#15007) 2024-12-15 20:25:04 -05:00
Alex Waygood
d848182340
Pin mdformat plugins in pre-commit (#14992) 2024-12-15 19:37:45 +00:00
InSync
7173e6a20b
Use stripping block (|-) for page descriptions (#14980)
## Summary

Resolves #14976.

Currently, we uses this "[plain
scalar](https://yaml.org/spec/1.2.2/#733-plain-style)" format:

```yaml
description: Checks for `if key in dictionary: del dictionary[key]`.
```

Plain scalar must not contain the sequence `: `, however, so the above
is invalid.

This PR changes that to:

```yaml
description: |-
  Checks for `if key in dictionary: del dictionary[key]`.
```

`|` denotes a "[block
scalar](https://yaml.org/spec/1.2.2/#81-block-scalar-styles)", whereas
[the `-` chomping
indicator](https://yaml.org/spec/1.2.2/#8112-block-chomping-indicator)
requires that a trailing newline, if any, must be stripped.

## Test Plan


![](https://github.com/user-attachments/assets/f00b606a-d6fe-46ac-a1c5-6a8665204ea3)
2024-12-15 17:07:29 +01:00
w0nder1ng
4a7536dc94
[perflint] Fix panic in perf401 (#14971)
Fixes #14969.

The issue was that this line:

```rust
let from_assign_to_loop = TextRange::new(binding_stmt.end(), for_stmt.start());
```

was not safe if the binding was after the target. The only way (at least
that I can think of) this can happen is if they are in different scopes,
so it now checks for that before checking if there are usages between
the two.
2024-12-15 16:22:04 +01:00
Dimitri Papadopoulos Orfanos
2d15d7d1af
Improve the documentation of E201/E202 (#14983)
## Summary

The summary is misleading, as well as the
`whitespace-after-open-bracket` and `whitespace-before-close-bracket`
names - it's not only brackets, but also parentheses and braces. Align
the documentation with the actual behaviour.

Don't change the names, but align the documentation with the behaviour.

## Test Plan

No test (documentation).
2024-12-15 16:20:04 +01:00
Rebecca Chen
112e9d2d82
[ruff_python_ast] Add name and default functions to TypeParam. (#14964)
## Summary

This change adds `name` and `default` functions to `TypeParam` to access
the corresponding attributes more conveniently. I currently have these
as helper functions in code built on top of ruff_python_ast, and they
seemed like they might be generally useful.

## Test Plan

Ran the checks listed in CONTRIBUTING.md#development.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-15 12:04:51 +00:00
Alex Waygood
1389cb8e59
[red-knot] Emit an error if a bare Annotated or Literal is used in a type expression (#14973) 2024-12-15 02:00:52 +00:00
Alex Waygood
fa46ba2306
[red-knot] Fix bugs relating to assignability of dynamic type[] types (#14972) 2024-12-15 01:15:10 +00:00
github-actions[bot]
53c7ef8bfe
Sync vendored typeshed stubs (#14977)
Co-authored-by: typeshedbot <>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-15 01:02:41 +00:00
Alex Waygood
4d64cdb83c
[red-knot] ClassLiteral(<T>) is not a disjoint type from Instance(<metaclass of T>) (#14970)
## Summary

A class is an instance of its metaclass, so `ClassLiteral("ABC")` is not
disjoint from `Instance("ABCMeta")`. However, we erroneously consider
the two types disjoint on the `main` branch. This PR fixes that.

This bug was uncovered by adding some more core types to the property
tests that provide coverage for classes that have custom metaclasses.
The additions to the property tests are included in this PR.

## Test Plan

New unit tests and property tests added. Tested with:
- `cargo test -p red_knot_python_semantic`
- `QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic --
--ignored types::property_tests::stable`

The assignability property test fails on this branch, but that's a known
issue that exists on `main`, due to
https://github.com/astral-sh/ruff/issues/14899.
2024-12-14 11:28:09 -08:00
Carl Meyer
ac31b26a0e
[red-knot] type[] is disjoint from None, LiteralString (#14967)
## Summary

Teach red-knot that `type[...]` is always disjoint from `None` and from
`LiteralString`. Fixes #14925.

This should properly be generalized to "all instances of final types
which are not subclasses of `type`", but until we support finality,
hardcoding `None` (which is known to be final) allows us to fix the
subtype transitivity property test.

## Test Plan

Existing tests pass, added new unit tests for `is_disjoint_from` and
`is_subtype_of`.

`QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic --
--ignored types::property_tests::stable` fails only the "assignability
is reflexive" test, which is known to fail on `main` (#14899).

The same command, with `property_tests.rs` edited to prevent generating
intersection tests (the cause of #14899), passes all quickcheck tests.
2024-12-14 11:02:49 +01:00
InSync
a80e934838
[red-knot] Error out when an mdtest code block is unterminated (#14965)
## Summary

Resolves #14934.

## Test Plan

Added a unit test.
2024-12-13 21:51:21 -08:00
Alex Waygood
224c8438bd
[red-knot] Minor simplifications to types.rs (#14962) 2024-12-13 20:31:51 +00:00
Alex Waygood
90a5439791
[red-knot] Use type[Unknown] rather than Unknown as the fallback metaclass for invalid classes (#14961) 2024-12-13 19:48:51 +00:00
Alex Waygood
4b2b126b9f
[red-knot] Make is_subtype_of exhaustive (#14924) 2024-12-13 19:31:22 +00:00
InSync
9798556eb5
[red-knot] Alphabetize rules (#14960)
## Summary

Follow-up from #14950.

## Test Plan

Purely stylistic change. Shouldn't affect any functionalities.
2024-12-13 10:39:18 -08:00
InSync
aa1938f6ba
[red-knot] Understand Annotated (#14950)
## Summary

Resolves #14922.

## Test Plan

Markdown tests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-13 09:41:37 -08:00
Dhruv Manilawala
3533d7f5b4
[red-knot] Display definition range in trace logs (#14955)
I've mainly opened this PR to get some opinions. I've found having some
additional information in the tracing logs to be useful to determine
what we are currently inferring. For the `Definition` ingredient, the
range seems to be much useful. I thought of using the identifier name
but we would have to deconstruct the `Expr` to find out the identifier
which seems a lot for just trace logs. Additionally, multiple
identifiers _could_ have the same name where range would be useful.

The ranges are isolated to the names that have been defined by the
definition except for the `except` block where the entire range is being
used because the name is optional.

***Before:***

```
3      ├─   0.074671s  54ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(1402)) } }
3      └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1402), file=/Users/dhruv/playground/ruff/type_inference/isolated3/play.py}
3      ┌─┘
3      ├─   0.074768s  54ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: inner_fn_name_(Id(2800)) } }
3      ├─   0.074807s  54ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_deferred_types(Id(1735)) } }
3      └─┐red_knot_python_semantic::types::infer::infer_deferred_types{definition=Id(1735), file=vendored://stdlib/typing.pyi}
3        ├─   0.074842s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(14f3)) } }
3        └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(14f3), file=vendored://stdlib/typing.pyi}
3          ├─   0.074871s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_expression_types(Id(1820)) } }
3          └─┐red_knot_python_semantic::types::infer::infer_expression_types{expression=Id(1820), file=vendored://stdlib/typing.pyi}
3            ├─   0.074924s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(1429)) } }
3            └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1429), file=vendored://stdlib/typing.pyi}
3              ├─   0.074958s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(1428)) } }
3              └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1428), file=vendored://stdlib/typing.pyi}
3              ┌─┘
```

***After:***

```
12      ├─   0.074609s  55ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(1402)) } }
12      └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1402), range=36..37, file=/Users/dhruv/playground/ruff/type_inference/isolated3/play.py}
12      ┌─┘
12      ├─   0.074705s  55ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: inner_fn_name_(Id(2800)) } }
12      ├─   0.074742s  55ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_deferred_types(Id(1735)) } }
12      └─┐red_knot_python_semantic::types::infer::infer_deferred_types{definition=Id(1735), range=30225..30236, file=vendored://stdlib/typing.pyi}
12        ├─   0.074775s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(14f3)) } }
12        └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(14f3), range=9472..9474, file=vendored://stdlib/typing.pyi}
12          ├─   0.074803s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_expression_types(Id(1820)) } }
12          └─┐red_knot_python_semantic::types::infer::infer_expression_types{expression=Id(1820), range=9477..9490, file=vendored://stdlib/typing.pyi}
12            ├─   0.074855s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(1429)) } }
12            └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1429), range=3139..3146, file=vendored://stdlib/typing.pyi}
12              ├─   0.074892s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(1428)) } }
12              └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1428), range=3102..3107, file=vendored://stdlib/typing.pyi}
12              ┌─┘
```
2024-12-13 14:29:53 +00:00
Alex Waygood
0bbe166720
[red-knot] Move the ClassBase enum to its own submodule (#14957) 2024-12-13 13:12:39 +00:00
David Peter
c3a64b44b7
[red-knot] mdtest: python version requirements (#14954)
## Summary

This is not strictly required yet, but makes these tests future-proof.
They need a `python-version` requirement as they rely on language
features that are not available in 3.9.
2024-12-13 10:40:38 +01:00
Wei Lee
dfd7f38009
[airflow]: Import modules that has been moved to airflow providers (AIR303) (#14764)
## Summary

Many core Airflow features have been deprecated and moved to Airflow
Providers since users might need to install an additional package (e.g.,
`apache-airflow-provider-fab==1.0.0`); a separate rule (AIR303) is
created for this.

As some of the changes only relate to the module/package moved, instead
of listing out all the functions, variables, and classes in a module or
a package, it warns the user to import from the new path instead of the
specific name.

The following is the ones that has been moved to
`apache-airflow-provider-fab==1.0.0`

* module moved
* `airflow.api.auth.backend.basic_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.basic_auth`
* `airflow.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.security_manager.override` →
`airflow.providers.fab.auth_manager.security_manager.override`
* classes (e.g., functions, classes) moved
* `airflow.www.security.FabAirflowSecurityManagerOverride` →
`airflow.providers.fab.auth_manager.security_manager.override.FabAirflowSecurityManagerOverride`
* `airflow.auth.managers.fab.fab_auth_manager.FabAuthManager` →
`airflow.providers.fab.auth_manager.security_manager.FabAuthManager`

## Test Plan


A test fixture has been included for the rule.
2024-12-13 10:38:07 +01:00
David Peter
e96b13c027
[red-knot] Support typing.TYPE_CHECKING (#14952)
## Summary

Add support for `typing.TYPE_CHECKING` and
`typing_extensions.TYPE_CHECKING`.

relates to: https://github.com/astral-sh/ruff/issues/14170

## Test Plan

New Markdown-based tests
2024-12-13 09:24:48 +00:00
Micha Reiser
f52b1f4a4d
Add tracing support to mdtest (#14935)
## Summary

This PR extends the mdtest configuration with a `log` setting that can
be any of:

* `true`: Enables tracing
* `false`: Disables tracing (default)
* String: An ENV_FILTER similar to `RED_KNOT_LOG`

```toml
log = true
```

Closes https://github.com/astral-sh/ruff/issues/13865

## Test Plan

I changed a test and tried `log=true`, `log=false`, and `log=INFO`
2024-12-13 09:10:01 +00:00
Micha Reiser
1c8f356e07
Re-enable the fuzzer job on PRs (#14953)
## Summary
This reverts https://github.com/astral-sh/ruff/pull/14478

I now broke main twice because I wasn't aware that the API was used by
the fuzzer.

## Test Plan
2024-12-13 09:07:27 +00:00
David Peter
2ccc9b19a7
[red-knot] Improve match mdtests (#14951)
## Summary

Minor improvement for the `match` tests to make sure we can't infer
statically whether or not a certain `case` applies.
2024-12-13 09:50:17 +01:00
Micha Reiser
c1837e4189
Rename custom-typeshed-dir, target-version and current-directory CLI options (#14930)
## Summary

This PR renames the `--custom-typeshed-dir`, `target-version`, and
`--current-directory` cli options to `--typeshed`,
`--python-version`, and `--project` as discussed in the CLI proposal
document.
I added aliases for `--target-version` (for Ruff compat) and
`--custom-typeshed-dir` (for Alex)

## Test Plan

Long help

```
An extremely fast Python type checker.

Usage: red_knot [OPTIONS] [COMMAND]

Commands:
  server  Start the language server
  help    Print this message or the help of the given subcommand(s)

Options:
      --project <PROJECT>
          Run the command within the given project directory.
          
          All `pyproject.toml` files will be discovered by walking up the directory tree from the project root, as will the project's virtual environment (`.venv`).
          
          Other command-line arguments (such as relative paths) will be resolved relative to the current working directory."#,

      --venv-path <PATH>
          Path to the virtual environment the project uses.
          
          If provided, red-knot will use the `site-packages` directory of this virtual environment to resolve type information for the project's third-party dependencies.

      --typeshed-path <PATH>
          Custom directory to use for stdlib typeshed stubs

      --extra-search-path <PATH>
          Additional path to use as a module-resolution source (can be passed multiple times)

      --python-version <VERSION>
          Python version to assume when resolving types
          
          [possible values: 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13]

  -v, --verbose...
          Use verbose output (or `-vv` and `-vvv` for more verbose output)

  -W, --watch
          Run in watch mode by re-running whenever files change

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version
```

Short help 

```
An extremely fast Python type checker.

Usage: red_knot [OPTIONS] [COMMAND]

Commands:
  server  Start the language server
  help    Print this message or the help of the given subcommand(s)

Options:
      --project <PROJECT>         Run the command within the given project directory
      --venv-path <PATH>          Path to the virtual environment the project uses
      --typeshed-path <PATH>      Custom directory to use for stdlib typeshed stubs
      --extra-search-path <PATH>  Additional path to use as a module-resolution source (can be passed multiple times)
      --python-version <VERSION>  Python version to assume when resolving types [possible values: 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13]
  -v, --verbose...                Use verbose output (or `-vv` and `-vvv` for more verbose output)
  -W, --watch                     Run in watch mode by re-running whenever files change
  -h, --help                      Print help (see more with '--help')
  -V, --version                   Print version

```

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-13 08:21:52 +00:00
David Peter
d7ce548893
[red-knot] Add narrowing for 'while' loops (#14947)
## Summary

Add type narrowing for `while` loops and corresponding `else` branches.

closes #14861 

## Test Plan

New Markdown tests.
2024-12-13 07:40:14 +01:00
Krishnan Chandra
be4ce16735
[ruff] Skip SQLModel base classes for mutable-class-default (RUF012) (#14949)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14892, by adding
`sqlmodel.SQLModel` to the list of classes with default copy semantics.

## Test Plan

Added a test into `RUF012.py` containing the example from the original
issue.
2024-12-12 22:19:21 -06:00
David Peter
657d26ff20
[red-knot] Tests for 'while' loop boundness (#14944)
## Summary

Regression test(s) for something that broken while implementing #14759.
We have similar tests for other control flow elements, but feel free to
let me know if this seems superfluous.

## Test Plan

New mdtests
2024-12-12 21:06:56 +01:00
Alex Waygood
dbc191d2d6
[red-knot] Fixes to Type::to_meta_type (#14942) 2024-12-12 19:55:11 +00:00
David Peter
d2712c7669
ruff_python_ast: Make Singleton Copy (#14943)
## Summary

Minor changed pulled out from #14759, as it seems to make sense in
isolation.

## Test Plan

—
2024-12-12 20:49:54 +01:00
Chandra Kiran G
e5cb4d6388
[flake8-pyi]: More autofixes for redundant-none-literal (PYI061) (#14872)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-12 19:44:32 +00:00
Sergey Mezentsev
68e8496260
[flake8-use-pathlib] Extend check for invalid path suffix to include the case "." (PTH210) (#14902)
## Summary

`PTH210` renamed to `invalid-pathlib-with-suffix` and extended to check for `.with_suffix(".")`. This caused the fix availability to be downgraded to "Sometimes", since there is no fix offered in this case.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Dylan <53534755+dylwil3@users.noreply.github.com>
2024-12-12 13:30:17 -06:00
Alex Waygood
71239f248e
[red-knot] Add explicit TODO branches for many typing special forms and qualifiers (#14936) 2024-12-12 17:57:26 +00:00
Alex Waygood
58930905eb
[red-knot] Fixup a few edge cases regarding type[] (#14918) 2024-12-12 16:53:03 +00:00
Dhruv Manilawala
53f2d72e02
Revert certain double quotes from workflow shell script (#14939)
Follow-up from #14938
2024-12-12 20:29:48 +05:30
Dhruv Manilawala
3629cbf35a
Use double quotes consistently for shell scripts (#14938)
## Summary

The release failed
(3432150963)
because the shell script in the Docker release workflow was using single
quotes instead of double quotes.

This is related to https://www.shellcheck.net/wiki/SC2016. I found it
via [`actionlint`](https://github.com/rhysd/actionlint). Related #14893.

I also went ahead and fixed https://www.shellcheck.net/wiki/SC2086 which
were raised in a couple of places.
2024-12-12 08:45:08 -06:00
Dylan
37f433814c
Bump version to 0.8.3 (#14937)
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-12-12 14:13:06 +00:00
Alex Waygood
45b565cbb5
[red-knot] Any cannot be parameterized (#14933) 2024-12-12 11:50:34 +00:00
Micha Reiser
82faa9bb62
Add tests demonstrating f-strings with debug expressions in replacements that contain escaped characters (#14929) 2024-12-12 09:33:20 +00:00
w0nder1ng
2eac00c60f
[perflint] fix invalid hoist in perf401 (#14369)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-12 09:11:09 +01:00
Alex Waygood
033ecf5a4b
Also have zizmor check for low-severity security issues (#14893)
## Summary

This PR changes our zizmor configuration to also flag low-severity
security issues in our GitHub Actions workflows. It's a followup to
https://github.com/astral-sh/ruff/pull/14844. The issues being fixed
here were all flagged by [zizmor's `template-injection`
rule](https://woodruffw.github.io/zizmor/audits/#template-injection):

> Detects potential sources of code injection via template expansion.
>
> GitHub Actions allows workflows to define template expansions, which
occur within special `${{ ... }}` delimiters. These expansions happen
before workflow and job execution, meaning the expansion of a given
expression appears verbatim in whatever context it was performed in.
>
> Template expansions aren't syntax-aware, meaning that they can result
in unintended shell injection vectors. This is especially true when
they're used with attacker-controllable expression contexts, such as
`github.event.issue.title` (which the attacker can fully control by
supplying a new issue title).

[...]

> To fully remediate the vulnerability, you should not use `${{
env.VARNAME }}`, since that is still a template expansion. Instead, you
should use `${VARNAME}` to ensure that the shell itself performs the
variable expansion.

## Test Plan

I tested that this passes all zizmore warnings by running `pre-commit
run -a zizmor` locally. The other test is obviously to check that the
workflows all still run correctly in CI 😄
2024-12-12 07:43:17 +00:00
Peter Tripp
5509a3d7ae
Add LSP settings example for Zed editor (#14894)
## Summary

Add Zed settings examples to in addition to NeoVim and VSCode.

<img width="373" alt="Screenshot 2024-12-11 at 9 54 57"
src="https://github.com/user-attachments/assets/2f00cd8b-b23e-4ecb-8a0d-732ec275ee6b"
/>

## Test Plan

[*] Tested locally. No errors. Looks fine to me.
2024-12-12 12:44:56 +05:30
InSync
e4885a2fb2
[red-knot] Understand typing.Tuple (#14927)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-12 00:58:06 +00:00
David Peter
a7e5e42b88
[red-knot] Make attributes.md test future-proof (#14923)
## Summary

Using `typing.LiteralString` breaks as soon as we understand
`sys.version_info` branches, as it's only available in 3.11 and later.

## Test Plan

Made sure it didn't fail on my #14759 branch anymore.
2024-12-11 20:46:24 +01:00
Alex Waygood
c361cf66ad
[red-knot] Precise inference for __class__ attributes on objects of all types (#14921) 2024-12-11 17:30:34 +00:00