7 commits

Author	SHA1	Message	Date
David Peter	3a77768f79	[ty] Reachability constraints (#18621) ... ## Summary * Completely removes the concept of visibility constraints. Reachability constraints are now used to model the static visibility of bindings and declarations. Reachability constraints are *much* easier to reason about / work with, since they are applied at the beginning of a branch, and not applied retroactively. Removing the duplication between visibility and reachability constraints also leads to major code simplifications [^1]. For an overview of how the new constraint system works, see the updated doc comment in `reachability_constraints.rs`. * Fixes a [control-flow modeling bug (panic)](https://github.com/astral-sh/ty/issues/365) involving `break` statements in loops * Fixes a [bug where](https://github.com/astral-sh/ty/issues/624) where `elif` branches would have wrong reachability constraints * Fixes a [bug where](https://github.com/astral-sh/ty/issues/648) code after infinite loops would not be considered unreachble * Fixes a panic on the `pywin32` ecosystem project, which we should be able to move to `good.txt` once this has been merged. * Removes some false positives in unreachable code because we infer `Never` more often, due to the fact that reachability constraints now apply retroactively to *all* active bindings, not just to bindings inside a branch. * As one example, this removes the `division-by-zero` diagnostic from https://github.com/astral-sh/ty/issues/443 because we now infer `Never` for the divisor. * Supersedes and includes similar test changes as https://github.com/astral-sh/ruff/pull/18392 closes https://github.com/astral-sh/ty/issues/365 closes https://github.com/astral-sh/ty/issues/624 closes https://github.com/astral-sh/ty/issues/642 closes https://github.com/astral-sh/ty/issues/648 ## Benchmarks Benchmarks on black, pandas, and sympy showed that this is neither a performance improvement, nor a regression. ## Test Plan Regression tests for: - [x] https://github.com/astral-sh/ty/issues/365 - [x] https://github.com/astral-sh/ty/issues/624 - [x] https://github.com/astral-sh/ty/issues/642 - [x] https://github.com/astral-sh/ty/issues/648 [^1]: I'm afraid this is something that @carljm advocated for since the beginning, and I'm not sure anymore why we have never seriously tried this before. So I suggest we do *not* attempt to do a historical deep dive to find out exactly why this ever became so complicated, and just enjoy the fact that we eventually arrived here. --------- Co-authored-by: Carl Meyer <carl@astral.sh>	2025-06-17 09:24:28 +02:00
Shunsuke Shibayama	0858896bc4	[ty] type narrowing by attribute/subscript assignments (#18041) ... ## Summary This PR partially solves https://github.com/astral-sh/ty/issues/164 (derived from #17643). Currently, the definitions we manage are limited to those for simple name (symbol) targets, but we expand this to track definitions for attribute and subscript targets as well. This was originally planned as part of the work in #17643, but the changes are significant, so I made it a separate PR. After merging this PR, I will reflect this changes in #17643. There is still some incomplete work remaining, but the basic features have been implemented, so I am publishing it as a draft PR. Here is the TODO list (there may be more to come): * [x] Complete rewrite and refactoring of documentation (removing `Symbol` and replacing it with `Place`) * [x] More thorough testing * [x] Consolidation of duplicated code (maybe we can consolidate the handling related to name, attribute, and subscript) This PR replaces the current `Symbol` API with the `Place` API, which is a concept that includes attributes and subscripts (the term is borrowed from Rust). ## Test Plan `mdtest/narrow/assignment.md` is added. --------- Co-authored-by: David Peter <sharkdp@users.noreply.github.com> Co-authored-by: Carl Meyer <carl@astral.sh>	2025-06-04 17:24:27 -07:00
Micha Reiser	9ae698fe30	Switch to Rust 2024 edition (#18129)	2025-05-16 13:25:28 +02:00
Micha Reiser	e7f97a3e4b	[ty] Reduce log level of 'symbol .. (via star import) not found' log message (#18087)	2025-05-14 09:20:23 +02:00
Dhruv Manilawala	78054824c0	[ty] Add support for __all__ (#17856) ... ## Summary This PR adds support for the `__all__` module variable. Reference spec: https://typing.python.org/en/latest/spec/distributing.html#library-interface-public-and-private-symbols This PR adds a new `dunder_all_names` query that returns a set of `Name`s defined in the `__all__` variable of the given `File`. The query works by implementing the `StatementVisitor` and collects all the names by recognizing the supported idioms as mentioned in the spec. Any idiom that's not recognized are ignored. The current implementation is minimum to what's required for us to remove all the false positives that this is causing. Refer to the "Follow-ups" section below to see what we can do next. I'll a open separate issue to keep track of them. Closes: astral-sh/ty#106 Closes: astral-sh/ty#199 ### Follow-ups * Diagnostics: * Add warning diagnostics for unrecognized `__all__` idioms, `__all__` containing non-string element * Add an error diagnostic for elements that are present in `__all__` but not defined in the module. This could lead to runtime error * Maybe we should return `<type>` instead of `Unknown | <type>` for `module.__all__`. For example: https://playknot.ruff.rs/2a6fe5d7-4e16-45b1-8ec3-d79f2d4ca894 * Mark a symbol that's mentioned in `__all__` as used otherwise it could raise (possibly in the future) "unused-name" diagnostic Supporting diagnostics will require that we update the return type of the query to be something other than `Option<FxHashSet<Name>>`, something that behaves like a result and provides a way to check whether a name exists in `__all__`, loop over elements in `__all__`, loop over the invalid elements, etc. ## Ecosystem analysis The following are the maximum amount of diagnostics **removed** in the ecosystem: * "Type <module '...'> has no attribute ..." * `collections.abc` - 14 * `numpy` - 35534 * `numpy.ma` - 296 * `numpy.char` - 37 * `numpy.testing` - 175 * `hashlib` - 311 * `scipy.fft` - 2 * `scipy.stats` - 38 * "Module '...' has no member ..." * `collections.abc` - 85 * `numpy` - 508 * `numpy.testing` - 741 * `hashlib` - 36 * `scipy.stats` - 68 * `scipy.interpolate` - 7 * `scipy.signal` - 5 The following modules have dynamic `__all__` definition, so `ty` assumes that `__all__` doesn't exists in that module: * `scipy.stats` (95a5d6ea8b/scipy/stats/__init__.py (L665)) * `scipy.interpolate` (95a5d6ea8b/scipy/interpolate/__init__.py (L221)) * `scipy.signal` (indirectly via 95a5d6ea8b/scipy/signal/_signal_api.py (L30)) * `numpy.testing` (de784cd6ee/numpy/testing/__init__.py (L16-L18)) ~There's this one category of **false positives** that have been added:~ Fixed the false positives by also ignoring `__all__` from a module that uses unrecognized idioms. <details><summary>Details about the false postivie:</summary> <p> The `scipy.stats` module has dynamic `__all__` and it imports a bunch of symbols via star imports. Some of those modules have a mix of valid and invalid `__all__` idioms. For example, in 95a5d6ea8b/scipy/stats/distributions.py (L18-L24), 2 out of 4 `__all__` idioms are invalid but currently `ty` recognizes two of them and says that the module has a `__all__` with 5 values. This leads to around **2055** newly added false positives of the form: ``` Type <module 'scipy.stats'> has no attribute ... ``` I think the fix here is to completely ignore `__all__`, not only if there are invalid elements in it, but also if there are unrecognized idioms used in the module. </p> </details> ## Test Plan Add a bunch of test cases using the new `ty_extensions.dunder_all_names` function to extract a module's `__all__` names. Update various test cases to remove false positives around `*` imports and re-export convention. Add new test cases for named import behavior as `*` imports covers all of it already (thanks Alex!).	2025-05-07 21:42:42 +05:30
Micha Reiser	fa628018b2	Use #[expect(lint)] over #[allow(lint)] where possible (#17822)	2025-05-03 21:20:31 +02:00
Micha Reiser	b51c4f82ea	Rename Red Knot (#17820)	2025-05-03 19:49:15 +02:00

Renamed from crates/red_knot_python_semantic/src/semantic_index/visibility_constraints.rs (Browse further)