<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Closes https://github.com/astral-sh/ty/issues/989
There are various situations where users expect the Python packages
installed in the same environment as ty itself to be considered during
type checking. A minimal example would look like:
```
uv venv my-env
uv pip install my-env ty httpx
echo "import httpx" > foo.py
./my-env/bin/ty check foo.py
```
or
```
uv tool install ty --with httpx
echo "import httpx" > foo.py
ty check foo.py
```
While these are a bit contrived, there are real-world situations where a
user would expect a similar behavior to work. Notably, all of the other
type checkers consider their own environment when determining search
paths (though I'll admit that I have not verified when they choose not
to do this).
One common situation where users are encountering this today is with
`uvx --with-requirements script.py ty check script.py` — which is
currently our "best" recommendation for type checking a PEP 723 script,
but it doesn't work.
Of the options discussed in
https://github.com/astral-sh/ty/issues/989#issuecomment-3307417985, I've
chosen (2) as our criteria for including ty's environment in the search
paths.
- If no virtual environment is discovered, we will always include ty's
environment.
- If a `.venv` is discovered in the working directory, we will _prepend_
ty's environment to the search paths. The dependencies in ty's
environment (e.g., from `uvx --with`) will take precedence.
- If a virtual environment is active, e.g., `VIRTUAL_ENV` (i.e.,
including conda prefixes) is set, we will not include ty's environment.
The reason we need to special case the `.venv` case is that we both
1. Recommend `uvx ty` today as a way to check your project
2. Want to enable `uvx --with <...> ty`
And I don't want (2) to break when you _happen_ to be in a project
(i.e., if we only included ty's environment when _no_ environment is
found) and don't want to remove support for (1).
I think long-term, I want to make `uvx <cmd>` layer the environment on
_top_ of the project environment (in uv), which would obviate the need
for this change when you're using uv. However, that change is breaking
and I think users will expect this behavior in contexts where they're
not using uv, so I think we should handle it in ty regardless.
I've opted not to include the environment if it's non-virtual (i.e., a
system environment) for now. It seems better to start by being more
restrictive. I left a comment in the code.
## Test Plan
I did some manual testing with the initial commit, then subsequently
added some unit tests.
```
❯ echo "import httpx" > example.py
❯ uvx --with httpx ty check example.py
Installed 8 packages in 19ms
error[unresolved-import]: Cannot resolve imported module `httpx`
--> foo/example.py:1:8
|
1 | import httpx
| ^^^^^
|
info: Searched in the following paths during module resolution:
info: 1. /Users/zb/workspace/ty/python (first-party code)
info: 2. /Users/zb/workspace/ty (first-party code)
info: 3. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
info: rule `unresolved-import` is enabled by default
Found 1 diagnostic
❯ uvx --from . --with httpx ty check example.py
All checks passed!
```
```
❯ uv init --script foo.py
Initialized script at `foo.py`
❯ uv add --script foo.py httpx
warning: The Python request from `.python-version` resolved to Python 3.13.8, which is incompatible with the script's Python requirement: `>=3.14`
Updated `foo.py`
❯ echo "import httpx" >> foo.py
❯ uvx --with-requirements foo.py ty check foo.py
error[unresolved-import]: Cannot resolve imported module `httpx`
--> foo.py:15:8
|
13 | if __name__ == "__main__":
14 | main()
15 | import httpx
| ^^^^^
|
info: Searched in the following paths during module resolution:
info: 1. /Users/zb/workspace/ty/python (first-party code)
info: 2. /Users/zb/workspace/ty (first-party code)
info: 3. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
info: rule `unresolved-import` is enabled by default
Found 1 diagnostic
❯ uvx --from . --with-requirements foo.py ty check foo.py
All checks passed!
```
Notice we do not include ty's environment if `VIRTUAL_ENV` is set
```
❯ VIRTUAL_ENV=.venv uvx --with httpx ty check foo/example.py
error[unresolved-import]: Cannot resolve imported module `httpx`
--> foo/example.py:1:8
|
1 | import httpx
| ^^^^^
|
info: Searched in the following paths during module resolution:
info: 1. /Users/zb/workspace/ty/python (first-party code)
info: 2. /Users/zb/workspace/ty (first-party code)
info: 3. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: 4. /Users/zb/workspace/ty/.venv/lib/python3.13/site-packages (site-packages)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
info: rule `unresolved-import` is enabled by default
Found 1 diagnostic
```
Summary
--
This PR unifies the two different ways Ruff and ty construct syntax
errors. Ruff has been storing the primary message in the diagnostic
itself, while ty attached the message to the primary annotation:
```
> ruff check try.py
invalid-syntax: name capture `x` makes remaining patterns unreachable
--> try.py:2:10
|
1 | match 42:
2 | case x: ...
| ^
3 | case y: ...
|
Found 1 error.
> uvx ty check try.py
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
Checking ------------------------------------------------------------ 1/1 files
error[invalid-syntax]
--> try.py:2:10
|
1 | match 42:
2 | case x: ...
| ^ name capture `x` makes remaining patterns unreachable
3 | case y: ...
|
Found 1 diagnostic
```
I think there are benefits to both approaches, and I do like ty's
version, but I feel like we should pick one (and it might help with
#20901 eventually). I slightly prefer Ruff's version, so I went with
that. Hopefully this isn't too controversial, but I'm happy to close
this if it is.
Note that this shouldn't change any other diagnostic formats in ty
because
[`Diagnostic::primary_message`](98d27c4128/crates/ruff_db/src/diagnostic/mod.rs (L177))
was already falling back to the primary annotation message if the
diagnostic message was empty. As a result, I think this change will
partially resolve the FIXME therein.
Test Plan
--
Existing tests with updated snapshots
## Summary
Fixes https://github.com/astral-sh/ty/issues/1242
From finding references with the LSP, `FileResolver::path` is only
called once, in `UnifiedFile::path`, so I went through those references,
and it looked safe to make this change in every case. Most of the
references are in the various output formats, where we inherited the
absolute vs relative path decision from Ruff. Two other uses are as
fallbacks if converting a relativized path to a string fails. Finally,
we use the path for sorting and in `UnifiedFile::relative_path`.
## Test Plan
Existing tests, with snapshots updated to show absolute paths (in the
`TestDb` this just added a `/` in front of the file names). I also
updated the GitLab CLI test to set the `CI_PROJECT_DIR` environment
variable and ran a test in GitLab CI:
<img width="613" height="114" alt="image"
src="https://github.com/user-attachments/assets/8ab81dba-54fd-4a24-9110-77ef89293cff"
/>
## Summary
This PR wires up the GitHub output format moved to `ruff_db` in #20320
to the ty CLI.
It's a bit smaller than the GitLab version (#20155) because some of the
helpers were already in place, but I did factor out a few
`DisplayDiagnosticConfig` constructor calls in Ruff. I also exposed the
`GithubRenderer` and a wrapper `DisplayGithubDiagnostics` type because
we needed a way to configure the program name displayed in the GitHub
diagnostics. This was previously hard-coded to `Ruff`:
<img width="675" height="247" alt="image"
src="https://github.com/user-attachments/assets/592da860-d2f5-4abd-bc5a-66071d742509"
/>
Another option would be to drop the program name in the output format,
but I think it can be helpful in workflows with multiple programs
emitting annotations (such as Ruff and ty!)
## Test Plan
New CLI test, and a manual test with `--config 'terminal.output-format =
"github"'`
## Summary
This wires up the GitLab output format moved into `ruff_db` in
https://github.com/astral-sh/ruff/pull/20117 to the ty CLI.
While I was here, I made one unrelated change to the CLI docs. Clap was
rendering the escapes around the `\[default\]` brackets for the `full`
output, so I just switched those to parentheses:
```
--output-format <OUTPUT_FORMAT>
The format to use for printing diagnostic messages
Possible values:
- full: Print diagnostics verbosely, with context and helpful hints \[default\]
- concise: Print diagnostics concisely, one per line
- gitlab: Print diagnostics in the JSON format expected by GitLab Code Quality reports
```
## Test Plan
New CLI test, and a manual test with `--config 'terminal.output-format =
"gitlab"'` to make sure this works as a configuration option too. I also
tried piping the output through jq to make sure it's at least valid JSON
## Summary
This PR moves most of the work of rendering concise diagnostics in Ruff
into `ruff_db`, where the code is shared with ty. To accomplish this
without breaking backwards compatibility in Ruff, there are two main
changes on the `ruff_db`/ty side:
- Added the logic from Ruff for remapping notebook line numbers to cells
- Reordered the fields in the diagnostic to match Ruff and rustc
```text
# old
error[invalid-assignment] try.py:3:1: Object of type `Literal[1]` is not
assignable to `str`
# new
try.py:3:1: error[invalid-assignment]: Object of type `Literal[1]` is
not assignable to `str`
```
I don't think the notebook change failed any tests on its own, and only
a handful of snaphots changed in ty after reordering the fields, but
this will obviously affect any other uses of the concise format, outside
of tests, too.
The other big change should only affect Ruff:
- Added three new `DisplayDiagnosticConfig` options
Micha and I hoped that we could get by with one option
(`hide_severity`), but Ruff also toggles `show_fix_status` itself,
independently (there are cases where we want neither severity nor the
fix status), and during the implementation I realized we also needed
access to an `Applicability`. The main goal here is to suppress the
severity (`error` above) because ruff only uses the `error` severity and
to use the secondary/noqa code instead of the line name
(`invalid-assignment` above).
```text
# ty - same as "new" above
try.py:3:1: error[invalid-assignment]: Object of type `Literal[1]` is
not assignable to `str`
# ruff
try.py:3:1: RUF123 [*] Object of type `Literal[1]` is not assignable to
`str`
```
This part of the concise diagnostic is actually shared with the `full`
output format in Ruff, but with the settings above, there are no
snapshot changes to either format.
## Test Plan
Existing tests with the handful of updates mentioned above, as well as
some new tests in the `concise` module.
Also this PR. Swapping the fields might have broken mypy_primer, unless
it occasionally times out on its own.
I also ran this script in the root of my Ruff checkout, which also has
CPython in it:
```shell
flags=(--isolated --no-cache --no-respect-gitignore --output-format concise .)
diff <(target/release/ruff check ${flags[@]} 2> /dev/null) \
<(ruff check ${flags[@]} 2> /dev/null)
```
This yielded an expected diff due to some t-string error changes on main
since 0.12.4:
```diff
33622c33622
< crates/ruff_python_parser/resources/inline/err/f_string_lambda_without_parentheses.py:1:15: SyntaxError: Expected an element of or the end of the f-string
---
> crates/ruff_python_parser/resources/inline/err/f_string_lambda_without_parentheses.py:1:15: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
33742c33742
< crates/ruff_python_parser/resources/inline/err/implicitly_concatenated_unterminated_string_multiline.py:4:1: SyntaxError: Expected an element of or the end of the f-string
---
> crates/ruff_python_parser/resources/inline/err/implicitly_concatenated_unterminated_string_multiline.py:4:1: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
34131c34131
< crates/ruff_python_parser/resources/inline/err/t_string_lambda_without_parentheses.py:2:15: SyntaxError: Expected an element of or the end of the t-string
---
> crates/ruff_python_parser/resources/inline/err/t_string_lambda_without_parentheses.py:2:15: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
```
So modulo color, the results are identical on 38,186 errors in our test
suite and CPython 3.10.
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
Fixes https://github.com/astral-sh/ty/issues/640. If a user passes
`--python=<some-virtual-environment>/bin/python`, we must avoid
canonicalizing the path until we've traversed upwards to find the
`sys.prefix` directory (`<some-virtual-environment>`). On Unix systems,
`<sys.prefix>/bin/python` is often a symlink to a system interpreter; if
we resolve the symlink too easily then we'll add the system
interpreter's `site-packages` directory as a search path rather than the
virtual environment's directory.
## Test Plan
I added an integration test to
`crates/ty/tests/cli/python_environment.rs` which fails on `main`. I
also manually tested locally that running `cargo run -p ty check foo.py
--python=.venv/bin/python -vv` now prints this log to the terminal
```
2025-06-20 18:35:24.57702 DEBUG Resolved site-packages directories for this virtual environment are: SitePackagesPaths({"/Users/alexw/dev/ruff/.venv/lib/python3.13/site-packages"})
```
Whereas it previously resolved `site-packages` to my system
intallation's `site-packages` directory
