ruff/crates/ty_completion_eval/truth
Andrew Gallant 3771f1567c [ty] Add an evaluation for completions
This is still early days, but I hope the framework introduced here makes
it very easy to add new truth data. Truth data should be seen as a form
of regression test for non-ideal ranking of completion suggestions.

I think it would help to read `crates/ty_completion_eval/README.md`
first to get an idea of what you're reviewing.
2025-10-08 08:44:21 -04:00
..
higher-level-symbols-preferred [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
import-deprioritizes-dunder [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
import-deprioritizes-sunder [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
internal-typeshed-hidden [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
numpy-array [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
object-attr-instance-methods [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
raise-uses-base-exception [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
scope-existing-over-new-import [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
scope-prioritize-closer [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
scope-simple-long-identifier [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
ty-extensions-lower-stdlib [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
type-var-typing-over-ast [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00
README.md [ty] Add an evaluation for completions 2025-10-08 08:44:21 -04:00

This directory contains truth data for ty's completion evaluation.

Adding new truth data

To add new truth data, you can either add a new <CURSOR> directive to an existing Python project in this directory or create a new Python project. To create a new directory, just cp -a existing new and modify it as needed. Then:

  1. Check completion.toml for relevant settings.
  2. Run uv.lock after updating pyproject.toml (if necessary) to ensure the dependency versions are locked.