[ty] Re-arrange "list modules" implementation for Salsa caching

This basically splits `list_modules` into a higher level "aggregation"
routine and a lower level "get modules for one search path" routine.
This permits Salsa to cache the lower level components, e.g., many
search paths refer to directories that rarely change. This saves us
interaction with the system.

This did require a fair bit of surgery in terms of being careful about
adding file roots. Namely, now that we rely even more on file roots
existing for correct handling of cache invalidation, there were several
spots in our code that needed to be updated to add roots (that we
weren't previously doing). This feels Not Great, and it would be better
if we had some kind of abstraction that handled this for us. But it
isn't clear to me at this time what that looks like.
This commit is contained in:
Andrew Gallant 2025-08-14 14:18:40 -04:00 committed by Andrew Gallant
parent 468eb37d75
commit ddd4bab67c
4 changed files with 177 additions and 47 deletions

View file

@ -7,7 +7,7 @@ use config::SystemKind;
use parser as test_parser;
use ruff_db::Db as _;
use ruff_db::diagnostic::{Diagnostic, DisplayDiagnosticConfig};
use ruff_db::files::{File, system_path_to_file};
use ruff_db::files::{File, FileRootKind, system_path_to_file};
use ruff_db::panic::catch_unwind;
use ruff_db::parsed::parsed_module;
use ruff_db::system::{DbWithWritableSystem as _, SystemPath, SystemPathBuf};
@ -184,6 +184,8 @@ fn run_test(
let project_root = SystemPathBuf::from("/src");
db.create_directory_all(&project_root)
.expect("Creating the project root to succeed");
db.files()
.try_add_root(db, &project_root, FileRootKind::Project);
let src_path = project_root.clone();
let custom_typeshed_path = test.configuration().typeshed();
@ -255,6 +257,8 @@ fn run_test(
// Create a custom typeshed `VERSIONS` file if none was provided.
if let Some(typeshed_path) = custom_typeshed_path {
db.files()
.try_add_root(db, typeshed_path, FileRootKind::LibrarySearchPath);
if !has_custom_versions_file {
let versions_file = typeshed_path.join("stdlib/VERSIONS");
let contents = typeshed_files