Add a script that tests formatter stability on repositories (#5055)

## Summary

We want to ensure that once formatted content stays the same when
formatted again, which is known as formatter stability or formatter
idempotency, and that the formatter prints syntactically valid code. As
our test cases cover only a limited amount of code, this allows checking
entire repositories.

This adds a new subcommand to `ruff_dev` which can be invoked as `cargo
run --bin ruff_dev -- check-formatter-stability <repo>`. While initially
only intended to check stability, it has also found cases where the
formatter printed invalid syntax or panicked.

 ## Test Plan

Running this on cpython is already identifying bugs
(https://github.com/astral-sh/ruff/pull/5089)
This commit is contained in:
konstin 2023-06-19 16:13:38 +02:00 committed by GitHub
parent 0e028142f4
commit b8d378b0a3
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 309 additions and 17 deletions

View file

@ -24,7 +24,7 @@ mod commands;
mod diagnostics;
mod panic;
mod printer;
mod resolve;
pub mod resolve;
#[derive(Copy, Clone)]
pub enum ExitStatus {

View file

@ -14,7 +14,7 @@ use crate::args::Overrides;
/// Resolve the relevant settings strategy and defaults for the current
/// invocation.
pub(crate) fn resolve(
pub fn resolve(
isolated: bool,
config: Option<&Path>,
overrides: &Overrides,