mirrors/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-10-15 04:49:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Use CommentRanges in backwards lexing (#7360)

Browse source 
## Summary

The tokenizer was split into a forward and a backwards tokenizer. The
backwards tokenizer uses the same names as the forwards ones (e.g.
`next_token`). The backwards tokenizer gets the comment ranges that we
already built to skip comments.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
This commit is contained in:
konsti 2023-09-16 05:21:45 +02:00 • committed by GitHub
parent 1f6e1485f9
commit 2cbe1733c8
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
41 changed files with 744 additions and 628 deletions
Download patch file Download diff file Expand all files Collapse all files

4
crates/ruff_python_formatter/src/comments/debug.rs
View file

@ -182,6 +182,7 @@ mod tests {
use ruff_formatter::SourceCode;
use ruff_python_ast::node::AnyNode;
use ruff_python_ast::{StmtBreak, StmtContinue};
use ruff_python_trivia::CommentRanges;
use ruff_text_size::{TextRange, TextSize};
use crate::comments::map::MultiMap;
@ -231,7 +232,8 @@ break;
),
);
let comments = Comments::new(comments_map);
let comment_ranges = CommentRanges::default();
let comments = Comments::new(comments_map, &comment_ranges);
assert_debug_snapshot!(comments.debug(source_code));
}