mirrors/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-07-19 02:55:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Remove duplication around is_trivia functions (#11956)

Browse source 
## Summary

This PR removes the duplication around `is_trivia` functions.

There are two of them in the codebase:
1. In `pycodestyle`, it's for newline, indent, dedent, non-logical
newline and comment
2. In the parser, it's for non-logical newline and comment

The `TokenKind::is_trivia` method used (1) but that's not correct in
that context. So, this PR introduces a new `is_non_logical_token` helper
method for the `pycodestyle` crate and updates the
`TokenKind::is_trivia` implementation with (2).

This also means we can remove `Token::is_trivia` method and the
standalone `token_source::is_trivia` function and use the one on
`TokenKind`.

## Test Plan

`cargo insta test`
This commit is contained in:
Dhruv Manilawala 2024-06-21 15:32:40 +05:30 • committed by GitHub
parent 690e94f4fb
commit 4667d8697c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 41 additions and 59 deletions
Download patch file Download diff file Expand all files Collapse all files

8
crates/ruff_python_parser/src/token_source.rs
View file

@ -114,7 +114,7 @@ impl<'src> TokenSource<'src> {
fn do_bump(&mut self) {
loop {
let kind = self.lexer.next_token();
if is_trivia(kind) {
if kind.is_trivia() {
self.tokens
.push(Token::new(kind, self.current_range(), self.current_flags()));
continue;
@ -127,7 +127,7 @@ impl<'src> TokenSource<'src> {
fn next_non_trivia_token(&mut self) -> TokenKind {
loop {
let kind = self.lexer.next_token();
if is_trivia(kind) {
if kind.is_trivia() {
continue;
}
break kind;
@ -187,7 +187,3 @@ fn allocate_tokens_vec(contents: &str) -> Vec<Token> {
let lower_bound = contents.len().saturating_mul(15) / 100;
Vec::with_capacity(lower_bound)
}
fn is_trivia(token: TokenKind) -> bool {
matches!(token, TokenKind::Comment | TokenKind::NonLogicalNewline)
}