Commit graph

54 commits

Author SHA1 Message Date
Micha Reiser
2cf00fee96
Remove parser dependency from ruff-python-ast (#6096) 2023-07-26 17:47:22 +02:00
Micha Reiser
d351761f5d
SimpleTokenizer: Fix infinite loop when lexing empty quotes (#5917) 2023-07-20 15:18:35 +02:00
Micha Reiser
76e9ce6dc0
Fix SimpleTokenizer's backward lexing of # (#5878) 2023-07-20 11:54:18 +02:00
Charlie Marsh
5f3da9955a
Rename ruff_python_whitespace to ruff_python_trivia (#5886)
## Summary

This crate now contains utilities for dealing with trivia more broadly:
whitespace, newlines, "simple" trivia lexing, etc. So renaming it to
reflect its increased responsibilities.

To avoid conflicts, I've also renamed `Token` and `TokenKind` to
`SimpleToken` and `SimpleTokenKind`.
2023-07-19 11:48:27 -04:00