Dhruv Manilawala 60ea72a6bc Add list terminator kind for error recovery (#11843) ... ## Summary This PR adds a new enum to determine the kind of terminator token i.e., is it actually terminates the list or is it used for error recovery. This is important because the parser should take the error recovery route in case the terminator token is used for better error recovery. This will then try to re-lex the token if it's the case. I haven't updated any reference to use this new enum as otherwise it'll update the snapshots. I plan to do that in a follow-up PR so that it's easier to reason about. ## Test plan `cargo insta test`		2024-06-12 08:33:26 +00:00
..
lexer	Maintain synchronicity between the lexer and the parser (#11457)	2024-06-03 18:23:50 +05:30
parser	Add list terminator kind for error recovery (#11843)	2024-06-12 08:33:26 +00:00
snapshots	Lexer should consider BOM for the start offset (#11732)	2024-06-04 08:45:46 +00:00
error.rs	Maintain synchronicity between the lexer and the parser (#11457)	2024-06-03 18:23:50 +05:30
lexer.rs	Use the existing ruff_python_trivia::is_python_whitespace function (#11844)	2024-06-12 05:59:19 +00:00
lib.rs	Build CommentRanges outside the parser (#11792)	2024-06-09 09:55:17 +00:00
string.rs	Maintain synchronicity between the lexer and the parser (#11457)	2024-06-03 18:23:50 +05:30
token.rs	Maintain synchronicity between the lexer and the parser (#11457)	2024-06-03 18:23:50 +05:30
token_set.rs	Replace LALRPOP parser with hand-written parser (#10036)	2024-04-18 17:57:39 +05:30
token_source.rs	Build CommentRanges outside the parser (#11792)	2024-06-09 09:55:17 +00:00
typing.rs	Update type annotation parsing API to return Parsed (#11739)	2024-06-05 12:59:43 +05:30