8 commits

Author	SHA1	Message	Date
Ran Benita	5cc208cc43	Add NonLogicalNewline token ... This token is completely ignored by the parser, but it's useful for other users of the lexer, such as the Ruff linter. For example, the token is helpful for a "trailing comma" lint. The same idea exists in Python's `tokenize` module - there is a NEWLINE token (logical newline), and a NL token (non-logical newline). Fixes #4385.	2023-01-12 16:47:12 +02:00
harupy	1a28e49adc	Include comment text in token	2023-01-06 23:29:20 +09:00
harupy	eb75c4fa46	Address comments	2023-01-05 18:24:54 +09:00
harupy	7a78e12d52	Simplify string check	2023-01-04 23:57:50 +09:00
harupy	faec9372f9	Fix FormattedValue location	2022-12-30 21:39:29 +09:00
Jeong YunWon	9250260e20	Fix nightly clippy warnings	2022-12-05 12:18:16 +09:00
Charlie Marsh	c3f246693b	Implement Tok::Comment	2022-11-07 10:33:55 -05:00
Jeong YunWon	060d153bb3	reorganize compiler crates	2022-08-22 04:42:42 +09:00