Charlie Marsh 3711f8ad59 Expand SimpleTokenizer to all keywords and single-character tokens (#6518) ... ## Summary For #6485, I need to be able to use the `SimpleTokenizer` to lex the space between any two adjacent expressions (i.e., the space between a preceding and following node). This requires that we support a wider range of keywords (like `and`, to connect the pieces of `x and y`), and some additional single-character tokens (like `-` and `>`, to support `->`). Note that the `SimpleTokenizer` does not support multi-character tokens, so the `->` in a function signature is lexed as a `-` followed by a `>` -- but this is fine for our purposes.		2023-08-14 10:35:31 -04:00
..
src	Expand SimpleTokenizer to all keywords and single-character tokens (#6518)	2023-08-14 10:35:31 -04:00
Cargo.toml	Pull in RustPython parser (#6099)	2023-07-27 09:29:11 +00:00