ruff/crates
Charlie Marsh 86ccdcc9d9
Add support for multi-character operator tokens to SimpleTokenizer (#6563)
## Summary

Allows for proper lexing of tokens like `->`.

The main challenge is to ensure that our forward and backwards
representations are the same for cases like `===`. Specifically, we want
that to lex as `==` followed by `=` regardless of whether it's a
forwards or backwards lex. To do so, we identify the range of the
sequential characters (the full span of `===`), lex it forwards, then
return the last token.

## Test Plan

`cargo test`
2023-08-16 09:09:19 -04:00
..
flake8_to_ruff Bump version to 0.0.284 (#6453) 2023-08-09 13:32:33 -05:00
ruff Allow top-level await in Jupyter notebooks (#6607) 2023-08-15 23:59:05 -04:00
ruff_benchmark Remove some extraneous newlines in Cargo.toml (#6577) 2023-08-14 23:39:41 +00:00
ruff_cache Error on zero tab width (#6429) 2023-08-08 16:51:37 -04:00
ruff_cli Remove some extraneous newlines in Cargo.toml (#6577) 2023-08-14 23:39:41 +00:00
ruff_dev Use a faster diffing library for the formatter ecosystem checks (#6497) 2023-08-11 15:51:54 +02:00
ruff_diagnostics Skip partial duplicates when applying multi-edit fixes (#6144) 2023-07-29 12:11:57 +00:00
ruff_formatter Replace inline(always) with inline (#6590) 2023-08-15 08:58:11 +02:00
ruff_index Add unreachable code rule (#5384) 2023-07-04 14:27:23 +00:00
ruff_macros Remove parser dependency from ruff-python-ast (#6096) 2023-07-26 17:47:22 +02:00
ruff_python_ast Replace inline(always) with inline (#6590) 2023-08-15 08:58:11 +02:00
ruff_python_codegen Remove Stmt::TryStar (#6566) 2023-08-14 13:39:44 -04:00
ruff_python_formatter Add support for multi-character operator tokens to SimpleTokenizer (#6563) 2023-08-16 09:09:19 -04:00
ruff_python_index Remove some extraneous newlines in Cargo.toml (#6577) 2023-08-14 23:39:41 +00:00
ruff_python_literal Remove allow(pedantic) from formatter (#6549) 2023-08-14 14:02:06 +02:00
ruff_python_parser Remove Stmt::TryStar (#6566) 2023-08-14 13:39:44 -04:00
ruff_python_resolver Replace .map_or(false, $closure) with .is_some_and(closure) (#6244) 2023-08-01 19:29:42 +02:00
ruff_python_semantic Remove Stmt::TryStar (#6566) 2023-08-14 13:39:44 -04:00
ruff_python_stdlib Replace .map_or(false, $closure) with .is_some_and(closure) (#6244) 2023-08-01 19:29:42 +02:00
ruff_python_trivia Add support for multi-character operator tokens to SimpleTokenizer (#6563) 2023-08-16 09:09:19 -04:00
ruff_shrinking Use Jupyter mode while parsing Notebook files (#5552) 2023-08-05 00:32:07 +00:00
ruff_source_file Skip BOM when determining Locator's line starts (#6159) 2023-07-29 11:47:13 +00:00
ruff_text_size Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_wasm Set a default on PythonVersion (#6446) 2023-08-09 15:19:27 +00:00