Commit graph

20 commits

Author SHA1 Message Date
Tetsuharu Ohzeki
f474bd77be parser: Fix warnings about clippy str_to_string rule 2024-02-10 01:00:40 +09:00
Nicholas Nethercote
858f4aca6c Rename the unescaping functions.
`unescape_literal` becomes `unescape_unicode`, and `unescape_c_string`
becomes `unescape_mixed`. Because rfc3349 will mean that C string
literals will no longer be the only mixed utf8 literals.
2024-01-25 12:28:11 +11:00
Matthias Krüger
7d4980a4d8 Rollup merge of #119172 - nnethercote:earlier-NulInCStr, r=petrochenkov
Detect `NulInCStr` error earlier.

By making it an `EscapeError` instead of a `LitError`. This makes it like the other errors produced when checking string literals contents, e.g. for invalid escape sequences or bare CR chars.

NOTE: this means these errors are issued earlier, before expansion, which changes behaviour. It will be possible to move the check back to the later point if desired. If that happens, it's likely that all the string literal contents checks will be delayed together.

One nice thing about this: the old approach had some code in `report_lit_error` to calculate the span of the nul char from a range. This code used a hardwired `+2` to account for the `c"` at the start of a C string literal, but this should have changed to a `+3` for raw C string literals to account for the `cr"`, which meant that the caret in `cr"` nul error messages was one short of where it should have been. The new approach doesn't need any of this and avoids the off-by-one error.

r? ```@fee1-dead```
2024-01-18 10:34:17 +01:00
Laurențiu Nicola
6bbd106c70 Merge commit '9d8889cdfc' into sync-from-ra 2024-01-15 11:40:09 +02:00
Nicholas Nethercote
6001c50cac Detect NulInCStr error earlier.
By making it an `EscapeError` instead of a `LitError`. This makes it
like the other errors produced when checking string literals contents,
e.g. for invalid escape sequences or bare CR chars.

NOTE: this means these errors are issued earlier, before expansion,
which changes behaviour. It will be possible to move the check back to
the later point if desired. If that happens, it's likely that all the
string literal contents checks will be delayed together.

One nice thing about this: the old approach had some code in
`report_lit_error` to calculate the span of the nul char from a range.
This code used a hardwired `+2` to account for the `c"` at the start of
a C string literal, but this should have changed to a `+3` for raw C
string literals to account for the `cr"`, which meant that the caret in
`cr"` nul error messages was one short of where it should have been. The
new approach doesn't need any of this and avoids the off-by-one error.
2024-01-12 16:19:37 +11:00
Laurențiu Nicola
d1d111d09e Merge commit '3b7c7f97e4' into sync-from-ra 2023-11-08 08:15:03 +02:00
Laurențiu Nicola
bcfc997eac Merge commit '258b15c506' into sync-from-ra 2023-09-18 12:33:49 +03:00
Laurențiu Nicola
c48062fe2a Merge commit 'aa9bc86125' into sync-from-ra 2023-06-05 12:04:23 +03:00
Laurențiu Nicola
bc45c7659a ⬆️ rust-analyzer 2023-02-13 13:55:14 +02:00
arcnmx
25242fe93f ⬆️ rust-analyzer
Merge commit '368e0bb32f'
2023-01-09 10:36:22 -08:00
Jonas Schievink
9bd11459ba Revert "Auto merge of #12149 - jonas-schievink:literally-just-a-literal, r=jonas-schievink"
This reverts commit cc9ae2b89e, reversing
changes made to 7dfd1cb572.
2022-05-13 15:08:14 +02:00
Jonas Schievink
2fe38d3b63 Indicate the number of float tokens in the first token 2022-05-05 16:28:59 +02:00
Jonas Schievink
1bc3305d95 Split float literal tokens at the . 2022-05-05 16:28:58 +02:00
Jonas Schievink
1f50e19eb2 Add a Converter type for token conversion 2022-05-02 17:47:12 +02:00
Aleksey Kladov
f4cb0ff9be internal: move ws attachment logic to the parser crate
This has to re-introduce the `sink` pattern, because doing this purely
with iterators is awkward :( Maaaybe the event vector was a false start?

But, anyway, I like the current factoring more -- it sort-of obvious
that we do want to keep ws-attachment business in the parser, and that
we also don't want that to depend on the particular tree structure. I
think `shortcuts` module achieves that.
2021-12-26 16:47:10 +03:00
Aleksey Kladov
74de79b1da internal: rename 2021-12-25 22:02:26 +03:00
Aleksey Kladov
92dad471bc
Update crates/parser/src/lexed_str.rs
Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>
2021-12-18 17:34:55 +03:00
Aleksey Kladov
a022ad68c9 internal: move all the lexing to the parser crate 2021-12-18 17:20:38 +03:00
Aleksey Kladov
78926027e3 converting lexed str to tokens 2021-12-18 15:36:21 +03:00
Aleksey Kladov
8b9d145dea soa all the things 2021-12-18 15:31:50 +03:00
Renamed from crates/parser/src/lexer_token.rs (Browse further)