Commit graph

27 commits

Author SHA1 Message Date
Joshua Warner
4f32f43048
Implement block / indent based parsing
... and enforce that defs can only occur in blocks (or, inside parenthesized expressions)
2024-07-27 13:34:16 -07:00
Anton-4
ab217ede3f
Merge remote-tracking branch 'upstream/main' into unmacro_parsers 2024-06-22 20:02:30 +02:00
Anton-4
c303be7553
Merge branch 'main' into rust-1-76-0-upgrade 2024-04-30 19:36:01 +02:00
Jackson Wambolt
f6c977fb96
Un-macro succeed
This implementation requires that the type of the value impls `Clone`,
which means the function version of `succeed` cannot be used to succeed
with values of non-`Clone` types.

If this becomes an issue, we could either bring back the macro version,
or create a new `succeed_with` function that takes a `Fn() -> T`
argument. `succeed_with` could then be used to succeed with values that
aren't `Clone` by creating the value on-demand each time the parser is
run.
2024-04-22 20:03:04 -05:00
Jackson Wambolt
41d7d02e2a
Un-macro add 2024-04-22 20:03:03 -05:00
Joshua Warner
50d2111919
Remove unnecessary arg to check_indent
Quick refactoring / follow-up to #6634
2024-04-19 20:25:55 -07:00
Anton-4
1a5e065055
Merge branch 'main' into rust-1-76-0-upgrade 2024-04-16 13:33:27 +02:00
Anton-4
8502a37b3c
fmt 2024-04-15 16:51:07 +02:00
Anton-4
e4b814ce1c
clippy 2024-04-15 16:50:44 +02:00
Luke Boswell
c32fa5b600
support optional suffied last def 2024-04-14 13:45:06 +10:00
Luke Boswell
b13adf6898
more flexible statement parsing 2024-04-14 13:45:06 +10:00
Jonathan Schear
723c390f52
Treat two hash marks followed by text as a doc comment 2024-03-24 14:31:27 -04:00
Luke Boswell
1590b30b19
Add a lexing-based 'highlight' mode to the parser
basic syntax highlighting

added more syntax highlighting coverage

add example of a markdown table with styling

move FIXED_TOKEN logic into highlight

refactor highlight, add support for backpassing

escape html from source code

fix bug with <pre> tag ordering

refactor out html from roc_parse

remove test, put highlight functionality into separate file

fix typo
2023-02-28 17:03:49 +11:00
Anton-4
bbf35af8fa
Merge branch 'main' into rust1_65
Signed-off-by: Anton-4 <17049058+Anton-4@users.noreply.github.com>
2023-01-17 18:14:30 +01:00
Anton-4
a97c53d7ca
1.65 changes and clippy fixes 2023-01-17 18:09:16 +01:00
Joshua Warner
0b8e68f70d
Fix approximately a bajillion fmt and parsing bugs
(discovered by fuzzing)

There's more to come, but this seems like a good batch for now.
2023-01-13 18:20:44 -08:00
Joshua Warner
07ebc81cba
Be more lenient with indentation in record parsing
... and extract a shared helper with collection_trailing_sep_e, thereby resolving some inconsistencies with normal collection parsing.

... and delete some dead code while I'm at it
2023-01-02 18:53:15 -08:00
Joshua Warner
0da50a612d
Be more lenient with required indentation in collections
... and also remove a bunch of now-dead errors that can't be triggered.
2023-01-01 17:49:51 -08:00
Joshua Warner
e83cd8f191
Re-improve perf of skipping spaces and comments
On my M1 mac this shows as ~25% faster at parsing Num.roc than the old implementation, probably because nobody wrote any NEON code.

Even on my x86_64 linux box (Ryzen 2700x), this shows as 10% faster than the current SSE implementation (running with RUSTFLAGS="-C target-cpu=native").
2022-12-11 08:43:22 -08:00
Joshua Warner
5f29402297
Give parser fuzzing some TLC
* The header + expr fuzzers can both be run again (header fuzzer had regressed).
* I ran the expr fuzzer for ~60 seconds with no additional panics uncovered
* "tab_crash" hit supposedly unreachable code in blankspace.rs - and I went to the liberty of dramatically simplifying all that code, rather than just trying to fix the bug
* Other failures were straight-forward error cases that should have been handled (and passed up the chain) instead of panicking
2022-12-11 08:39:52 -08:00
Joshua Warner
2d9aba2242
Refactor parser methods to not return State as part of ParseError
As previously discovered with #4464, it's easy to accidentally mis-use the State value returned on the Err path.

There were mixed assumptions about what that State represents: (1) the State where the error occurred, or (2) the State at the beginning of the thing we were just parsing.

I fixed this up to always mean (2) - at which point we don't actually need to return the State at all - so it's impossible for further discrepency to creep in.

I also took the liberty to refactor a few more methods to be purely combinator-based, rather than calling `parse` directly.
2022-11-18 19:52:23 -05:00
Joshua Warner
ac5aa8d8ee
Combine bounds 2022-11-05 17:41:06 -04:00
Joshua Warner
5d5b71a083
Refactor several parse methods to use more combinators 2022-11-05 09:44:08 -04:00
Joshua Warner
07be8ec000
Refactor Parser trait to pass min_indent
This removes the need to explicitly pass thru min_indent when using the parser combinators.

My ultimate goal here is to evolve the current parser closer toward a purely combinator-based parser,
at which point we can more easily transition smoothly to a formal(ish) grammar, or expand the meanings of combinators
to include things like:
* Incremental (re)parsing
* Unified parsing and formatting code
* Better error recovery
* Using the main parser directly for syntax highlighting
2022-10-31 13:31:47 -07:00
Joshua Warner
4d4c0d9483 Don't allow closures to gobble unindented expr lines following them 2022-10-08 11:05:05 -07:00
Joshua Warner
ada8af25cc Correct indentation for when expressions
... and simultaneously remove the need for State::indent_column field / multiline tracking in blankspace.rs.

Fixes #2889
2022-07-17 19:26:33 -07:00
Anton-4
eee85fa45d
moved all crates into seperate folder + related path fixes 2022-07-01 17:37:43 +02:00
Renamed from compiler/parse/src/blankspace.rs (Browse further)