internal: replace TreeSink with a data structure

The general theme of this is to make parser a better independent
library.

The specific thing we do here is replacing callback based TreeSink with
a data structure. That is, rather than calling user-provided tree
construction methods, the parser now spits out a very bare-bones tree,
effectively a log of a DFS traversal.

This makes the parser usable without any *specifc* tree sink, and allows
us to, eg, move tests into this crate.

Now, it's also true that this is a distinction without a difference, as
the old and the new interface are equivalent in expressiveness. Still,
this new thing seems somewhat simpler. But yeah, I admit I don't have a
suuper strong motivation here, just a hunch that this is better.
This commit is contained in:
Aleksey Kladov 2021-12-19 17:36:23 +03:00
parent 2f63558dc5
commit d0d05075ed
10 changed files with 172 additions and 110 deletions

View file

@ -10,9 +10,8 @@
use std::mem;
use crate::{
ParseError,
tree_traversal::TreeTraversal,
SyntaxKind::{self, *},
TreeSink,
};
/// `Parser` produces a flat list of `Event`s.
@ -77,7 +76,7 @@ pub(crate) enum Event {
},
Error {
msg: ParseError,
msg: String,
},
}
@ -88,7 +87,8 @@ impl Event {
}
/// Generate the syntax tree with the control of events.
pub(super) fn process(sink: &mut dyn TreeSink, mut events: Vec<Event>) {
pub(super) fn process(mut events: Vec<Event>) -> TreeTraversal {
let mut res = TreeTraversal::default();
let mut forward_parents = Vec::new();
for i in 0..events.len() {
@ -117,15 +117,17 @@ pub(super) fn process(sink: &mut dyn TreeSink, mut events: Vec<Event>) {
for kind in forward_parents.drain(..).rev() {
if kind != TOMBSTONE {
sink.start_node(kind);
res.enter_node(kind);
}
}
}
Event::Finish => sink.finish_node(),
Event::Finish => res.leave_node(),
Event::Token { kind, n_raw_tokens } => {
sink.token(kind, n_raw_tokens);
res.token(kind, n_raw_tokens);
}
Event::Error { msg } => sink.error(msg),
Event::Error { msg } => res.error(msg),
}
}
res
}