language-servers/rust-analyzer
1
0
Fork 0
mirror of https://github.com/rust-lang/rust-analyzer.git synced 2025-09-27 20:42:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

internal: replace TreeSink with a data structure

Browse source 
The general theme of this is to make parser a better independent
library.

The specific thing we do here is replacing callback based TreeSink with
a data structure. That is, rather than calling user-provided tree
construction methods, the parser now spits out a very bare-bones tree,
effectively a log of a DFS traversal.

This makes the parser usable without any *specifc* tree sink, and allows
us to, eg, move tests into this crate.

Now, it's also true that this is a distinction without a difference, as
the old and the new interface are equivalent in expressiveness. Still,
this new thing seems somewhat simpler. But yeah, I admit I don't have a
suuper strong motivation here, just a hunch that this is better.
This commit is contained in:
Aleksey Kladov 2021-12-19 17:36:23 +03:00
parent 2f63558dc5
commit d0d05075ed
10 changed files with 172 additions and 110 deletions
Download patch file Download diff file Expand all files Collapse all files

17
crates/mbe/src/syntax_bridge.rs
View file

@ -1,6 +1,5 @@
//! Conversions between [`SyntaxNode`] and [`tt::TokenTree`].
use parser::{ParseError, TreeSink};
use rustc_hash::{FxHashMap, FxHashSet};
use syntax::{
ast::{self, make::tokens::doc_comment},
@ -56,8 +55,18 @@ pub fn token_tree_to_syntax_node(
_ => TokenBuffer::from_subtree(tt),
};
let parser_tokens = to_parser_tokens(&buffer);
let tree_traversal = parser::parse(&parser_tokens, entry_point);
let mut tree_sink = TtTreeSink::new(buffer.begin());
parser::parse(&parser_tokens, &mut tree_sink, entry_point);
for event in tree_traversal.iter() {
match event {
parser::TraversalStep::Token { kind, n_raw_tokens } => {
tree_sink.token(kind, n_raw_tokens)
}
parser::TraversalStep::EnterNode { kind } => tree_sink.start_node(kind),
parser::TraversalStep::LeaveNode => tree_sink.finish_node(),
parser::TraversalStep::Error { msg } => tree_sink.error(msg.to_string()),
}
}
if tree_sink.roots.len() != 1 {
return Err(ExpandError::ConversionError);
}
@ -643,7 +652,7 @@ fn delim_to_str(d: tt::DelimiterKind, closing: bool) -> &'static str {
&texts[idx..texts.len() - (1 - idx)]
}
impl<'a> TreeSink for TtTreeSink<'a> {
impl<'a> TtTreeSink<'a> {
fn token(&mut self, kind: SyntaxKind, mut n_tokens: u8) {
if kind == LIFETIME_IDENT {
n_tokens = 2;
@ -741,7 +750,7 @@ impl<'a> TreeSink for TtTreeSink<'a> {
*self.roots.last_mut().unwrap() -= 1;
}
fn error(&mut self, error: ParseError) {
fn error(&mut self, error: String) {
self.inner.error(error, self.text_pos)
}
}