language-servers/rust-analyzer
1
0
Fork 0
mirror of https://github.com/rust-lang/rust-analyzer.git synced 2025-09-10 12:26:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

internal: replace TreeSink with a data structure

Browse source 
The general theme of this is to make parser a better independent
library.

The specific thing we do here is replacing callback based TreeSink with
a data structure. That is, rather than calling user-provided tree
construction methods, the parser now spits out a very bare-bones tree,
effectively a log of a DFS traversal.

This makes the parser usable without any *specifc* tree sink, and allows
us to, eg, move tests into this crate.

Now, it's also true that this is a distinction without a difference, as
the old and the new interface are equivalent in expressiveness. Still,
this new thing seems somewhat simpler. But yeah, I admit I don't have a
suuper strong motivation here, just a hunch that this is better.
This commit is contained in:
Aleksey Kladov 2021-12-19 17:36:23 +03:00
parent 2f63558dc5
commit d0d05075ed
10 changed files with 172 additions and 110 deletions
Download patch file Download diff file Expand all files Collapse all files

34
crates/syntax/src/parsing.rs
View file

@ -4,24 +4,18 @@
mod text_tree_sink;
mod reparsing;
use parser::SyntaxKind;
use text_tree_sink::TextTreeSink;
use crate::{syntax_node::GreenNode, AstNode, SyntaxError, SyntaxNode};
use crate::{
parsing::text_tree_sink::build_tree, syntax_node::GreenNode, AstNode, SyntaxError, SyntaxNode,
};
pub(crate) use crate::parsing::reparsing::incremental_reparse;
pub(crate) fn parse_text(text: &str) -> (GreenNode, Vec<SyntaxError>) {
let lexed = parser::LexedStr::new(text);
let parser_tokens = lexed.to_tokens();
let mut tree_sink = TextTreeSink::new(lexed);
parser::parse_source_file(&parser_tokens, &mut tree_sink);
let (tree, parser_errors) = tree_sink.finish();
(tree, parser_errors)
let tree_traversal = parser::parse_source_file(&parser_tokens);
let (node, errors, _eof) = build_tree(lexed, tree_traversal, false);
(node, errors)
}
/// Returns `text` parsed as a `T` provided there are no parse errors.
@ -34,20 +28,12 @@ pub(crate) fn parse_text_as<T: AstNode>(
return Err(());
}
let parser_tokens = lexed.to_tokens();
let tree_traversal = parser::parse(&parser_tokens, entry_point);
let (node, errors, eof) = build_tree(lexed, tree_traversal, true);
let mut tree_sink = TextTreeSink::new(lexed);
// TextTreeSink assumes that there's at least some root node to which it can attach errors and
// tokens. We arbitrarily give it a SourceFile.
use parser::TreeSink;
tree_sink.start_node(SyntaxKind::SOURCE_FILE);
parser::parse(&parser_tokens, &mut tree_sink, entry_point);
tree_sink.finish_node();
let (tree, parser_errors, eof) = tree_sink.finish_eof();
if !parser_errors.is_empty() || !eof {
if !errors.is_empty() || !eof {
return Err(());
}
SyntaxNode::new_root(tree).first_child().and_then(T::cast).ok_or(())
SyntaxNode::new_root(node).first_child().and_then(T::cast).ok_or(())
}