language-servers/rust-analyzer
1
0
Fork 0
mirror of https://github.com/rust-lang/rust-analyzer.git synced 2025-09-27 04:19:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

internal: replace TreeSink with a data structure

Browse source 
The general theme of this is to make parser a better independent
library.

The specific thing we do here is replacing callback based TreeSink with
a data structure. That is, rather than calling user-provided tree
construction methods, the parser now spits out a very bare-bones tree,
effectively a log of a DFS traversal.

This makes the parser usable without any *specifc* tree sink, and allows
us to, eg, move tests into this crate.

Now, it's also true that this is a distinction without a difference, as
the old and the new interface are equivalent in expressiveness. Still,
this new thing seems somewhat simpler. But yeah, I admit I don't have a
suuper strong motivation here, just a hunch that this is better.
This commit is contained in:
Aleksey Kladov 2021-12-19 17:36:23 +03:00
parent 2f63558dc5
commit d0d05075ed
10 changed files with 172 additions and 110 deletions
Download patch file Download diff file Expand all files Collapse all files

38
crates/parser/src/lib.rs
View file

@ -25,31 +25,19 @@ mod event;
mod parser;
mod grammar;
mod tokens;
mod tree_traversal;
#[cfg(test)]
mod tests;
pub(crate) use token_set::TokenSet;
pub use crate::{lexed_str::LexedStr, syntax_kind::SyntaxKind, tokens::Tokens};
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ParseError(pub Box<String>);
/// `TreeSink` abstracts details of a particular syntax tree implementation.
pub trait TreeSink {
/// Adds new token to the current branch.
fn token(&mut self, kind: SyntaxKind, n_tokens: u8);
/// Start new branch and make it current.
fn start_node(&mut self, kind: SyntaxKind);
/// Finish current branch and restore previous
/// branch as current.
fn finish_node(&mut self);
fn error(&mut self, error: ParseError);
}
pub use crate::{
lexed_str::LexedStr,
syntax_kind::SyntaxKind,
tokens::Tokens,
tree_traversal::{TraversalStep, TreeTraversal},
};
/// rust-analyzer parser allows you to choose one of the possible entry points.
///
@ -74,11 +62,11 @@ pub enum ParserEntryPoint {
}
/// Parse given tokens into the given sink as a rust file.
pub fn parse_source_file(tokens: &Tokens, tree_sink: &mut dyn TreeSink) {
parse(tokens, tree_sink, ParserEntryPoint::SourceFile);
pub fn parse_source_file(tokens: &Tokens) -> TreeTraversal {
parse(tokens, ParserEntryPoint::SourceFile)
}
pub fn parse(tokens: &Tokens, tree_sink: &mut dyn TreeSink, entry_point: ParserEntryPoint) {
pub fn parse(tokens: &Tokens, entry_point: ParserEntryPoint) -> TreeTraversal {
let entry_point: fn(&'_ mut parser::Parser) = match entry_point {
ParserEntryPoint::SourceFile => grammar::entry_points::source_file,
ParserEntryPoint::Path => grammar::entry_points::path,
@ -99,7 +87,7 @@ pub fn parse(tokens: &Tokens, tree_sink: &mut dyn TreeSink, entry_point: ParserE
let mut p = parser::Parser::new(tokens);
entry_point(&mut p);
let events = p.finish();
event::process(tree_sink, events);
event::process(events)
}
/// A parsing function for a specific braced-block.
@ -119,11 +107,11 @@ impl Reparser {
///
/// Tokens must start with `{`, end with `}` and form a valid brace
/// sequence.
pub fn parse(self, tokens: &Tokens, tree_sink: &mut dyn TreeSink) {
pub fn parse(self, tokens: &Tokens) -> TreeTraversal {
let Reparser(r) = self;
let mut p = parser::Parser::new(tokens);
r(&mut p);
let events = p.finish();
event::process(tree_sink, events);
event::process(events)
}
}