b57c60a78c
Before this commit there was a single `parse_expr(u8)` method, which
was called both
1) from within the expression parser (to parse subexpression consisting
of operators with higher priority than the current one), and
2) from the top-down parser both
a) to parse true expressions (such as an item of the SELECT list or
the condition after WHERE or after ON), and
b) to parse sequences which are not exactly "expressions".
This starts cleaning this up by renaming the `parse_expr(u8)` method to
`parse_subexpr()` and using it only for (1) - i.e. usually providing a
non-zero precedence parameter.
The non-intuitively called `parse()` method is renamed to `parse_expr()`,
which became available and is used for (2a).
While reviewing the existing callers of `parse_expr`, four points to
follow up on were identified (marked "TBD (#)" in the commit):
1) Do not lose parens (e.g. `(1+2)*3`) when roundtripping
String->AST->String by using SQLNested.
2) Incorrect precedence of the NOT unary
3) `parse_table_factor` accepts any expression where a SELECT subquery
is expected.
4) parse_delete uses parse_expr() to retrieve a table name
These are dealt with in the commits to follow.
|
||
|---|---|---|
| docs | ||
| examples | ||
| src | ||
| tests | ||
| .gitignore | ||
| .travis.yml | ||
| Cargo.toml | ||
| LICENSE.TXT | ||
| README.md | ||
Extensible SQL Lexer and Parser for Rust
The goal of this project is to build a SQL lexer and parser capable of parsing SQL that conforms with the ANSI SQL:2011 standard but also making it easy to support custom dialects so that this crate can be used as a foundation for vendor-specific parsers.
This parser is currently being used by the DataFusion query engine and LocustDB.
Example
The current code is capable of parsing some trivial SELECT and CREATE TABLE statements.
let sql = "SELECT a, b, 123, myfunc(b) \
FROM table_1 \
WHERE a > b AND b < 100 \
ORDER BY a DESC, b";
let dialect = GenericSqlDialect{}; // or AnsiSqlDialect, or your own dialect ...
let ast = Parser::parse_sql(&dialect,sql.to_string()).unwrap();
println!("AST: {:?}", ast);
This outputs
AST: SQLSelect { projection: [SQLIdentifier("a"), SQLIdentifier("b"), SQLLiteralLong(123), SQLFunction { id: "myfunc", args: [SQLIdentifier("b")] }], relation: Some(SQLIdentifier("table_1")), selection: Some(SQLBinaryExpr { left: SQLBinaryExpr { left: SQLIdentifier("a"), op: Gt, right: SQLIdentifier("b") }, op: And, right: SQLBinaryExpr { left: SQLIdentifier("b"), op: Lt, right: SQLLiteralLong(100) } }), order_by: Some([SQLOrderBy { expr: SQLIdentifier("a"), asc: false }, SQLOrderBy { expr: SQLIdentifier("b"), asc: true }]), group_by: None, having: None, limit: None }
Design
This parser is implemented using the Pratt Parser design, which is a top-down operator-precedence parser.
I am a fan of this design pattern over parser generators for the following reasons:
- Code is simple to write and can be concise and elegant (this is far from true for this current implementation unfortunately, but I hope to fix that using some macros)
- Performance is generally better than code generated by parser generators
- Debugging is much easier with hand-written code
- It is far easier to extend and make dialect-specific extensions compared to using a parser generator
Supporting custom SQL dialects
This is a work in progress but I started some notes on writing a custom SQL parser.
Contributing
Contributors are welcome! Please see the current issues and feel free to file more!