The name was confusing:
SQLStatement::SQLSelect(
SQLQuery {
body: SQLSetExpr::Select(SQLSelect)
}
)
Fix the `large_enum_variant` clippy lint for `SQLStatement::SQLQuery`
`SQLStatement::SQLCreateView`, and `SQLSetExpr::Select`, while we're
changing the AST anyway
https://rust-lang.github.io/rust-clippy/master/index.html#large_enum_variant
Also move more things to use SQLIdent instead of String in the hope of
making it a newtype eventually.
Add tests that quoted identifiers round-trip parsing/serialization correctly.
Store (and parse) `table_name: SQLObjectName` instead of
`relation: Option<Box<ASTNode>>`, which can be an arbitrary expression.
Also remove the `Option<>`: the table name is not optional in any dialects
I'm familiar with. While the FROM keyword itself _is_ optional in some
dialects, there are more things to implement for those dialects, see
https://stackoverflow.com/a/4484271/1026
...to match the name of the recently introduced `SQLObjectName` struct
and to avoid any reservations about using it with multi-part names of
objects other than tables (as in the `type_name` case).
(To store "A name of a table, view, custom type, etc., possibly
multi-part, i.e. db.schema.obj".)
Before this change
- some places used `String` for this (these are updated in this commit)
- while others (notably SQLStatement::SQLDelete::relation, which is
the reason for this series of commits) relied on
ASTNode::SQLCompoundIdentifier (which is also backed by a
Vec<SQLIdent>, but, as a variant of ASTNode enum, is not convenient
to use when you know you need that specific variant).
This makes the parser more strict when handling SELECTs nested
somewhere in the main statement:
1) instead of accepting SELECT anywhere in the expression where an
operand was expected, we only accept it inside parens. (I've added a
test for the currently supported syntax, <scalar subquery> in ANSI
SQL terms)
2) instead of accepting any expression in the derived table context:
`FROM ( ... )` - we only look for a SELECT subquery there.
Due to #1, I had to swith the 'ansi' test from invoking the expression
parser to the statement parser.
Before this change an expression like `(a+b)-(c+d)` was parsed correctly
(as a Minus node with two Plus nodes as children), but when serializing
back to an SQL string, it came up as a+b-c+d, since we don't store
parens in AST and don't attempt to insert them when necessary during
serialization. The latter would be hard, and we already had an SQLNested
enum variant, so I changed the code to wrap the AST node for the
parenthesized expression in it.
Before this commit there was a single `parse_expr(u8)` method, which
was called both
1) from within the expression parser (to parse subexpression consisting
of operators with higher priority than the current one), and
2) from the top-down parser both
a) to parse true expressions (such as an item of the SELECT list or
the condition after WHERE or after ON), and
b) to parse sequences which are not exactly "expressions".
This starts cleaning this up by renaming the `parse_expr(u8)` method to
`parse_subexpr()` and using it only for (1) - i.e. usually providing a
non-zero precedence parameter.
The non-intuitively called `parse()` method is renamed to `parse_expr()`,
which became available and is used for (2a).
While reviewing the existing callers of `parse_expr`, four points to
follow up on were identified (marked "TBD (#)" in the commit):
1) Do not lose parens (e.g. `(1+2)*3`) when roundtripping
String->AST->String by using SQLNested.
2) Incorrect precedence of the NOT unary
3) `parse_table_factor` accepts any expression where a SELECT subquery
is expected.
4) parse_delete uses parse_expr() to retrieve a table name
These are dealt with in the commits to follow.
Parser::parse_sql() can now parse a semicolon-separated list of
statements, returning them in a Vec<SQLStatement>.
To support this we:
- Move handling of inter-statement tokens from the end of individual
statement parsers (`parse_select` and `parse_delete`; this was not
implemented for other top-level statements) to the common
statement-list parsing code (`parse_sql`);
- Change the "Unexpected token at end of ..." error, which didn't have
tests and prevented us from parsing successive statements ->
"Expected end of statement" (i.e. a delimiter - currently only ";" -
or the EOF);
- Add PartialEq on ParserError to be able to assert_eq!() that parsing
statements that do not terminate properly returns an expected error.
(The primary motivation was that it makes the tests more resilient to
the upcoming changes to the SQLSelectStatement to support `AS` aliases
and `UNION`.)
Also start using `&'static str` literals consistently instead of
String::from for the `let sql` test strings.
Continuing from https://github.com/andygrove/sqlparser-rs/pull/33#issuecomment-453060427
This stops the parser from accepting (and the AST from being able to
represent) SQL look-alike code that makes no sense, e.g.
SELECT ... FROM (CREATE TABLE ...) foo
SELECT ... FROM (1+CAST(...)) foo
Generally this makes the AST less "partially typed": meaning certain
parts are strongly typed (e.g. SELECT can only contain projections,
relations, etc.), while everything that didn't get its own type is
dumped into ASTNode, effectively untyped. After a few more fixes (yet
to be implemented), `ASTNode` could become an `SQLExpression`. The
Pratt-style expression parser (returning an SQLExpression) would be
invoked from the top-down parser in places where a generic expression
is expected (e.g. after SELECT <...>, WHERE <...>, etc.), while things
like select's `projection` and `relation` could be more appropriately
(narrowly) typed.
Since the diff is quite large due to necessarily large number of
mechanical changes, here's an overview:
1) Interface changes:
- A new AST enum - `SQLStatement` - is split out of ASTNode:
- The variants of the ASTNode enum, which _only_ make sense as a top
level statement (INSERT, UPDATE, DELETE, CREATE, ALTER, COPY) are
_moved_ to the new enum, with no other changes.
- SQLSelect is _duplicated_: now available both as a variant in
SQLStatement::SQLSelect (top-level SELECT) and ASTNode:: (subquery).
- The main entry point (Parser::parse_sql) now expects an SQL statement
as input, and returns an `SQLStatement`.
2) Parser changes: instead of detecting the top-level constructs deep
down in the precedence parser (`parse_prefix`) we are able to do it
just right after setting up the parser in the `parse_sql` entry point
(SELECT, again, is kept in the expression parser to demonstrate how
subqueries could be implemented).
The rest of parser changes are mechanical ASTNode -> SQLStatement
replacements resulting from the AST change.
3) Testing changes: for every test - depending on whether the input was
a complete statement or an expresssion - I used an appropriate helper
function:
- `verified` (parses SQL, checks that it round-trips, and returns
the AST) - was replaced by `verified_stmt` or `verified_expr`.
- `parse_sql` (which returned AST without checking it round-tripped)
was replaced by:
- `parse_sql_expr` (same function, for expressions)
- `one_statement_parses_to` (formerly `parses_to`), extended to
deal with statements that are not expected to round-trip.
The weird name is to reduce further churn when implementing
multi-statement parsing.
- `verified_stmt` (in 4 testcases that actually round-tripped)
This will allow re-using it for SQLStatement in a later commit.
(Also split the new struct into a separate file, other query-related
types will be moved here in a follow-up commit.)
...as in `FROM foo bar WHERE bar.x > 1`.
To avoid ambiguity as to whether a token is an alias or a keyword, we
maintain a blacklist of keywords, that can follow a "table factor", to
prevent parsing them as an alias. This "context-specific reserved
keyword" approach lets us accept more SQL that's valid in some dialects,
than a list of globally reserved keywords. Also some dialects (e.g.
Oracle) apparently don't reserve some keywords (like JOIN), while
presumably they won't accept them as an alias (`FROM foo JOIN` meaning
`FROM foo AS JOIN`).
