The Ident type was previously an alias for a String. Turn it into a full
fledged struct, so that the parser can preserve the distinction between
identifier value and quote style already made by the tokenizer's Word
structure.
The SQL standard requires that numeric literals with a decimal point,
like 1.23, are represented exactly, up to some precision. That means
that parsing these literals into f64s is invalid, as it is impossible
to represent many decimal numbers exactly in binary floating point (for
example, 0.3).
This commit parses all numeric literals into a new `Value` variant
`Number(String)`, removing the old `Long(u64)` and `Double(f64)`
variants. This is slightly less convenient for downstream consumers, but
far more flexible, as numbers that do not fit into a u64 and f64 are now
representable.
The note about WindowFrameBound::Following being only valid "in
WindowFrame::end_bound" was both
- confusing, as it was based on the ANSI SQL syntax the parser doesn't
adhere to -- though it sounded like a promise about the AST one could
expect to get from the parser
- and incomplete, as the reality is that the bounds validation the SQL
engine might want to perform is more complex. For example Postgres
documentation says <https://www.postgresql.org/docs/11/sql-expressions.html#SYNTAX-WINDOW-FUNCTIONS>:
> Restrictions are that frame_start cannot be UNBOUNDED FOLLOWING,
> frame_end cannot be UNBOUNDED PRECEDING, and the frame_end choice
> cannot appear earlier in the above list of frame_start and frame_end
> options than the frame_start choice does — for example RANGE BETWEEN
> CURRENT ROW AND offset PRECEDING is not allowed. But, for example,
> ROWS BETWEEN 7 PRECEDING AND 8 PRECEDING is allowed, even though it
> would never select any rows.
It used to consume the `RParen` closing the encompassing `OVER (`, even
when no window frame was parsed, which confused me a bit, even though
I wrote it initially.
After fixing that, I took the opportunity to reduce nesting and
duplication a bit.
To use the new helper effectively, a few related changes were required:
- Each of the parse_..._list functions (`parse_cte_list`,
`parse_order_by_expr_list`, `parse_select_list`) was replaced with a
version that parses a single element of the list (e.g. `parse_cte`),
with their callers now using
`self.parse_comma_separated(Parser::parse_<one_element>)?`
- `parse_with_options` now parses the WITH keyword and a separate
`parse_sql_option` function (named after the struct it produces) was
added to parse a single k=v option.
- `parse_list_of_ids` is gone, with the '.'-separated parsing moved to
`parse_object_name`.
Custom comma-separated parsing is still used in:
- parse_transaction_modes (where the comma separator is optional)
- parse_columns (allows optional trailing comma, before the closing ')')
It turns out implementing Hash alone is not very useful, as
std::collection::HashMap keys are required to implement both Hash and
Eq.
Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
I realized a moment too late that I'd missed a type name in
when removing the "SQL" prefix from types in ac555d7e8. As far as I can
tell, this was the only oversight.
The rationale here is the same as the last commit: since this crate
exclusively parses SQL, there's no need to restate that in every type
name. (The prefix seems to be an artifact of this crate's history as a
submodule of Datafusion, where it was useful to explicitly call out
which types were related to SQL parsing.)
This commit has the additional benefit of making all type names
consistent; over type we'd added some types which were not prefixed with
"SQL".
The ASTNode enum was confusingly named. In the past, the name made
sense, as the enum contained nearly all of the nodes in the AST, but
over time, pieces have been split into different structs, like
SQLStatement and SQLQuery. The ASTNode enum now contains only contains
expression nodes, so Expr is a better name.
Also rename the UnnamedExpression and ExpressionWithAlias variants
of SQLSelectItem to UnnamedExpr and ExprWithAlias, respectively, to
match the new shorthand for the word "expression".
T-SQL (and Oracle) support non-standard syntax, which is similar in
functionality to LATERAL joins in ANSI and PostgreSQL
<https://blog.jooq.org/tag/lateral-derived-table/>: it allows to use
the columns from the tables defined to the left of `APPLY` in the
"derived tables" (subqueries) to the right of `APPLY`. Unlike ANSI
LATERAL (but like Postgres' implementation), APPLY is also used with
table-valued function calls.
Despite them being similar, we represent "APPLY" joins with
`JoinOperator`s of its own (`CrossApply` and `OuterApply`). Doing
otherwise seemed like it would cause unnecessary confusion, as those
interested in dialect-specific parsing would probably not expect APPLY
being parsed as LATERAL, and those wanting to forbid non-standard SQL
would not be helped by this either.
This also renames existing JoinOperator::Cross -> CrossJoin to avoid
confusion with CrossApply.
`SELECT * FROM a OUTER JOIN b` was previously being parsed as an inner
join where table `a` was aliased to `OUTER`. This is extremely
surprising, as the user likely intended to say FULL OUTER JOIN. Since
the SQL specification lists OUTER as a keyword, we are well within our
rights to return an error here.
The SQL specification prohibits constructions like
SELECT * FROM a NATURAL JOIN (b)
where b sits alone inside parentheses. Parentheses in a FROM entry
always introduce either a derived table or a join.
This commit adds support for derived tables (i.e., subqueries) that
incorporate set operations, like:
SELECT * FROM (((SELECT 1) UNION (SELECT 2)) t1 AS NATURAL JOIN t2)
This introduces a bit of complexity around determining whether a left
paren starts a subquery, starts a nested join, or belongs to an
already-started subquery. The details are explained in a comment within
the patch.
`parse_value` handles parsing a non-integer value in expression context,
and we use parse_literal_uint() when expecting a number in other
contexts (such as `FETCH FIRST ... ROWS`)
