Commit graph

81 commits

Author SHA1 Message Date
Daniël Heres
94ff46802c
Support ANALYZE TABLE syntax (#285)
* Support analyze table

* Cleanup
2020-12-28 10:08:32 -07:00
Dmitry Patsura
17f2930885
Introduce support for EXPLAIN [ANALYZE] [VERBOSE] <STATEMENT> syntax
Introduce support for EXPLAIN [ANALYZE] [VERBOSE] <STATEMENT> syntax
2020-12-28 12:22:03 +01:00
Nickolay Ponomarev
929fc6764f
Merge pull request #260 from eyalleshem/single_tables_in_parens
[snowflake] Support `FROM (table_name) alias`
2020-10-13 09:59:38 +03:00
Nickolay Ponomarev
ad72cda6b0 [snowflake] Support specifying an alias after FROM (table_factor)
Snowflake diverges from the standard and from most of the other
implementations by allowing extra parentheses not only around a join,
but around lone table names (e.g. `FROM (mytable [AS alias])`) and
around derived tables (e.g. `FROM ((SELECT ...)  [AS alias])`) as well.

Initially this was implemented in https://github.com/ballista-compute/sqlparser-rs/issues/154
by (ab)using `TableFactor::NestedJoin` to represent anything nested in
extra set of parens.

Afterwards we learned in https://github.com/ballista-compute/sqlparser-rs/issues/223
that in cases of such extraneous nesting Snowflake allows specifying the
alias both inside and outside parens, but not both - consider:

    FROM (table_factor AS inner_alias) AS outer_alias

We've considered implementing this by changing `TableFactor::NestedJoin`
to a `TableFactor::Nested { inner: TableWithJoins, alias:
Option<TableAlias> }`, but that seemed too generic, as no known dialect
supports duplicate aliases, as shown above, nor naming nested joins
`(foo NATURAL JOIN bar) alias`. So we decided on making a smaller change
(with no modifications to the AST), that is also more appropriate to the
contributors to the Snowflake dialect:


1) Revert #154 by rejecting `FROM (table or derived table)` in most dialects.

2) For `dialect_of!(self is SnowflakeDialect | GenericDialect)` parse
and strip the extraneous parentheses, e.g.

   `(mytable) AS alias` -> `(mytable AS alias)`


Co-authored-by: Eyal Leshem <eyal@satoricyber.com>
2020-10-13 09:51:02 +03:00
rhanqtl
9f772f03b0
Add support for Recursive CTEs (#278)
i.e. `WITH RECURSIVE ... AS ( ... ) SELECT` - see https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#with-clause

Fixes #277
2020-10-11 09:43:51 +03:00
Alex Dukhno
1ac208307c
Support IF NOT EXISTS for CREATE SCHEMA (#276)
This is a Postgres-specific clause: https://www.postgresql.org/docs/12/sql-createschema.html

Also add a test for `DROP SCHEMA IF EXISTS schema_name`, which is already supported in the parser.
2020-10-02 17:35:20 +03:00
Alex Dukhno
926b03a31d
Add parsing for PostgreSQL math operators (#267) 2020-09-30 05:29:31 +03:00
Nickolay Ponomarev
66505ebf9e Don't fail parsing a column definition with unexpected tokens
Since PR https://github.com/ballista-compute/sqlparser-rs/pull/93
`parse_column_def` parses a set of column options in a loop, e.g. given:

```
                  _______ column_def _______
CREATE TABLE foo (bar INT NOT NULL DEFAULT 1, )
                          -------- ---------
                          option 1  option 2
````

it parses column options until it encounters one of the delimiter tokens

First when we only supported `CREATE TABLE`, the set of delimiters that
stopped the parsing used to be `Token::Comma | Token::RParen`.

Then we added support for `ALTER TABLE ADD COLUMN <column_def>`. Turns
out the parser started to bail if the statement ended with a semicolon,
while attempting to parse the semicolon as a column option, as we forgot
to add it to the set of delimiter tokens.

This was recently fixed in https://github.com/ballista-compute/sqlparser-rs/pull/246
by including Token::SemiColon to the list, but it felt wrong to have
to update this list, and to have a common list of delimiters for two
different contexts (CREATE TABLE with parens vs ALTER TABLE ADD COLUMN
without parens).

Also our current approach cannot handle multiple statements NOT
separated by a semicolon, as is common in MS SQL DDL. We don't
explicitly support it in `parse_statements`, but that's a use-case
like to keep in mind nevertheless.
2020-08-10 17:12:33 +03:00
Nickolay Ponomarev
23f5c7e7ce Move parse_column_def below parse_columns
...because in the section related to CREATE TABLE parsing, the callers
are defined above the callees.
2020-08-10 17:09:34 +03:00
eyalleshem
1b46e82eec
Enable dialect specific behaviours in the parser (#254)
* Change `Parser { ... }` to store the dialect used:
    `Parser<'a> { ... dialect: &'a dyn Dialect }`

    Thanks to @c7hm4r for the initial version of this submitted as
    part of https://github.com/ballista-compute/sqlparser-rs/pull/170

* Introduce `dialect_of!(parser is SQLiteDialect |  GenericDialect)` helper
    to branch on the dialect's type

* Use the new functionality to make `AUTO_INCREMENT` and `AUTOINCREMENT`
  parsing dialect-dependent.


Co-authored-by: Christoph Müller <pmzqxfmn@runbox.com>
Co-authored-by: Nickolay Ponomarev <asqueella@gmail.com>
2020-08-10 16:51:59 +03:00
eyalleshem
61431b087d
Support TABLE functions in FROM (#253)
Support `TABLE(...)` syntax in `FROM`, for example:

    select * from TABLE(SOME_FUNCTION(some_arg))

The ANSI spec allows routine invocations (and some other kinds of expressions we don't currently support) inside TABLE:
https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#PTF-derived-table
https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#table-function-derived-table
2020-08-05 08:59:43 +03:00
eyalleshem
1cc3bf4099
Support named arguments in function invocations (#250)
This commit supports functions with argument names.

the format is :
"Select some_function( a => exp, b => exp2 .. ) FROM table1
OR
"select * from table(function(a => exp)) f;"

see:
https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#named-argument-assignment-token
or the motivating example from snowflake:
https://docs.snowflake.com/en/sql-reference/functions/flatten.html
2020-08-02 08:04:55 +03:00
mz
9c1a5a781d
Don't fail parsing ALTER TABLE ADD COLUMN ending with a semicolon (#246)
This is a follow-up to https://github.com/ballista-compute/sqlparser-rs/pull/203
where ALTER TABLE ADD COLUMN support was initially implemented.

Fixes #233.
2020-07-31 18:10:53 +03:00
mz
4452f9bad1
Support specifying ASC/DESC in index columns (#249)
...by reusing `OrderByExpr` for `columns` in `Statement::CreateIndex`.

This supports SQLite's indexed-column syntax https://www.sqlite.org/syntax/indexed-column.html

MSSQL's (`ON <object> ( column [ ASC | DESC ] [ ,...n ] )`)
https://docs.microsoft.com/en-us/sql/t-sql/statements/create-index-transact-sql?view=sql-server-ver15

And most of PostgreSQL syntax (except for opclass):
`( { column_name | ( expression ) } [ COLLATE collation ] [ opclass ] [ ASC | DESC ] [ NULLS { FIRST | LAST } ] [, ...] )`
https://www.postgresql.org/docs/12/sql-createindex.html
2020-07-30 15:37:58 +03:00
mz
09ca14fe8e
Support dialect-specific auto-increment column options for MySQL and SQLite (#234)
In MySQL it's AUTO_INCREMENT
(see https://dev.mysql.com/doc/refman/8.0/en/create-table.html)
and in SQLite it's AUTOINCREMENT.

We use `ColumnOption::DialectSpecific(Vec<Token>)` to avoid adding a new variant for each vendor-specific column option.
2020-07-28 23:34:21 +03:00
Steven
8020b2e5f0
Add Postgres-specific PREPARE, EXECUTE and DEALLOCATE (#243)
Adds top-statements PREPARE, EXECUTE and DEALLOCATE for Postgres-specific feature prepared statement.
2020-07-28 12:01:52 +03:00
Daniël Heres
d2e4340a32
Support create or replace view/table (#239)
* Support create or replace table

* Support create or replace view

* Simplify create or replace table parser

* Add tests for create or replace external table and materialized view

* Formatting

* Address review comments

* Create error if we didn't see a (external) table or (materialized) view afer create or replace
2020-07-27 21:59:08 +02:00
Daniël Heres
583f22b929
Remove PostgreSQL version of assert (#229)
Remove PostgreSQL procedural assert statement. This also simplifies code somewhat.
2020-07-17 13:20:49 +02:00
Daniël Heres
c24b0e01db
Implement ASSERT statement (#226)
As supported by PostgreSQL and BigQuery (with some differences between them)
2020-07-16 17:28:03 +02:00
mz
a53f1d26ef
Support SQLite CREATE VIRTUAL TABLE (#209)
`CREATE VIRTUAL TABLE .. USING <module_name> (<module_args>)`

https://www.sqlite.org/lang_createvtab.html
2020-06-28 04:31:33 +03:00
mz
0c83e5d9e8
Support SQLite's WITHOUT ROWID in CREATE TABLE (#208)
Per https://sqlite.org/lang_createtable.html

Co-authored-by: mashuai <mashuai@bytedance.com>
2020-06-26 15:11:46 +03:00
Daniël Heres
15d5f71646
Add CREATE TABLE AS support (#206)
We parse it as a regular `CREATE TABLE` statement
followed by an `AS <query>`, which is how BigQuery works:
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_table_statement


ANSI SQL and PostgreSQL only support a plain list of columns
after the table name in a CTAS
    `CREATE TABLE t (a) AS SELECT a FROM foo`

We currently only allow specifying a full schema with data
types, or omitting it altogether.

https://www.postgresql.org/docs/12/sql-createtableas.html
https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#as-subquery-clause


Finally, when no schema is specified, we print empty parens after a
plain `CREATE TABLE t ();` as required by PostgreSQL, but skip them
in a CTAS: `CREATE TABLE t AS ...`. This affects serialization only,
the parser allows omitting the schema in a regular `CREATE TABLE` too
since the first release of the parser:
7d27abdfb4/src/sqlparser.rs (L325-L332)

Co-authored-by: Nickolay Ponomarev <asqueella@gmail.com>
2020-06-23 16:30:22 +03:00
Jovansonlee Cesar
26361fd854
Implement ALTER TABLE DROP COLUMN (#148)
This implements `DROP [ COLUMN ] [ IF EXISTS ] column_name [ CASCADE ]`
sub-command of `ALTER TABLE`, which is what PostgreSQL supports https://www.postgresql.org/docs/12/sql-altertable.html
(except for the RESTRICT option)

Co-authored-by: Nickolay Ponomarev <asqueella@gmail.com>
2020-06-16 23:39:52 +03:00
mz
faeb7d440a
Implement ALTER TABLE ADD COLUMN and RENAME (#203)
Based on sqlite grammar
https://www.sqlite.org/lang_altertable.html
2020-06-16 22:52:37 +03:00
Daniël Heres
b24dbe513c
Replace FromStr with normal parser function for FileFormat (#201)
The previous version accepted quoting file format keywords
(`STORED AS "TEXTFILE"`) and was inconsistent with the
way WindowFrameUnits was parsed.
2020-06-13 15:38:01 +03:00
Daniël Heres
68afa2a764
Make FileFormat case insensitive (#200) 2020-06-12 18:10:44 +03:00
Daniël Heres
f4fbd9b6b3
Take slice as input for parse_keywords (#199) 2020-06-12 02:10:17 +03:00
Max Countryman
6cdd4a146d
Support general "typed string" literals (#187)
Fixes #168 by enabling `DATE` and other keywords to be used as
identifiers when not followed by a string literal.

A "typed string" is our term for generalized version of `DATE '...'`/`TIME '...'`/
`TIMESTAMP '...'` literals, represented as `TypedString { data_type, value }`
in the AST.

Unlike DATE/TIME/TIMESTAMP literals, this is a non-standard extension
supported by PostgreSQL at least.

This is a port of MaterializeInc/materialize#3146

Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
Co-authored-by: Nickolay Ponomarev <asqueella@gmail.com>
2020-06-12 00:04:43 +03:00
Daniël Heres
34548e890b
Change Word::keyword to a enum (#193)
This improves performance and paves the way to future API enhancements as discussed in the PR https://github.com/andygrove/sqlparser-rs/pull/193
2020-06-11 22:00:35 +03:00
Nickolay Ponomarev
0fe3a8ec39
Use Token::EOF instead of Option<Token> (#195)
This simplifies codes slightly, removing the need deal with the EOF case explicitly.

The clone kludge in `_ => self.expected("date/time field",
Token::Word(w.clone())))` will become unnecessary once we stop using
a separate match for the keywords, as suggested in
https://github.com/andygrove/sqlparser-rs/pull/193#issuecomment-641607194
2020-06-10 14:05:17 +03:00
Max Countryman
846c52f450
Allow omitting units after INTERVAL (#184)
Alter INTERVAL to support postgres syntax

This patch updates our INTERVAL implementation such that the Postgres
and Redshfit variation of the syntax is supported: namely that 'leading
field' is optional.

Fixes #177.
2020-06-10 09:32:13 +03:00
Daniël Heres
d842f495db
Add line and column number to TokenizerError (#194)
Addresses https://github.com/andygrove/sqlparser-rs/issues/179 for tokenize errors
2020-06-10 09:15:44 +03:00
Daniël Heres
d32df527e6
Accept &str in Parse::parse_sql (#182)
It is more generic to accept a `&str` than a `String` in an API,
and avoids having to convert a string to a `String` when not
needed, avoiding a copy.
2020-06-03 23:31:41 +03:00
Daniël Heres
b4699bd4a7
Support bitwise and, or, xor (#181)
Operator precedence is coming from:

https://cloud.google.com/bigquery/docs/reference/standard-sql/operators
2020-06-03 19:02:05 +03:00
Daniël Heres
00dc490f72
Support the string concat operator (#178)
The selected precedence is based on BigQuery documentation, where it is equal to `*` and `/`:

https://cloud.google.com/bigquery/docs/reference/standard-sql/operators
2020-06-02 21:24:30 +03:00
Max Countryman
5f3c1bda01
Provide LISTAGG implementation (#174)
This patch provides an initial implemenation of LISTAGG[1]. Notably this
implemenation deviates from ANSI SQL by allowing both WITHIN GROUP and
the delimiter to be optional. We do so because Redshift SQL works this
way and this approach is ultimately more flexible.

Fixes #169.

[1] https://modern-sql.com/feature/listagg
2020-05-30 18:50:17 +03:00
QP Hou
418b9631ce
add nulls first/last support to order by expression (#176)
Following `<sort specification list>` from the standard https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#_10_10_sort_specification_list
2020-05-30 17:05:15 +03:00
Alex Dukhno
91f769e460 added create and drop schema 2020-05-28 19:50:16 +03:00
Christoph Müller
98f97d09db
Add support for "on delete cascade" column option (#170)
Specifically, `FOREIGN KEY REFERENCES <foreign_table> (<referred_columns>)`
can now be followed by `ON DELETE <referential_action>` and/or by
`ON UPDATE <referential_action>`.
2020-05-27 18:24:23 +03:00
Nickolay Ponomarev
320d2f2d05 Update CHANGELOG.md and a fix last-minute review nit 2020-05-27 05:04:22 +03:00
mashuai
5aacc5ebcd add create index and drop index support 2020-05-27 09:27:57 +08:00
Nickolay Ponomarev
327e6cd9f1 Report an error for unterminated string literals
...updated the TODOs regarding single-quoted literals parsing while at it.
2020-05-10 21:21:01 +03:00
Alex Dukhno
5ad578e3e5
Implement CREATE TABLE IF NOT EXISTS (#163)
A non-standard feature supported at least by Postgres

https://www.postgresql.org/docs/12/sql-createtable.html
2020-04-21 16:28:02 +03:00
Matt Jibson
c0b0b5924d Add support for OFFSET with the ROWS keyword
MySQL doesn't support the ROWS part of OFFSET. Teach the parser to
remember which variant it saw, including just ROW.
2020-04-19 20:06:08 -06:00
Nickolay Ponomarev
05a29212ff Update comments (follow-up to PR #155)
https://github.com/andygrove/sqlparser-rs/pull/155
2020-04-20 04:58:24 +03:00
Eyal Leshem
3255fd3ea8 Add support to to table_name inside parenthesis 2020-04-12 20:31:09 +03:00
Alex Kyllo
172ba42001 Add support for MSSQL's SELECT TOP N syntax (#150)
Add support for MSSQL SELECT TOP (N) [PERCENT] [WITH TIES] syntax.
2020-01-12 23:20:48 -05:00
Robert Grimm
b1cbc55128
Turn type Ident into struct Ident
The Ident type was previously an alias for a String. Turn it into a full
fledged struct, so that the parser can preserve the distinction between
identifier value and quote style already made by the tokenizer's Word
structure.
2019-10-20 00:16:41 -04:00
gaffneyk
2bb38c9b27
Parse START TRANSACTION when followed by a semicolon
Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
2019-09-13 22:54:21 -04:00
Nikhil Benesch
e9c5567b04
Merge pull request #135 from andygrove/show-columns
Support MySQL `SHOW COLUMNS` statement
2019-09-02 07:40:57 -04:00