mirrors/datafusion-sqlparse
1
0
Fork 0
mirror of https://github.com/apache/datafusion-sqlparser-rs.git synced 2025-08-17 20:50:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
936 commits 3 branches 88 tags 113 MiB
Commit graph

62 commits

Author SHA1 Message Date
Jeffrey
2c20ec0be5
Support parsing scientific notation (such as 10e5) (#768) 2022-12-28 08:28:53 -05:00
Audun Skaugen
6d6eb4bc9b
Support json operators @> <@ #- @? and @@ 
postgres supports a bunch more json operators. See
https://www.postgresql.org/docs/15/functions-json.html

Skipping operators starting with a question mark for now, since those
are hard to distinguish from placeholders without more context.
 2022-12-14 14:54:56 -05:00
zidaye
6c545195e1
support create function definition with $$ (#755) 
* support create function definition using '2700775'

* fix warn
 2022-12-13 17:15:33 -05:00
Ankur Goyal
813f4a2eff
Introduce location tracking in the tokenizer and parser (#710) 
* Add locations

* Add PartialEq

* Add PartialEq

* Add some tests

* Fix rebase conflicts

* Fix clippy

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
 2022-12-05 14:47:42 -05:00
Han YANG
e22aa78315
fix: handle nested comments (#726) 2022-11-30 13:22:29 -05:00
Augusto Fotino
5e1d9f8d6e
Derive PartialOrd, Ord, and Copy whenever possible (#717) 
This allow other projects to use our structures inside others that need
those.
 2022-11-30 12:25:59 -05:00
Ophir LOJKINE
f2b0efec0c
clippy fixes (#656) 2022-10-07 16:41:33 -04:00
Ophir LOJKINE
604f755a59
Fix parse error on some prepared statement placeholders (#604) 
sqlparser can now parse all the prepared statement placeholders supported by SQLite:

 - ?
 - ?NNN
 - @VVV
 - :VVV
 - $VVV

See: https://www.sqlite.org/lang_expr.html#varparam

This does not break existing support for postgresql's '@' operator

Fixes #603
 2022-09-27 09:58:26 -04:00
Maciej Skrzypkowski
d4e5b4d5e8
Support National string literal with lower case n (#612) 
* National string literal with lower case n

It's used by Snowflake

* Corrected DB name

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
 2022-09-27 09:07:13 -04:00
Andrew Lamb
36de9a69c6
Update for new clippy ints (#571) 2022-08-11 17:18:46 -04:00
Mykhailo Bondarenko
c2ccc80c28
Count characters instead of bytes (#529) 
* Count characters instead of bytes

* cargo fmt

* add tests to PR #529
 2022-07-07 13:45:59 -04:00
Yoshiyuki Komazaki
17c238bda7
feat: Support double quoted string (#530) 2022-06-28 16:41:23 -04:00
Dmitry Patsura
2c0886d9fe
feat: Support escaped string literals (PostgreSQL) (#502) 
* feat: Support escaped string literals (PostgreSQL)

Signed-off-by: Dmitry Patsura <talk@dmtry.me>

* lint

* escape ', \r, \t

* Update src/ast/value.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Update src/tokenizer.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* test: two slashes

* remove dead code

* test: parsing error

* support generic dialect too (for DF)

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
 2022-05-25 14:42:14 -04:00
Alex Yaroslavsky
dd805e9a6b
Support unicode whitespace (#482) 
* Support unicode whitespace

* Add test
 2022-05-15 14:58:56 -04:00
Maciej Skrzypkowski
7fc6361fe8
Add Redshift dialect, handle square brackets properly (#471) 
* Redshift square bracket handling

We need to detect `[` or `"` for Redshift quotes around indentifier and at the same time exclude
treating JSON paths as indentifer

* RedshiftSqlDialect documentation update

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Renamed _chars to chars

* Fixed warnings

* Missing license

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
 2022-05-04 11:11:00 -04:00
Poonai
d035784bdf
Add support for postgres json operators ->, ->>, #>, and #>> (#458) 
* add support for postgres json selection

Signed-off-by: password <rbalajis25@gmail.com>

* fix clippy

Signed-off-by: password <rbalajis25@gmail.com>

* add support for postgres `#>` and `#>>` json operator

* fix clippy

Signed-off-by: poonai <rbalajis25@gmail.com>

* resolve comments

Signed-off-by: password <rbalajis25@gmail.com>
 2022-04-19 15:46:56 -04:00
gamife
899f91b1f6
feat: add arg placeholder (#420) 
Co-authored-by: gamife <gamife9886@gmail.com>
 2022-02-17 07:55:21 -05:00
Markus Westerlind
34fedf311d
fix: Handle double quotes inside quoted identifiers correctly (#411) 
* fix: Handle double quotes inside quoted identifiers correctly

This fixes #410 for standard SQL, however I don't know enough about other dialects to know if they
handle this differently. May need more extensive testing as well.

* refactor: Make quoted identifier parsing a seperate function

* test: Check that quoted identifier tokenization works

Added `pretty_assertions` so that the `assert_eq!` in the tokenization is readable

* test: Check that quoted identifiers work in mysql

* chore: cargo clippy
 2022-02-07 10:05:17 -05:00
Alex Vasilev
ea0eb1be61
fix: mysql backslash escaping (#373) 
* fix: mysql backslash escaping

* fixes
 2021-12-23 08:50:33 -05:00
Qinxuan Chen
c9f8a44b55
Move the keywords module (#352) 
* Move the keywords mod from dialect mod into the root of library

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* re-export keywords from dialect for backwards compatiblity

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
 2021-09-26 07:59:39 -04:00
John
014b82f03d
Add logical xor (#357) 2021-09-18 06:39:35 -04:00
Qinxuan Chen
a8901becc3
Rname Token::Mult to Token::Mul (#353) 
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
 2021-09-16 16:50:10 -04:00
Qinxuan Chen
3d6637c4f3
Improve tokenizer error (#347) 
* Improve tokenizer error

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Add test for TokenizerError

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
 2021-09-10 11:33:13 -04:00
Qinxuan Chen
d8adb1708c
Eliminate redundant string copy of Tokenizer (#343) 
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
 2021-09-02 09:20:31 +02:00
TheSchemm
70ffa4166c
Support Parsing of hexadecimal literals that start with 0x (#324) 
* Add support for 0x prefixed bin literal

* Add tests

* Fix formatting for test
 2021-08-25 12:57:32 -04:00
Qinxuan Chen
5bc109a7ad
Support no_std (#332) 
* Support `no_std`

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Fix typo

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
 2021-08-21 10:15:43 +02:00
Andrew Lamb
783bc21d29
Merge pull request #328 from b41sh/regexp_match 
Add support for PostgreSQL regex match
 2021-08-20 13:54:24 -04:00
Qinxuan Chen
67e17b27f5
Make clippy happy (#330) 
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
 2021-08-19 23:02:15 +02:00
b41sh
d312837d7a fmt 2021-08-18 22:40:40 +08:00
baishen
3c3bc3e97e
Update src/tokenizer.rs 
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
 2021-08-18 09:38:46 -05:00
b41sh
80759a4deb support regex match 2021-08-13 19:50:14 +08:00
Mike Seddon
e6e37b47db
Implement TRY_CAST (#299) 
Adds support for `TRY_CAST` and fixes a clippy error
 2021-03-21 23:26:16 +01:00
Daniël Heres
f40955ee82
Parse floats without leading number (#294) 
* Parse floats without leading number

* Move period token test

* Comments

* Enable test
 2021-02-08 08:11:01 +01:00
Stephen Carman
8a214f9919
Implement Hive QL Parsing (#235) 2021-02-04 12:53:20 -07:00
Dmitry Patsura
17f2930885
Introduce support for EXPLAIN [ANALYZE] [VERBOSE] <STATEMENT> syntax 
Introduce support for EXPLAIN [ANALYZE] [VERBOSE] <STATEMENT> syntax
 2020-12-28 12:22:03 +01:00
rhanqtl
9f772f03b0
Add support for Recursive CTEs (#278) 
i.e. `WITH RECURSIVE ... AS ( ... ) SELECT` - see https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#with-clause

Fixes #277
 2020-10-11 09:43:51 +03:00
Alex Dukhno
926b03a31d
Add parsing for PostgreSQL math operators (#267) 2020-09-30 05:29:31 +03:00
eyalleshem
1c6077c0db
[snowflake] Support single line comments starting with '#' or '//' (#264) 
Co-authored-by: Eyal Leshem <eyal@satoricyber.com>
 2020-09-07 03:57:37 +03:00
eyalleshem
1cc3bf4099
Support named arguments in function invocations (#250) 
This commit supports functions with argument names.

the format is :
"Select some_function( a => exp, b => exp2 .. ) FROM table1
OR
"select * from table(function(a => exp)) f;"

see:
https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html#named-argument-assignment-token
or the motivating example from snowflake:
https://docs.snowflake.com/en/sql-reference/functions/flatten.html
 2020-08-02 08:04:55 +03:00
mz
09ca14fe8e
Support dialect-specific auto-increment column options for MySQL and SQLite (#234) 
In MySQL it's AUTO_INCREMENT
(see https://dev.mysql.com/doc/refman/8.0/en/create-table.html)
and in SQLite it's AUTOINCREMENT.

We use `ColumnOption::DialectSpecific(Vec<Token>)` to avoid adding a new variant for each vendor-specific column option.
 2020-07-28 23:34:21 +03:00
Daniël Heres
34548e890b
Change Word::keyword to a enum (#193) 
This improves performance and paves the way to future API enhancements as discussed in the PR https://github.com/andygrove/sqlparser-rs/pull/193
 2020-06-11 22:00:35 +03:00
Nickolay Ponomarev
0fe3a8ec39
Use Token::EOF instead of Option<Token> (#195) 
This simplifies codes slightly, removing the need deal with the EOF case explicitly.

The clone kludge in `_ => self.expected("date/time field",
Token::Word(w.clone())))` will become unnecessary once we stop using
a separate match for the keywords, as suggested in
https://github.com/andygrove/sqlparser-rs/pull/193#issuecomment-641607194
 2020-06-10 14:05:17 +03:00
Daniël Heres
d842f495db
Add line and column number to TokenizerError (#194) 
Addresses https://github.com/andygrove/sqlparser-rs/issues/179 for tokenize errors
 2020-06-10 09:15:44 +03:00
Nickolay Ponomarev
10b0b7f884
Update CHANGELOG (#192) 
Also remove a comment with a trailing space, which rustfmt doesn't like
 2020-06-07 20:43:44 +03:00
Daniël Heres
a42121de52
Use binary search to speed up matching keywords (#191) 2020-06-07 20:25:10 +03:00
Daniël Heres
b4699bd4a7
Support bitwise and, or, xor (#181) 
Operator precedence is coming from:

https://cloud.google.com/bigquery/docs/reference/standard-sql/operators
 2020-06-03 19:02:05 +03:00
Daniël Heres
00dc490f72
Support the string concat operator (#178) 
The selected precedence is based on BigQuery documentation, where it is equal to `*` and `/`:

https://cloud.google.com/bigquery/docs/reference/standard-sql/operators
 2020-06-02 21:24:30 +03:00
Nickolay Ponomarev
327e6cd9f1 Report an error for unterminated string literals 
...updated the TODOs regarding single-quoted literals parsing while at it.
 2020-05-10 21:21:01 +03:00
Alex Kyllo
172ba42001 Add support for MSSQL's SELECT TOP N syntax (#150) 
Add support for MSSQL SELECT TOP (N) [PERCENT] [WITH TIES] syntax.
 2020-01-12 23:20:48 -05:00
Nikhil Benesch
070d0192bf
Add missing license header 2019-08-13 15:04:42 -04:00