Commit graph

84 commits

Author SHA1 Message Date
Lukasz Stefaniak
7b3cc18229
snowflake: Fix handling of /~% in the stage name (#1009) 2023-10-26 15:23:57 -04:00
Lukasz Stefaniak
4903bd4b8b
Add test for clickhouse: tokenize == as Token::DoubleEq (#981) 2023-10-02 07:39:44 -04:00
Ilya
71c35d4dfd
Add support for == operator for Sqlite (#970) 2023-09-19 21:31:11 -04:00
dawg
b02c3f87ec
feat: show location info in parse errors (#958) 2023-09-07 16:23:09 -04:00
Andrew Lamb
3a412152b9
fix parsing of identifiers after % symbol (#927) 2023-07-21 05:55:41 -04:00
canalun
f98a2f9dca
feat: mysql no-escape mode (#870)
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-07-19 16:36:52 -04:00
Andrew Lamb
c45451850c
Clean up JSON operator tokenizing code (#923) 2023-07-17 16:09:55 -04:00
Igor Izvekov
df45db1375
fix: parsing JsonOperator (#913) 2023-07-17 15:03:48 -04:00
Igor Izvekov
a50671d95d
feat: support PGOverlap operator (#912) 2023-07-06 09:27:18 -04:00
dawg
75f18ecfda
Add support for DuckDB's CREATE MACRO statements (#897) 2023-06-21 15:12:58 -04:00
eitsupi
33b12acce7
feat: add DuckDB dialect (#878)
* feat: add DuckDB dialect

* formatting

* fix conflict

* support // in GenericDialect

* add DucDbDialect to all_dialects

* add comment from suggestion

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* fix: support // in GenericDialect

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-05-18 14:57:29 -04:00
AviRaboah
3e92ad349f
Support identifiers beginning with digits in MySQL (#856) 2023-04-26 09:27:04 -04:00
Coby Geralnik
04d9f3af2e
Added support for Mysql Backslash escapes (enabled by default) (#844) 2023-04-10 09:56:01 -04:00
Maciej Skrzypkowski
a1b7341b87
Non-Latin characters support (#840)
* Non latin characters

---------

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>

* Test for mysql

---------

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
2023-03-23 07:07:17 -04:00
sam
a8a8e65b7c
PostgreSQL: GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY and GENERATED ALWAYS AS ( generation_expr ) support (#832)
* GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( sequence_options ) basic impl - test are failing.

* PostgreSQL GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( sequence_options ) and GENERATED ALWAYS AS ( generation_expr ) STORED implementation.
2023-03-16 05:54:00 -04:00
Y Togami
58de3c1222
feat: support raw string literal of BigQuery (#812)
* add tests

* feat: parse raw literal of bq

* merge double quoted & single quoted to raw string literal

* Update src/ast/value.rs

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-03-01 14:11:42 -05:00
Y Togami
0c0d088ec2
feat: support byte string literal in bq (#802)
* rebase

* review

* lint
2023-02-19 10:38:03 -05:00
Maciej Skrzypkowski
488e8a8156
Support MySQL Character Set Introducers (#788)
* MySQL Character Set Introducers

* Documentation fix

* Parsing string introducer from Token::word

* Fixed lint

* fix clippy

---------

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-02-17 13:38:43 -05:00
Y Togami
b31ede7733
chore: fix clippy error in ci (#803)
* chore: fix clippy error in ci

* chore: fix fmt
2023-02-17 13:24:50 -05:00
Ophir LOJKINE
524b8a7e7b
Add a mutable visitor (#782)
* Add a mutable visitor

This adds the ability to mutate parsed sql queries.
Previously, only visitors taking an immutable reference to the visited structures were allowed.

* add utility functions for mutable visits

* bump version numbers
2023-01-02 06:48:20 -05:00
Alex Vasilev
072ccc0d76
feat: dollar-quoted strings support (#772)
* feat: support dollar-quoted strings

* remove comment

* unused code

* removed debugging

* added tests

* fmt

* clippy

* updated tests
2022-12-29 07:52:17 -05:00
Raphael Taylor-Davies
dec3c2b818
Add derive based AST visitor (#765)
* Add derive based AST visitor

* Fix BigDecimal

* Fix no visitor feature

* Add test

* Rename visit_table to visit_relation

* Review feedback

* Add pre and post visit

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-12-28 10:07:12 -05:00
Jeffrey
2c20ec0be5
Support parsing scientific notation (such as 10e5) (#768) 2022-12-28 08:28:53 -05:00
Audun Skaugen
6d6eb4bc9b
Support json operators @> <@ #- @? and @@
postgres supports a bunch more json operators. See
https://www.postgresql.org/docs/15/functions-json.html

Skipping operators starting with a question mark for now, since those
are hard to distinguish from placeholders without more context.
2022-12-14 14:54:56 -05:00
zidaye
6c545195e1
support create function definition with $$ (#755)
* support create function definition using '2700775'

* fix warn
2022-12-13 17:15:33 -05:00
Ankur Goyal
813f4a2eff
Introduce location tracking in the tokenizer and parser (#710)
* Add locations

* Add PartialEq

* Add PartialEq

* Add some tests

* Fix rebase conflicts

* Fix clippy

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-12-05 14:47:42 -05:00
Han YANG
e22aa78315
fix: handle nested comments (#726) 2022-11-30 13:22:29 -05:00
Augusto Fotino
5e1d9f8d6e
Derive PartialOrd, Ord, and Copy whenever possible (#717)
This allow other projects to use our structures inside others that need
those.
2022-11-30 12:25:59 -05:00
Ophir LOJKINE
f2b0efec0c
clippy fixes (#656) 2022-10-07 16:41:33 -04:00
Ophir LOJKINE
604f755a59
Fix parse error on some prepared statement placeholders (#604)
sqlparser can now parse all the prepared statement placeholders supported by SQLite:

 - ?
 - ?NNN
 - @VVV
 - :VVV
 - $VVV

See: https://www.sqlite.org/lang_expr.html#varparam

This does not break existing support for postgresql's '@' operator

Fixes #603
2022-09-27 09:58:26 -04:00
Maciej Skrzypkowski
d4e5b4d5e8
Support National string literal with lower case n (#612)
* National string literal with lower case n

It's used by Snowflake

* Corrected DB name

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
2022-09-27 09:07:13 -04:00
Andrew Lamb
36de9a69c6
Update for new clippy ints (#571) 2022-08-11 17:18:46 -04:00
Mykhailo Bondarenko
c2ccc80c28
Count characters instead of bytes (#529)
* Count characters instead of bytes

* cargo fmt

* add tests to PR #529
2022-07-07 13:45:59 -04:00
Yoshiyuki Komazaki
17c238bda7
feat: Support double quoted string (#530) 2022-06-28 16:41:23 -04:00
Dmitry Patsura
2c0886d9fe
feat: Support escaped string literals (PostgreSQL) (#502)
* feat: Support escaped string literals (PostgreSQL)

Signed-off-by: Dmitry Patsura <talk@dmtry.me>

* lint

* escape ', \r, \t

* Update src/ast/value.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Update src/tokenizer.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* test: two slashes

* remove dead code

* test: parsing error

* support generic dialect too (for DF)

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-25 14:42:14 -04:00
Alex Yaroslavsky
dd805e9a6b
Support unicode whitespace (#482)
* Support unicode whitespace

* Add test
2022-05-15 14:58:56 -04:00
Maciej Skrzypkowski
7fc6361fe8
Add Redshift dialect, handle square brackets properly (#471)
* Redshift square bracket handling

We need to detect `[` or `"` for Redshift quotes around indentifier and at the same time exclude
treating JSON paths as indentifer

* RedshiftSqlDialect documentation update

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Renamed _chars to chars

* Fixed warnings

* Missing license

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-04 11:11:00 -04:00
Poonai
d035784bdf
Add support for postgres json operators ->, ->>, #>, and #>> (#458)
* add support for postgres json selection

Signed-off-by: password <rbalajis25@gmail.com>

* fix clippy

Signed-off-by: password <rbalajis25@gmail.com>

* add support for postgres `#>` and `#>>` json operator

* fix clippy

Signed-off-by: poonai <rbalajis25@gmail.com>

* resolve comments

Signed-off-by: password <rbalajis25@gmail.com>
2022-04-19 15:46:56 -04:00
gamife
899f91b1f6
feat: add arg placeholder (#420)
Co-authored-by: gamife <gamife9886@gmail.com>
2022-02-17 07:55:21 -05:00
Markus Westerlind
34fedf311d
fix: Handle double quotes inside quoted identifiers correctly (#411)
* fix: Handle double quotes inside quoted identifiers correctly

This fixes #410 for standard SQL, however I don't know enough about other dialects to know if they
handle this differently. May need more extensive testing as well.

* refactor: Make quoted identifier parsing a seperate function

* test: Check that quoted identifier tokenization works

Added `pretty_assertions` so that the `assert_eq!` in the tokenization is readable

* test: Check that quoted identifiers work in mysql

* chore: cargo clippy
2022-02-07 10:05:17 -05:00
Alex Vasilev
ea0eb1be61
fix: mysql backslash escaping (#373)
* fix: mysql backslash escaping

* fixes
2021-12-23 08:50:33 -05:00
Qinxuan Chen
c9f8a44b55
Move the keywords module (#352)
* Move the keywords mod from dialect mod into the root of library

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* re-export keywords from dialect for backwards compatiblity

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2021-09-26 07:59:39 -04:00
John
014b82f03d
Add logical xor (#357) 2021-09-18 06:39:35 -04:00
Qinxuan Chen
a8901becc3
Rname Token::Mult to Token::Mul (#353)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2021-09-16 16:50:10 -04:00
Qinxuan Chen
3d6637c4f3
Improve tokenizer error (#347)
* Improve tokenizer error

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Add test for TokenizerError

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2021-09-10 11:33:13 -04:00
Qinxuan Chen
d8adb1708c
Eliminate redundant string copy of Tokenizer (#343)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2021-09-02 09:20:31 +02:00
TheSchemm
70ffa4166c
Support Parsing of hexadecimal literals that start with 0x (#324)
* Add support for 0x prefixed bin literal

* Add tests

* Fix formatting for test
2021-08-25 12:57:32 -04:00
Qinxuan Chen
5bc109a7ad
Support no_std (#332)
* Support `no_std`

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Fix typo

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2021-08-21 10:15:43 +02:00
Andrew Lamb
783bc21d29
Merge pull request #328 from b41sh/regexp_match
Add support for PostgreSQL regex match
2021-08-20 13:54:24 -04:00
Qinxuan Chen
67e17b27f5
Make clippy happy (#330)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2021-08-19 23:02:15 +02:00