Eyal Leshem
5458a2b21d
Implement zero-copy tokenization for Word, SingleQuotedString, and Whitespace
...
Convert token string fields to use Cow<'a, str> to enable zero-copy tokenization
for commonly used tokens:
- Word.value: Regular identifiers and keywords now borrow from source
- SingleQuotedString: String literals borrow when no escape processing needed
- Whitespace: Single-line and multi-line comments borrow from source
Also add benchmark for measuring tokenization performance
2025-12-21 16:39:10 +02:00
Eyal Leshem
0f17b327b9
Add lifetime parameter to BorrowedToken for zero-copy tokenization
...
This change introduces a lifetime parameter 'a to BorrowedToken enum
to prepare for zero-copy tokenization support. This is a foundational
step toward reducing memory allocations during SQL parsing.
Changes:
- Added lifetime parameter to BorrowedToken<'a> enum
- Added _Phantom(Cow<'a, str>) variant to carry the lifetime
- Implemented Visit and VisitMut traits for Cow<'a, str> to support
the visitor pattern with the new lifetime parameter
- Fixed lifetime issues in visitor tests by using tokenized_owned()
instead of tokenize() where owned tokens are required
- Type alias Token = BorrowedToken<'static> maintains backward
compatibility
2025-12-18 17:23:16 +02:00
eyalleshem
c6933bb8ec
Immutable parser (Tokenize borrow strings cont) ( #2098 )
...
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Eyal Leshem <eyal@satoricyber.com>
2025-12-18 05:27:50 +01:00
eyalleshem
b098976cab
Reduce string copies cow ( #2075 )
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-12-04 09:45:15 +01:00
eyalleshem
c8acf9f52d
Prepare tokenizer for using borrowed strings instead of allocations. ( #2073 )
...
Rust / compile-no-std (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Eyal Leshem <eyal@satoricyber.com>
2025-11-25 11:25:51 +01:00
Luca Cappelletti
1114d6a2bc
Add PostgreSQL Operator DDL Support ( #2096 )
...
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-11-19 14:19:39 +00:00
r1b
1198c1ad11
fix: parse error on unnamed arg with default syntax ( #2091 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-11-18 09:41:48 +00:00
xitep
4f79997c8e
Include DML keyword in statement span ( #2090 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-11-13 09:27:58 +00:00
Luca Cappelletti
78be8b178b
Complete PostgreSQL CREATE TYPE Support ( #2094 )
...
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-11-13 09:25:27 +00:00
Christopher Watford
a00d5cd967
feat: Add support for SET SESSION AUTHORIZATION #2086 ( #2087 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-11-12 09:52:15 +00:00
etgarperets
f69407b344
Add support for INSERT INTO VALUE ( #2085 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-11-11 08:10:42 +00:00
Andriy Romanov
c439ee9419
Add snowflake dynamic table parsing ( #2083 )
2025-11-11 08:10:37 +00:00
Christopher Watford
eabde4b41e
feat: Add RESET to the base dialect #2078 ( #2079 )
2025-11-11 08:10:32 +00:00
Alexander Beedie
308a7231bc
Make BitwiseNot ("~") available for all dialects, not just PostgreSQL ( #2081 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-10-30 11:57:25 +00:00
Andriy Romanov
bc6e1d6e1a
Added TIMESTAMP_NTZ type support with precision to Snowflake dialect ( #2080 )
2025-10-30 09:20:41 +00:00
Thomas Kluyver
67684c84d4
SQLite: make period optional for CREATE TRIGGER ( #2071 )
...
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-10-22 09:12:06 +00:00
Yoav Cohen
6b352eaffd
Redshift: more copy options ( #2072 )
2025-10-22 09:11:19 +00:00
Luca Cappelletti
9cc9f9ab38
Refactored ColumnOption::Unique to reuse UniqueConstraint and PrimaryKeyConstraint ( #2064 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-10-21 08:05:02 +00:00
Luca Cappelletti
f861566c8d
Reused CheckConstraint in ColumnOption ( #2063 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-10-16 08:38:50 +00:00
niebayes
218f43cf2d
chore: add stack overflow warning for Visitor and VisitorMut ( #2068 )
2025-10-16 08:37:01 +00:00
Luca Cappelletti
c8531d41a1
Added support for MATCH syntax and unified column option ForeignKey ( #2062 )
...
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-10-15 11:15:55 +00:00
Luca Cappelletti
4490c8c55c
Added support for SQLite triggers ( #2037 )
...
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-10-11 11:16:56 +00:00
Luca Cappelletti
cc595cfd84
Moving several struct variants out of Statement enum to allow for trait impls for specific sub-variants ( #2057 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-10-10 14:08:20 +00:00
Joris Bayer
e7d42f3d1a
Increase version of sqlparser_derive from 0.3.0 to 0.4.0 ( #2060 )
Rust / codestyle (push) Waiting to run
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-10-10 11:11:07 +00:00
Dmitrii Blaginin
35220ee38c
Require PR ( #2052 )
2025-10-10 11:10:13 +00:00
Andrew Harper
0fb3b6b11c
Add support for procedure parameter default values ( #2041 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-10-08 11:04:36 +00:00
Luca Cappelletti
8c82fc0a19
Moved constraint variant outside of TableConstraint enum ( #2054 )
...
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-10-08 11:03:26 +00:00
nick young
ade4082656
[databricks] update dialect to support grouping by with modifier ( #2047 )
...
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-10-04 06:35:09 +00:00
Mohamed Abdeen
7461d8bd02
MySQL: CREATE INDEX: allow USING clause before ON ( #2029 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-09-26 10:46:18 +00:00
Joey Hain
c0998832a2
Correctly tokenize nested comments in Databricks, Clickhouse, and ANSI ( #2044 )
2025-09-26 08:05:23 +00:00
Yoav Cohen
a430838974
Snowflake: ALTER USER and KeyValueOptions Refactoring ( #2035 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-09-25 19:59:11 +00:00
Ophir LOJKINE
54a24e76a9
Link to actual change logs in CHANGELOG.md ( #2040 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-09-23 11:34:24 +00:00
Marcelo Altmann
a15c70dff9
Add support for INVISIBLE columns in MySQL ( #2033 )
2025-09-23 11:07:50 +00:00
Andrew Lamb
3c61db5dc0
Prepare for 0.59.0: Changelog and update version ( #2039 )
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
license / Release Audit Tool (RAT) (push) Has been cancelled
2025-09-19 12:51:30 +00:00
Chen Chongchen
0b723147b6
feat: support postgres alter schema ( #2038 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-09-19 08:37:27 +00:00
Mohamed Abdeen
ea7f9026f7
MySQL: Add support for unsigned numeric types ( #2031 )
2025-09-19 08:04:56 +00:00
Ryan Schneider
f642dd573c
DuckDB: Allow quoted date parts in EXTRACT ( #2030 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-09-16 18:25:24 +00:00
Luca Cappelletti
7021561474
Implemented the From method for all clear variants in Statement ( #2028 )
2025-09-16 18:21:55 +00:00
Sidney Cammeresi
e3fbfd91b2
MySQL: Support CROSS JOIN constraint ( #2025 )
Rust / test (beta) (push) Has been cancelled
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-09-15 07:33:17 +00:00
Luca Cappelletti
280f51889f
Moved CreateTrigger and DropTrigger out of Statement enum ( #2026 )
...
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-09-15 07:26:46 +00:00
Luca Cappelletti
b8539a52af
Added derive trait Copy to OrderByOptions struct ( #2021 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-09-09 05:39:56 +00:00
Dmitrii Blaginin
befc95e8e8
MQ Test ( #2019 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-09-05 07:49:05 +00:00
Dmitrii Blaginin
dd3342e417
Merge queue Infrastructure in sqlparser-rs ( #2007 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-09-04 21:21:09 +01:00
etgarperets
cff28334be
Add support for string literal concatenation ( #2003 )
...
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
Co-authored-by: Ifeanyi Ubah <ify1992@yahoo.com>
2025-09-02 13:20:34 +02:00
Yoav Cohen
eca65742ba
Redshift: Add support for UNLOAD statement ( #2013 )
2025-09-02 12:21:04 +02:00
Simon Sawert
9b6f6de056
Allow wilrdacrd for all SEMANTIC_VIEW types ( #2017 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-08-29 09:28:51 +02:00
Simon Sawert
ee707c72a7
Support wildcard metrics for SEMANTIC_VIEW ( #2016 )
license / Release Audit Tool (RAT) (push) Waiting to run
Rust / codestyle (push) Waiting to run
Rust / lint (push) Waiting to run
Rust / benchmark-lint (push) Waiting to run
Rust / compile (push) Waiting to run
Rust / docs (push) Waiting to run
Rust / compile-no-std (push) Waiting to run
Rust / test (beta) (push) Waiting to run
Rust / test (nightly) (push) Waiting to run
Rust / test (stable) (push) Waiting to run
2025-08-28 12:18:03 +02:00
Yoav Cohen
bc478b0531
Snowflake: Minus char in stage name ( #2014 )
license / Release Audit Tool (RAT) (push) Has been cancelled
Rust / codestyle (push) Has been cancelled
Rust / lint (push) Has been cancelled
Rust / benchmark-lint (push) Has been cancelled
Rust / compile (push) Has been cancelled
Rust / docs (push) Has been cancelled
Rust / compile-no-std (push) Has been cancelled
Rust / test (beta) (push) Has been cancelled
Rust / test (nightly) (push) Has been cancelled
Rust / test (stable) (push) Has been cancelled
2025-08-26 21:25:10 +02:00
Adam Johnson
9f515bf8c3
Add support for PostgreSQL JSON function 'RETURNING' clauses ( #2001 )
2025-08-26 21:22:26 +02:00
Denys Tsomenko
cffff30961
Add SECURE keyword for views in Snowflake ( #2004 )
2025-08-26 21:20:13 +02:00
