543 commits

Author	SHA1	Message	Date
Pekka Enberg	b4a331da38	Merge 'SDK tweaks' from Nikita Sivukhin ... Few small tweaks in SDKs: 1. Export `ConnectionSync` type from Python SDK 2. Remove dependency on `conn_raw_api` and corresponding methods from rust bindings - no one use them so let's drop support for this feature completely 3. Add Waker dependency in the sdk-kit in order to integrate it later to rust bindings 4. Add io_uring feature to the sdk-kit Closes #4284	2025-12-22 11:59:02 +02:00
Pere Diaz Bou	55de098ad3	Merge 'Local sync server' from Nikita Sivukhin ... This PR introduces local sync server and run tests for Go and Python against it in CI The local sync server implements 2 endpoints: 1. `/v2/pipeline` - subset of SQL over HTTP protocol (Hrana) to execute logical push operations from the client 2. `/pull-updates` - endpoint which returns page updates for client to apply locally The implementation is based on the local database file with **disabled checkpoint** in order to preserve whole DB history and allow server to respond to client which can have arbitrary stale DB. For implementation, sync server uses extra API exposed by the turso-core under `conn-raw-api` feature which includes `wal_state` / `wal_get_frame` methods. Usage: - `tursodb --sync-server 0.0.0.0:8080` - in-memory database - `tursodb local.db --sync-server 0.0.0.0:8080` - local db file Closes #4191	2025-12-19 11:05:12 +01:00
Pekka Enberg	3009ae14e0	Turso 0.4.0-pre.18	2025-12-19 09:57:09 +02:00
Nikita Sivukhin	327562fdf3	Merge branch 'main' into local-sync-server	2025-12-18 15:13:56 +04:00
Nikita Sivukhin	e5e7f68681	Merge branch 'main' into sdk-tweaks	2025-12-18 12:30:09 +04:00
Nikita Sivukhin	f4a1a91ba2	remove unnecessary dependencies	2025-12-18 12:22:06 +04:00
Jussi Saurio	770df9068e	uncommitted cargo lock change	2025-12-17 17:16:37 +02:00
Pekka Enberg	fcfcc04550	Merge 'Add dotnet bindings to Turso' from Kopylov Dmitriy ... Added C#/dotnet bindings for turso Reviewed-by: Nikita Sivukhin (@sivukhin) Closes #3594	2025-12-17 16:45:55 +02:00
Pere Diaz Bou	77841042d0	Merge 'Consider Order by expressions collation when deciding candidate index for iteration' from Pedro Muniz ... ## Description Does solve #4154, but I don't want to close it with this PR, because it does not solve the Affinity issue. We can only use an index to iterate over if the column collation in the order by clause matches the index collation <!-- Please include a summary of the changes and the related issue. --> ## Motivation and context Fix a bug in the optimizer <!-- Please include relevant motivation and context. Link relevant issues here. --> ## Description of AI Usage Used AI to write tests, fuzzers, and help me understand the optimizer code. Test prompt: <details> can you write tests in tcl that test that the correct collation sequence is properly maintained. ``` CREATE TABLE "t1" ("c1" TEXT COLLATE RTRIM); INSERT INTO "t1" VALUES (' '); CREATE INDEX "i1" ON "t1" ("c1" COLLATE RTRIM DESC); INSERT INTO "t1" VALUES (1025.1655084065987); SELECT "c1", typeof(c1) FROM "t1" ORDER BY "c1" COLLATE BINARY DESC, rowid ASC; ``` this is an example of a query that returned incorrect results because of this </details> <!-- Please disclose how AI was used to help create this PR. For example, you can share prompts, specific tools, or ways of working that you took advantage of. You can also share whether the creation of the PR was mainly driven by AI, or whether it was used for assistance. This is a good way of sharing knowledge to other contributors about how we can work more efficiently with AI tools. Note that the use of AI is encouraged, but the committer is still fully responsible for understanding and reviewing the output. --> Reviewed-by: Pere Diaz Bou <pere-altea@homail.com> Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com> Closes #4248	2025-12-17 14:26:25 +01:00
Pekka Enberg	7fd4b0ed63	Turso 0.4.0-pre.17	2025-12-17 10:18:19 +02:00
pedrocarlo	4c157e8c7a	add AI fuzz tests	2025-12-16 15:27:07 -03:00
Pekka Enberg	6043413068	Merge 'stress: Make random seed configurable' from Pekka Enberg ... When we're not running under Antithesis, allow the user to specify a seed for random number generation, which impacts the SQL operations we do. Although not deterministic, this makes reproducing some issues easier. Also, add a "scripts/run-until-fail.sh", which you can use to discover interesting seeds. For example, you can run ``` ./scripts/run-until-fail.sh cargo run -p turso_stress -- -t1 ``` to find a bug and then just copy-paste the reported seed to attempt to reproduce it. Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com> Closes #4227	2025-12-16 07:15:15 +02:00
pedrocarlo	adee0e11d6	simplify tokenization of cli sql	2025-12-15 16:54:00 -03:00
Pekka Enberg	9288471525	stress: Make random seed configurable ... When we're not running under Antithesis, allow the user to specify a seed for random number generation, which impacts the SQL operations we do. Although not deterministic, this makes reproducing some issues easier. Also, add a "scripts/run-until-fail.sh", which you can use to discover interesting seeds. For example, you can run ``` ./scripts/run-until-fail.sh cargo run -p turso_stress -- -t1 ``` to find a bug and then just copy-paste the reported seed to attempt to reproduce it.	2025-12-15 17:56:33 +02:00
Pekka Enberg	574c8258b0	Turso 0.4.0-pre.16	2025-12-15 09:58:23 +02:00
Pekka Enberg	944664a16e	Turso 0.4.0-pre.15	2025-12-15 09:12:29 +02:00
Nikita Sivukhin	f07a51968a	implement simple sync server in the CLI	2025-12-12 17:36:04 +04:00
Pekka Enberg	b6d4e5ff69	Turso 0.4.0-pre.14	2025-12-11 14:33:40 +02:00
Pekka Enberg	8f4c2f558d	Turso 0.4.0-pre.13	2025-12-11 13:34:47 +02:00
Pekka Enberg	b0400a3ed6	Turso 0.4.0-pre.12	2025-12-11 12:58:26 +02:00
Pekka Enberg	a0123f62e5	Turso 0.4.0-pre.11	2025-12-10 16:14:58 +02:00
Pekka Enberg	1ce09b26ea	Turso 0.4.0-pre.10	2025-12-10 15:09:56 +02:00
Pekka Enberg	01fc636da4	Turso 0.4.0-pre.9	2025-12-09 14:55:30 +02:00
Pekka Enberg	d2e26f944c	Turso 0.4.0-pre.8	2025-12-09 14:39:23 +02:00
PThorpe92	d81813990e	Bloom filter fixes	2025-12-04 07:39:34 -05:00
Nikita Sivukhin	510a61b5eb	Merge branch 'main' into sync-sdk-kit	2025-12-03 21:16:15 +04:00
pedrocarlo	e26c663616	do not pass mv store if we are in a bootstrap connection	2025-12-03 10:10:02 -03:00
Preston Thorpe	a26d34590b	Merge 'translate/vdbe: add bloom filter' from Preston Thorpe ... This PR adds a bloom filter to the program state so we can build it along with ephemeral indexes (and later, hashtables), to prevent having to do a BTree Seek for each comparison. Right now this looks like it's speeding up queries using ephemeral indexes by ~25% ```sql select users.id, products.id from users join products on users.first_name = products.name; ``` ### Main: `84ms` ### With Bloom filters: `64ms` ```sql explain select users.id, products.id from users join products on users.first_name = products.name; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 29 0 0 Start at 29 1 OpenRead 0 2 0 0 table=users, root=2, iDb=0 2 OpenRead 1 3 0 0 table=products, root=3, iDb=0 3 Rewind 0 28 0 0 Rewind table users 4 Once 14 0 0 0 goto 14 5 OpenAutoindex 2 0 0 0 cursor=2 6 Rewind 1 7 0 0 Rewind table products 7 Column 1 1 3 0 r[3]=products.name 8 RowId 1 4 0 0 r[4]=products.rowid 9 RowId 1 5 0 0 r[5]=products.rowid 10 MakeRecord 3 3 6 0 r[6]=mkrec(r[3..5]); for ephemeral_products_t2 11 FilterAdd 2 3 0 0 bloom_filter_add(3) 12 IdxInsert 2 6 3 0 key=r[6] 13 Next 1 7 0 0 14 Column 0 1 7 0 r[7]=users.first_name 15 IsNull 7 27 0 0 if (r[7]==NULL) goto 27 16 Filter 2 27 7 0 if !contains(7) goto 27 17 SeekGE 2 27 7 0 key=[7..7] 18 IdxGT 2 27 7 0 key=[7..7] 19 DeferredSeek 2 1 0 0 20 Column 0 1 9 0 r[9]=users.first_name 21 Column 2 0 10 0 r[10]=ephemeral_products_t2.name 22 Ne 9 10 26 Binary 0 if r[9]!=r[10] goto 26 23 RowId 0 1 0 0 r[1]=users.rowid 24 IdxRowId 2 2 0 0 r[2]=cursor 2 for index ephemeral_products_t2.rowid 25 ResultRow 1 2 0 0 output=r[1..2] 26 Next 2 18 0 0 27 Next 0 4 0 0 28 Halt 0 0 0 0 29 Transaction 0 1 28 0 iDb=0 tx_mode=Read 30 Goto 0 1 0 0 ``` Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com> Closes #4072	2025-12-03 07:44:41 -05:00
Pekka Enberg	2e3161942e	Turso 0.4.0-pre.7	2025-12-03 10:47:06 +02:00
Dmitriy	1a13b293c2	Fix after rebase	2025-12-03 12:44:21 +04:00
Dmitriy	9dc72df6b8	Fix build after rebase	2025-12-03 12:23:11 +04:00
Dmitriy	ff75ff766a	Add dotnet bindings	2025-12-03 12:23:10 +04:00
Pekka Enberg	0352247783	Turso 0.4.0-pre.6	2025-12-03 10:05:01 +02:00
Pekka Enberg	34964680ca	Turso 0.4.0-pre.5	2025-12-03 09:39:38 +02:00
PThorpe92	d9957b239f	Fix Cargo.toml/.lock	2025-12-02 16:15:17 -05:00
PThorpe92	e4c1474746	update Cargo.lock	2025-12-02 15:48:10 -05:00
PThorpe92	bf097a4fd4	Add plumbing for Bloom filter	2025-12-02 15:42:36 -05:00
Avinash Sajjanshetty	284db67025	Update AEGIS crate version	2025-12-02 23:12:22 +05:30
Nikita Sivukhin	5bb5bae01c	Merge branch 'main' into sync-sdk-kit	2025-12-02 17:28:06 +04:00
Nikita Sivukhin	8df7f7e416	adjust python bindings	2025-12-02 16:03:06 +04:00
Nikita Sivukhin	52f492b93d	slightly adjust database sdk-kit	2025-12-01 22:55:15 +04:00
Pekka Enberg	730836f3f3	Turso 0.4.0-pre.4	2025-12-01 19:49:10 +02:00
Nikita Sivukhin	73a94910d8	Merge branch 'main' into sdk-kit	2025-11-28 02:56:01 +04:00
Jussi Saurio	ea905d276c	Merge 'Add #[turso_macros::test] to automatically create tests that can run MVCC with minimal code changes' from Pedro Muniz ... - added procedural macro that creates Rust tests and with just a flag, creates a new test that runs the same with MVCC enabled - migrated almost all tests to use this new macro and added the mvcc flag to the tests that were not failing - added a `TempDatabase` builder to facilitate the proc_macro to generate the correct database options Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com> Closes #3991	2025-11-27 21:53:09 +02:00
Jussi Saurio	e3c2518259	mvcc: fix some test issues ... 1. BTreeCursors were initialized with negative table id - not sure how this worked before. 2. modification_during_scan test was also working incorrectly as far as I can tell	2025-11-27 12:19:09 +02:00
Nikita Sivukhin	e83327ddae	rename capi -> sdk-kit and implement both rsapi and capi	2025-11-26 17:10:01 +04:00
Nikita Sivukhin	bcdc877164	add statement finalize api	2025-11-25 17:45:06 +04:00
Nikita Sivukhin	8fabe12f5d	add turso capi bindings signatures	2025-11-25 12:56:10 +04:00
Pekka Enberg	b5c5666a49	Turso 0.4.0-pre.3	2025-11-22 09:56:18 +02:00
pedrocarlo	696bf2252c	create a basic test proc macro that takes a db path and an mvcc flag	2025-11-19 11:49:53 -03:00