68 commits

Author	SHA1	Message	Date
Nikita Sivukhin	5ab69bcf20	rename speculative load to prefetch (docs already uses this terminology)	2025-12-16 13:59:39 +04:00
Nikita Sivukhin	7cdc17db00	add comment	2025-12-10 17:09:29 +04:00
Nikita Sivukhin	4a180dd659	improve error messages at sync engine	2025-12-10 17:08:24 +04:00
Nikita Sivukhin	42e1b75a4a	do not send request for zero pages	2025-12-09 16:01:24 +04:00
Nikita Sivukhin	617aa2d713	add comments	2025-12-09 14:41:56 +04:00
Nikita Sivukhin	f513f66f20	cargo clippy --fix	2025-12-09 14:20:32 +04:00
Nikita Sivukhin	dc1ff11a5b	add speculative load logic	2025-12-09 14:07:10 +04:00
Nikita Sivukhin	2244f411ca	load larger chunks from the remote	2025-12-09 13:13:44 +04:00
Jussi Saurio	826ca4d44d	chore: remove experimental_indexes feature flags	2025-12-08 13:00:37 +02:00
Nikita Sivukhin	6e30a3ce90	cargo fmt	2025-12-02 18:18:14 +04:00
Nikita Sivukhin	201f3a55f6	fix clippy	2025-12-02 18:16:57 +04:00
Nikita Sivukhin	43992bf2c4	fix sync engine and sdk kit	2025-12-02 17:22:35 +04:00
Nikita Sivukhin	4b7cd523f5	handle memory io properly in sync engine	2025-12-02 16:03:45 +04:00
Nikita Sivukhin	8f4b0ba3cd	slightly refactor sync engine	2025-12-01 22:51:22 +04:00
Nikita Sivukhin	3cad64e5e4	rename weird ProtocolIo to the SyncEngineIo	2025-11-28 21:40:52 +04:00
Pekka Enberg	1f79fbc22c	Merge 'Partial sync basic' from Nikita Sivukhin ... This PR implements basic support for partial sync. Right now the scope is limited to only `:memory:` IO and later will be properly expanded to the file based IO later. The main addition is `PartialDatabaseStorage` which make request to the remote server for missing local pages on demand. The main change is that now tursodatabase JS bindings accept optional "external" IO event loop which in case of sync will drive `ProtocolIo` internal work associated with remote page fetching tasks. Closes #3931	2025-11-13 16:38:04 +02:00
Nikita Sivukhin	740ff2b4a6	fix clippy	2025-11-12 16:46:15 +04:00
Nikita Sivukhin	15dafd46c1	replace turso_assert -> assert	2025-11-12 16:40:38 +04:00
Nikita Sivukhin	3d14092679	fix	2025-11-12 16:38:04 +04:00
Nikita Sivukhin	72089d2682	adjust compilation	2025-11-12 16:30:50 +04:00
Nikita Sivukhin	aa65cfd55d	update Cargo.toml	2025-11-12 16:14:14 +04:00
Nikita Sivukhin	a25e3e76eb	wip	2025-11-12 13:21:34 +04:00
Nikita Sivukhin	6f7edcaddd	agent review fixes	2025-11-12 12:32:45 +04:00
Nikita Sivukhin	b73ff13b88	add simple implementation of Sparse IO	2025-11-12 12:04:12 +04:00
Nikita Sivukhin	d519945098	make ArenaBuffer unsafe Send + Sync	2025-11-12 10:54:40 +04:00
Nikita Sivukhin	33375697d1	add partial database storage implementation	2025-11-12 10:53:25 +04:00
Nikita Sivukhin	a855a657aa	report network stats	2025-11-12 10:53:25 +04:00
Nikita Sivukhin	98db727a99	integrate extra io stepping logic to the JS bindings	2025-11-12 10:53:25 +04:00
Nikita Sivukhin	d42b5c7bcc	wip	2025-11-12 10:53:25 +04:00
Nikita Sivukhin	34f1072071	add hooks to plug partial sync in the sync engine	2025-11-12 10:53:25 +04:00
pedrocarlo	1db13889e3	Change Value::Text to use a Cow<'static, str> instead of Vec<u8>	2025-11-11 16:11:46 -03:00
Nikita Sivukhin	da61fa32b4	use dyn DatabaseStorage instead of DatabaseFile	2025-11-06 17:42:03 +04:00
Nikita Sivukhin	d013876998	cargo fmt	2025-10-29 16:46:51 +04:00
Nikita Sivukhin	82d54999b1	fix pull operation in sync engine ... - before we fetched pull generation and last_change_id from the remote during pull - which is incorrect because fetched information can be inconsistent with WAL updates we received from the server (latest server state can be in "future" compared to the WAL updates we got since we can make push in parallel with updates pull operation) - now we read information about "server state" (pull generation, last_change_id) directly from the local DB right after we applied changes from the remote which get us consistent view on the state considering WAL updates we got - also fetching remote in the pull is bad - since pull block writes and network call with unpredictable latency poorly affect writes to the database	2025-10-29 16:24:05 +04:00
pedrocarlo	5a7390735d	rename Completion functions	2025-10-06 11:07:06 -03:00
pedrocarlo	e93add6c80	remove dyn DatabaseStorage and replace it with DatabaseFile	2025-10-03 14:14:15 -03:00
Nikita Sivukhin	5ec2d96bc1	support defered sync in the engine	2025-10-02 16:17:36 +04:00
Nikita Sivukhin	7016d9456e	fix test in the database tape	2025-10-02 16:17:36 +04:00
Nikita Sivukhin	91aeadd940	apply create DDL operation with IF NOT EXISTS clause in order to make them idempotent	2025-10-02 15:56:34 +04:00
Pavan-Nambi	49d5141f2d	Merge remote-tracking branch 'origin/main' into cdc_fail_autoincrement	2025-09-24 18:06:02 +05:30
Nikita Sivukhin	57c3d783e1	fix clippy	2025-09-23 18:24:52 +04:00
Nikita Sivukhin	e89dac98f3	use hrana batch stmt with Not(IsAutocommit) condition for push operation	2025-09-23 16:33:40 +04:00
Pavan Nambi	f1ac855441	Merge branch 'main' into cdc_fail_autoincrement	2025-09-22 21:11:26 +05:30
Nikita Sivukhin	1185298670	fix replay generator	2025-09-19 13:19:30 +04:00
Nikita Sivukhin	160119b12e	propagate long poll timeout	2025-09-19 13:19:30 +04:00
Nikita Sivukhin	b0f60a29ba	guard meta with mutex and make all methods to accept shared reference and offload locking of sync engine methods to the external wrapper	2025-09-19 13:19:30 +04:00
Pavan-Nambi	020921f803	Merge remote-tracking branch 'upstream/main' into cdc_fail_autoincrement	2025-09-18 19:27:19 +05:30
rajajisai	e605aff31b	Merge branch 'main' into enc-page-1	2025-09-16 10:06:00 -04:00
rajajisai	89caa868f9	Encryption support for database header page	2025-09-16 10:04:30 -04:00
pedrocarlo	3c91ae206b	move as many dependencies as possible to workspace to avoid multiple versions of the same dependency	2025-09-15 17:19:36 -03:00