Commit graph

93 commits

Author SHA1 Message Date
Nikita Sivukhin
5ab69bcf20 rename speculative load to prefetch (docs already uses this terminology) 2025-12-16 13:59:39 +04:00
Preston Thorpe
73289c6325
Merge 'Go driver' from Nikita Sivukhin
This PR brings back go driver to the turso monorepo and adds sync
support for the golang.
It depends on the pre-compiled binaries at
https://github.com/tursodatabase/turso-go-platform-libs repository
There is a CI which trigger build on the turso-go-platform-libs repo in
case when release tag is published

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #4085
2025-12-11 09:52:15 -05:00
Nikita Sivukhin
bd69bf2adb fix typo 2025-12-10 23:17:06 +04:00
Nikita Sivukhin
f409875c69 adjust python tests a little bit 2025-12-10 18:37:28 +04:00
Nikita Sivukhin
fd335d7f06 refine changes result usage and regenerate client 2025-12-10 18:23:45 +04:00
Nikita Sivukhin
2ea0709344 return null if no changes were fetched from remote 2025-12-10 18:23:45 +04:00
Nikita Sivukhin
a499906643 fix clippy 2025-12-10 18:23:44 +04:00
Nikita Sivukhin
cf350b917e fix use-after-free error with stats result 2025-12-10 18:23:44 +04:00
Nikita Sivukhin
6f3df5fa89 adjust header and capi bindings for turso-sync sdk kit 2025-12-10 18:23:41 +04:00
Nikita Sivukhin
7cdc17db00 add comment 2025-12-10 17:09:29 +04:00
Nikita Sivukhin
4a180dd659 improve error messages at sync engine 2025-12-10 17:08:24 +04:00
Nikita Sivukhin
42e1b75a4a do not send request for zero pages 2025-12-09 16:01:24 +04:00
Nikita Sivukhin
617aa2d713 add comments 2025-12-09 14:41:56 +04:00
Nikita Sivukhin
f513f66f20 cargo clippy --fix 2025-12-09 14:20:32 +04:00
Nikita Sivukhin
dc1ff11a5b add speculative load logic 2025-12-09 14:07:10 +04:00
Nikita Sivukhin
2244f411ca load larger chunks from the remote 2025-12-09 13:13:44 +04:00
Jussi Saurio
826ca4d44d chore: remove experimental_indexes feature flags 2025-12-08 13:00:37 +02:00
Nikita Sivukhin
6e30a3ce90 cargo fmt 2025-12-02 18:18:14 +04:00
Nikita Sivukhin
201f3a55f6 fix clippy 2025-12-02 18:16:57 +04:00
Nikita Sivukhin
43992bf2c4 fix sync engine and sdk kit 2025-12-02 17:22:35 +04:00
Nikita Sivukhin
4b7cd523f5 handle memory io properly in sync engine 2025-12-02 16:03:45 +04:00
Nikita Sivukhin
1da46bda6b more docs 2025-12-02 00:58:52 +04:00
Nikita Sivukhin
8f4b0ba3cd slightly refactor sync engine 2025-12-01 22:51:22 +04:00
Nikita Sivukhin
71be25fa14 add sdk-kit for turso-sync 2025-12-01 22:51:08 +04:00
Nikita Sivukhin
3cad64e5e4 rename weird ProtocolIo to the SyncEngineIo 2025-11-28 21:40:52 +04:00
Pekka Enberg
1f79fbc22c
Merge 'Partial sync basic' from Nikita Sivukhin
This PR implements basic support for partial sync. Right now the scope
is limited to only `:memory:` IO and later will be properly expanded to
the file based IO later.
The main addition is `PartialDatabaseStorage` which make request to the
remote server for missing local pages on demand.
The main change is that now tursodatabase JS bindings accept optional
"external" IO event loop which in case of sync will drive `ProtocolIo`
internal work associated with remote page fetching tasks.

Closes #3931
2025-11-13 16:38:04 +02:00
Nikita Sivukhin
740ff2b4a6 fix clippy 2025-11-12 16:46:15 +04:00
Nikita Sivukhin
15dafd46c1 replace turso_assert -> assert 2025-11-12 16:40:38 +04:00
Nikita Sivukhin
3d14092679 fix 2025-11-12 16:38:04 +04:00
Nikita Sivukhin
72089d2682 adjust compilation 2025-11-12 16:30:50 +04:00
Nikita Sivukhin
aa65cfd55d update Cargo.toml 2025-11-12 16:14:14 +04:00
Nikita Sivukhin
a25e3e76eb wip 2025-11-12 13:21:34 +04:00
Nikita Sivukhin
6f7edcaddd agent review fixes 2025-11-12 12:32:45 +04:00
Nikita Sivukhin
b73ff13b88 add simple implementation of Sparse IO 2025-11-12 12:04:12 +04:00
Nikita Sivukhin
d519945098 make ArenaBuffer unsafe Send + Sync 2025-11-12 10:54:40 +04:00
Nikita Sivukhin
33375697d1 add partial database storage implementation 2025-11-12 10:53:25 +04:00
Nikita Sivukhin
a855a657aa report network stats 2025-11-12 10:53:25 +04:00
Nikita Sivukhin
98db727a99 integrate extra io stepping logic to the JS bindings 2025-11-12 10:53:25 +04:00
Nikita Sivukhin
d42b5c7bcc wip 2025-11-12 10:53:25 +04:00
Nikita Sivukhin
34f1072071 add hooks to plug partial sync in the sync engine 2025-11-12 10:53:25 +04:00
pedrocarlo
1db13889e3 Change Value::Text to use a Cow<'static, str> instead of Vec<u8> 2025-11-11 16:11:46 -03:00
Nikita Sivukhin
da61fa32b4 use dyn DatabaseStorage instead of DatabaseFile 2025-11-06 17:42:03 +04:00
Nikita Sivukhin
d013876998 cargo fmt 2025-10-29 16:46:51 +04:00
Nikita Sivukhin
82d54999b1 fix pull operation in sync engine
- before we fetched pull generation and last_change_id from the remote during pull - which is incorrect because fetched information can be inconsistent with WAL updates we received from the server (latest server state can be in "future" compared to the WAL updates we got since we can make push in parallel with updates pull operation)
- now we read information about "server state" (pull generation, last_change_id) directly from the local DB right after we applied changes from the remote which get us consistent view on the state considering WAL updates we got
- also fetching remote in the pull is bad - since pull block writes and network call with unpredictable latency poorly affect writes to the database
2025-10-29 16:24:05 +04:00
pedrocarlo
5a7390735d rename Completion functions 2025-10-06 11:07:06 -03:00
pedrocarlo
e93add6c80 remove dyn DatabaseStorage and replace it with DatabaseFile 2025-10-03 14:14:15 -03:00
Nikita Sivukhin
5ec2d96bc1 support defered sync in the engine 2025-10-02 16:17:36 +04:00
Nikita Sivukhin
7016d9456e fix test in the database tape 2025-10-02 16:17:36 +04:00
Nikita Sivukhin
91aeadd940 apply create DDL operation with IF NOT EXISTS clause in order to make them idempotent 2025-10-02 15:56:34 +04:00
Pavan-Nambi
49d5141f2d
Merge remote-tracking branch 'origin/main' into cdc_fail_autoincrement 2025-09-24 18:06:02 +05:30