Commit graph

1320 commits

Author SHA1 Message Date
Nikita Sivukhin
e19efb07b2 fix python lint errors 2025-12-22 22:03:09 +04:00
Nikita Sivukhin
8b04ce7235 rename partial_sync_opts to partial_sync_experimental for Python driver 2025-12-22 21:52:11 +04:00
Nikita Sivukhin
1a1d06b708 rename partialSyncOpts to partialSyncOptsExperimental in the JS driver 2025-12-22 21:50:27 +04:00
Nikita Sivukhin
f63ae56ddb rename options in go driver to *Experimental 2025-12-22 21:50:27 +04:00
Pekka Enberg
b4a331da38
Merge 'SDK tweaks' from Nikita Sivukhin
Few small tweaks in SDKs:
1. Export `ConnectionSync` type from Python SDK
2. Remove dependency on `conn_raw_api` and corresponding methods from
rust bindings - no one use them so let's drop support for this feature
completely
3. Add Waker dependency in the sdk-kit in order to integrate it later to
rust bindings
4. Add io_uring feature to the sdk-kit

Closes #4284
2025-12-22 11:59:02 +02:00
Nikita Sivukhin
fc21324c61 fix test 2025-12-21 15:47:19 +04:00
Nikita Sivukhin
f94f38623d uvx ruff check --fix 2025-12-21 14:29:10 +04:00
Nikita Sivukhin
0270505210 fix exception type 2025-12-21 14:28:52 +04:00
Nikita Sivukhin
03c95fb1ad fix async operation resume API usage in python bindings 2025-12-21 14:24:42 +04:00
Nikita Sivukhin
1393482f1f add concurrency test in python 2025-12-21 12:10:23 +04:00
pedrocarlo
06326e6d16 for now use the _io() for rust bindings for implementing future. But in the future we would remove this and the let the user advance io 2025-12-19 17:20:46 -03:00
pedrocarlo
10577dd6c5 adjust run_one_step_blocking 2025-12-19 17:20:46 -03:00
pedrocarlo
848c529653 fix more places to avoid using run_once 2025-12-19 17:20:45 -03:00
Pekka Enberg
4c1bc1ba95
Merge 'Improve MVCC DX by dropping --experimental-mvcc flag' from Pekka Enberg
Some checks are pending
Build & publish @tursodatabase/database / db-bindings-x86_64-pc-windows-msvc - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / db-bindings-x86_64-unknown-linux-gnu - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / sync-bindings-aarch64-apple-darwin - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / sync-bindings-aarch64-unknown-linux-gnu - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / sync-bindings-wasm32-wasip1-threads - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / sync-bindings-x86_64-pc-windows-msvc - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / sync-bindings-x86_64-unknown-linux-gnu - node@20 (push) Waiting to run
Build & publish @tursodatabase/database / Test DB bindings on Linux-x64-gnu - node@20 (push) Blocked by required conditions
Build & publish @tursodatabase/database / Test DB bindings on browser@20 (push) Blocked by required conditions
Build & publish @tursodatabase/database / Publish (push) Blocked by required conditions
Python / sdist (push) Waiting to run
Python / Release (push) Blocked by required conditions
Python / configure-strategy (push) Waiting to run
Python / test (push) Blocked by required conditions
Python / lint (push) Waiting to run
Python / linux (x86_64) (push) Waiting to run
Python / macos-arm64 (aarch64) (push) Waiting to run
Rust / simulator (push) Waiting to run
Rust / test-limbo (push) Waiting to run
Rust / test-sqlite (push) Waiting to run
Rust / cargo-fmt-check (push) Waiting to run
Rust / build-native (blacksmith-4vcpu-ubuntu-2404) (push) Waiting to run
Rust / build-native (macos-latest) (push) Waiting to run
Rust / build-native (windows-latest) (push) Waiting to run
Rust / clippy (push) Waiting to run
Rust Benchmarks+Nyrkiö / tpc-h (push) Waiting to run
Rust Benchmarks+Nyrkiö / vfs-bench-compile (push) Waiting to run
Rust Benchmarks+Nyrkiö / bench (push) Waiting to run
Rust Benchmarks+Nyrkiö / clickbench (push) Waiting to run
Rust Benchmarks+Nyrkiö / tpc-h-criterion (push) Waiting to run
The DX is right now pretty terrible:
```
penberg@vonneumann turso % cargo run -- hello.db
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/tursodb hello.db`
Turso v0.4.0-pre.18
Enter ".help" for usage hints.
Did you know that Turso supports live materialized views? Type .manual materialized-views to learn more.
This software is in BETA, use caution with production data and ensure you have backups.
turso> PRAGMA journal_mode = 'experimental_mvcc';
  × Invalid argument supplied: MVCC is not enabled. Enable it with `--experimental-mvcc` flag in the CLI or by setting the MVCC option in `DatabaseOpts`

turso>
```
To add insult to the injury, many SDKs don't even have a way to enable
MVCC via database options. Therefore, let's remove the flag altogether.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #4294
2025-12-19 13:59:34 +02:00
Pekka Enberg
edd45ff7b8 Improve MVCC DX by dropping --experimental-mvcc flag
The DX is right now pretty terrible:

```
penberg@vonneumann turso % cargo run -- hello.db
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/tursodb hello.db`
Turso v0.4.0-pre.18
Enter ".help" for usage hints.
Did you know that Turso supports live materialized views? Type .manual materialized-views to learn more.
This software is in BETA, use caution with production data and ensure you have backups.
turso> PRAGMA journal_mode = 'experimental_mvcc';
  × Invalid argument supplied: MVCC is not enabled. Enable it with `--experimental-mvcc` flag in the CLI or by setting the MVCC option in `DatabaseOpts`

turso>
```

To add insult to the injury, many SDKs don't even have a way to enable
MVCC via database options. Therefore, let's remove the flag altogether.
2025-12-19 12:59:42 +02:00
Pere Diaz Bou
55de098ad3
Merge 'Local sync server' from Nikita Sivukhin
This PR introduces local sync server and run tests for Go and Python
against it in CI
The local sync server implements 2 endpoints:
1. `/v2/pipeline` - subset of SQL over HTTP protocol (Hrana) to execute
logical push operations from the client
2. `/pull-updates` - endpoint which returns page updates for client to
apply locally
The implementation is based on the local database file with **disabled
checkpoint** in order to preserve whole DB history and allow server to
respond to client which can have arbitrary stale DB.
For implementation, sync server uses extra API exposed by the turso-core
under `conn-raw-api` feature which includes `wal_state` /
`wal_get_frame` methods.
Usage:
- `tursodb --sync-server 0.0.0.0:8080` - in-memory database
- `tursodb local.db --sync-server 0.0.0.0:8080` - local db file

Closes #4191
2025-12-19 11:05:12 +01:00
Nikita Sivukhin
9326b0c6c0 fix tests 2025-12-19 13:24:26 +04:00
Pekka Enberg
3009ae14e0 Turso 0.4.0-pre.18 2025-12-19 09:57:09 +02:00
Nikita Sivukhin
9032c61e52 reformat python 2025-12-18 16:39:25 +04:00
Nikita Sivukhin
cc648f8c63 revert changes in js bindings 2025-12-18 16:31:20 +04:00
Nikita Sivukhin
7fc026d0db add local mode for python and go tests 2025-12-18 16:25:29 +04:00
Nikita Sivukhin
ec675dd08c fix compilation 2025-12-18 15:15:46 +04:00
Nikita Sivukhin
327562fdf3 Merge branch 'main' into local-sync-server 2025-12-18 15:13:56 +04:00
Nikita Sivukhin
8b1673747d wip 2025-12-18 15:00:03 +04:00
Nikita Sivukhin
d8a50ec5d2 trying to debug test 2025-12-18 14:55:41 +04:00
Nikita Sivukhin
e5e7f68681 Merge branch 'main' into sdk-tweaks 2025-12-18 12:30:09 +04:00
Nikita Sivukhin
27b4071c53 export ConnectionSync type from python sdk 2025-12-18 12:26:20 +04:00
Nikita Sivukhin
2b8020226a remove methods 2025-12-18 12:25:43 +04:00
Nikita Sivukhin
f4a1a91ba2 remove unnecessary dependencies 2025-12-18 12:22:06 +04:00
Nikita Sivukhin
028e13f283 propagate waker to sdk-kit 2025-12-18 12:17:57 +04:00
Dave Warnock
70f3ae2912
Merge branch 'tursodatabase:main' into Connection-small-refactor 2025-12-17 16:36:48 +00:00
Pekka Enberg
fcfcc04550
Merge 'Add dotnet bindings to Turso' from Kopylov Dmitriy
Added C#/dotnet bindings for turso

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3594
2025-12-17 16:45:55 +02:00
Nikita Sivukhin
5f002f9441 fix another bug 🤦 2025-12-17 15:54:05 +04:00
Nikita Sivukhin
07c7f90aea propagate partial sync settings in the web 2025-12-17 15:44:34 +04:00
Pekka Enberg
a0bd11c035
Merge 'No tempfiles on wasm' from Nikita Sivukhin
We now can attempt to create temp file in WASM for sorter and hash join
which will result in crash:
```
stderr | promise.test.ts > sorter-wasm
thread '<unnamed>' panicked at library/std/src/sys/pal/wasi/os.rs:125:5:
stderr | promise.test.ts > sorter-wasm
no filesystem on wasm
stderr | promise.test.ts > sorter-wasm
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
This PR make simple adjustment to always use in-memory IO for temp files
in WASM

Closes #4256
2025-12-17 12:23:09 +02:00
Pekka Enberg
7fd4b0ed63 Turso 0.4.0-pre.17 2025-12-17 10:18:19 +02:00
Nikita Sivukhin
fb568b9ead add test with huge hash join 2025-12-17 12:18:16 +04:00
Pekka Enberg
8eeec6811b
Merge 'fix bug in the sync engine wasm implementation' from Nikita Sivukhin
- We do unnecessary clone of the underlying DB object which we will
never be able to close explicitly
- For language with GC this means that user has no control over object
lifetime

Closes #4242
2025-12-17 10:07:09 +02:00
Nikita Sivukhin
8b781b6646 add tests with sorter which will attempt to create tempfile in wasm 2025-12-17 11:05:04 +04:00
Nikita Sivukhin
12ba0403ed
Merge 'update go mod name as we will serve module through custom proxy' from Nikita Sivukhin
`go get turso.tech/database/tursogo`
Depends on internal PR:
https://github.com/tursodatabase/frontend/pull/2275

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #4244
2025-12-17 10:51:34 +04:00
Nikita Sivukhin
afe2432e85 fix typos and wording 2025-12-16 16:39:28 +04:00
Nikita Sivukhin
28c239e00e update go mod name as we will serve module through custom proxy 2025-12-16 15:23:35 +04:00
Nikita Sivukhin
b373c60b3d fix bug in the sync engine wasm implementation
- we do unnecessary clone of the underlying DB object which we will never be able to close explicitly
2025-12-16 14:03:07 +04:00
Nikita Sivukhin
5ab69bcf20 rename speculative load to prefetch (docs already uses this terminology) 2025-12-16 13:59:39 +04:00
Pekka Enberg
46740ee37c
Merge 'core: Fix integrity_check pragma code generation' from Pekka Enberg
The translate_integrity_check function was missing a call to
add_pragma_result_column, causing num_columns() to return 0. This made
the Python bindings treat it as a non-row-returning statement,
finalizing it during execute() and leaving fetchone() to return None
instead of ("ok",).
Spotted by Antithesis.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #4215
2025-12-15 11:17:32 +02:00
Pekka Enberg
574c8258b0 Turso 0.4.0-pre.16 2025-12-15 09:58:23 +02:00
Pekka Enberg
df9c39cd1c core: Fix integrity_check pragma code generation
The translate_integrity_check function was missing a call to
add_pragma_result_column, causing num_columns() to return 0.
This made the Python bindings treat it as a non-row-returning
statement, finalizing it during execute() and leaving fetchone()
to return None instead of ("ok",).

Spotted by Antithesis.
2025-12-15 09:46:46 +02:00
Pekka Enberg
944664a16e Turso 0.4.0-pre.15 2025-12-15 09:12:29 +02:00
Nikita Sivukhin
96c1b7b055 wipwip 2025-12-13 21:53:05 +04:00
Dave Warnock
123b0cfdb0 Remove mut from connection clone test 2025-12-12 23:07:10 +00:00