## Description
run PRAGMA journal_mode=experimental_mvcc with mvcc
## Motivation and context
After https://github.com/tursodatabase/turso/pull/4294 we need a way run
with mvcc mode.
## Description of AI Usage
it wrote the remove first line
Closes#4298
## CI speed improvements
Let's use Sccache for compiling our code faster, cargo nextest to run
tester faster, run all tests in CI, automatically terminate workflows if
you push a new commit (so we don't waste more money running code that we
don't care anymore).
Also https://github.com/useblacksmith/rust-cache is deprecated, so we
should be using `Swatinem/rust-cache@v2`
## CI test setup fixes
We were ignoring entire suites of tests in CI. Re-enable them and fix
the failures. Most of them were stale test setups, apart from https://gi
thub.com/tursodatabase/turso/pull/4311/commits/a67eb0006d8ac23dd85918309
303701959fdc988 which appears to be a real bug.
Also fix linking sqlite not working on windows for compat tests
## Runtime code fixes
a67eb0006d
dd85918309303701959fdc988 appears to be a real bug in GroupCompletion
code - `succeeded()` requires that the result be set to `Some(None)`
## Motivation and context
Closes https://github.com/tursodatabase/turso/issues/4266
<!--
Please include relevant motivation and context.
Link relevant issues here.
-->
## Description of AI Usage
Ai just edited the files for me, I was the one asked it do the changes
in the workflows
<!--
Please disclose how AI was used to help create this PR. For example, you
can share prompts,
specific tools, or ways of working that you took advantage of. You can
also share whether the
creation of the PR was mainly driven by AI, or whether it was used for
assistance.
This is a good way of sharing knowledge to other contributors about how
we can work more efficiently with
AI tools. Note that the use of AI is encouraged, but the committer is
still fully responsible for understanding
and reviewing the output.
-->
Closes#4311
Few small tweaks in SDKs:
1. Export `ConnectionSync` type from Python SDK
2. Remove dependency on `conn_raw_api` and corresponding methods from
rust bindings - no one use them so let's drop support for this feature
completely
3. Add Waker dependency in the sdk-kit in order to integrate it later to
rust bindings
4. Add io_uring feature to the sdk-kit
Closes#4284
## Description
BTreeCursor sets null flag to false once `seek` is called. This PR does
the same for MVCC
## Motivation and context
join.test failed with some cases due to this bug
## Description of AI Usage
I asked AI to find the issue but I ended showing the agent why he did
things wrong and that he should be ashamed
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4296
`resume` was not re-entrant and panicked in case when was called after
operation completion.
```
thread '<unnamed>' panicked at sync/sdk-kit/src/turso_async_operation.rs:59:20:
`async fn` resumed after completion
...
```
This PR makes `resume` method for `PyTursoAsyncOperation` re-entrant
(like in the c-api) and also convert exception type in the sync module
Closes#4315
## Description
This PR adds missing affinity conversion to hash joins by applying
affinity conversion to build and probe keys before hashing.
```
turso> CREATE TABLE x(a INTEGER);
turso> CREATE TABLE y(b TEXT);
turso> INSERT INTO x VALUES (2),(3);
turso> INSERT INTO y VALUES ('02'),('2'),('2.0'),('3x'),('3.5');
turso> SELECT a, b
FROM x JOIN y ON a = b
ORDER BY a, b;
┌───┬─────┐
│ a │ b │
├───┼─────┤
│ 2 │ 02 │
├───┼─────┤
│ 2 │ 2 │
├───┼─────┤
│ 2 │ 2.0 │
└───┴─────┘
```
## Motivation and context
Fixes#3482.
Currently, Turso returns an empty result set:
```
turso> CREATE TABLE x(a INTEGER);
turso> CREATE TABLE y(b TEXT);
turso> INSERT INTO x VALUES (2),(3);
turso> INSERT INTO y VALUES ('02'),('2'),('2.0'),('3x'),('3.5');
turso> SELECT a, b
FROM x JOIN y ON a = b
ORDER BY a, b;
turso>
```
Expected behavior:
```
sqlite> CREATE TABLE x(a INTEGER);
sqlite> CREATE TABLE y(b TEXT);
sqlite> INSERT INTO x VALUES (2),(3);
sqlite> INSERT INTO y VALUES ('02'),('2'),('2.0'),('3x'),('3.5');
sqlite> SELECT a, b
...> FROM x JOIN y ON a = b
...> ORDER BY a, b;
2|02
2|2
2|2.0
```
## Description of AI Usage
This PR was developed with assistance from Claude Sonnet 4.5 through
code completions.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#4317
Set the testing DB's to WAL mode permanently so we don't keep making
changes to these every time we open them with `make test`
Reviewed-by: Pedro Muniz (@pedrocarlo)
Closes#4321
## Transaction fixes
- return BUSY_SNAPSHOT instead of BUSY when transaction tries to
promote from read to write and its snapshot is stale. this is
a case where retrying with busy_timeout will never help.
- fix bug in begin_read_tx() where it would not allow to use a
readmark that is lower than shared_max (causes an enormous amount
of busy errors / contention)
- fix another bug in begin_read_tx() where it would not decrement
shared lock if it needed to abort due to stale header values
- validate shared state again after deciding to use read0 lock -
otherwise we might miss frames (another TOCTOU issue)
- implement sqlite's exponential backoff algorithm for begin_read_tx()
to improve multiple threads' ability to acquire write lock
## Fix deadlock
use with_shared() instead of with_shared_mut() for lock ops
using shared_mut() is unnecessary because the locks use atomics, and
in fact using shared_mut() can cause a deadlock, example:
- Thread 1 calls with_shared_mut() at in try_begin_read_tx
to claim/update a read mark slot. This waits for readers to release
their shared read locks.
- Thread 2 tries to use with_shared_mut() in end_read_tx() and also
can't proceed
## Perf/throughput test adjustment
If `BusySnapshot` is returned, explicitly ROLLBACK and restart the
transaction
## Review guide
I've interactive rebased this to 10 commits that are fairly sensible, so
feel free to read one-by-one.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#4289
using shared_mut() is unnecessary because the locks use atomics, and
in fact using shared_mut() can cause a deadlock, example:
- Thread 1 calls with_shared_mut() at in try_begin_read_tx
to claim/update a read mark slot. This waits for readers to release
their shared read locks.
- Thread 2 tries to use with_shared_mut() in end_read_tx() and also
can't proceed
Extract try_begin_read_tx with TryBeginReadResult::Retry for transient
conditions. begin_read_tx now retries with SQLite's quadratic backoff:
immediate retries for first 5 attempts, yield for 6-9, then quadratic
microsecond delays matching SQLite's formula.
BusySnapshot indicates the transaction's snapshot is permanently stale
and must be rolled back. Unlike Busy, retrying with busy_timeout will
never help - the caller must rollback and restart the transaction.
