Closes#3536
# Description
This PR implements **dynamic journal mode switching** via `PRAGMA
journal_mode`, allowing users to switch between WAL and MVCC modes at
runtime.
### Key Changes
**Core Feature: Journal Mode Switching**
- Added new `JournalMode` module (`core/storage/journal_mode.rs`) to
parse and handle journal mode transitions
- Modified `op_journal_mode` to correctly parse journal modes and update
the database header
- Emit checkpoint when setting a new journal mode to ensure data
consistency
- Added MVCC checkpoint support to `Connection::checkpoint`
**Database Initialization Improvements**
- Read DB header on `Database::open` and simplified `init_pager`
- Made `Version` an enum for better comparison semantics
- Automatically convert legacy SQLite databases to WAL mode
- Ensure DB header is flushed to disk when header changes during open
- Clear page cache after header validation
**Bug Fixes**
- Fixed dirty page invalidation in pager when clearing dirty pages in
page cache
- Fixed `is_none_or` check for row version existence in DB file (handles
MvStore initialization and empty database cases)
- Added `btree_resident` field in `RowVersion` to track if
insert/deletion originated from a btree
**Testing**
- Added fuzz tests for `journal_mode` transitions with Database
operations in between
- Added integration tests for testing switching from the different modes
while checking the header version is correct
- Added some specific regression tests for delete operations lost on
mode switch
- Fixed `index_scan_compound_key_fuzz` to use separate databases for
Turso and SQLite in MVCC mode. Also had to decrease number of rows for
MVCC test, as insert was very slow.
# TODO's
- Remove sync hacks from `op_journal_mode`
- Expand fuzzer with different queries
- Add to Simulator
- Special handling for read only databases and not allow any header
changes
# Motivation and context
Facilitate our users to test MVCC and transition back and forth from it.
# AI Disclosure
Used AI to catch and fix bugs in MVCC, further my understanding with
MVCC, write tests in `tests` folder, most of the PR summary, and the
docs in the `docs/manual.md` file
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4074
## Description
Run TCL tests for MVCC in the CI. Right now almost everything is
commented out as we need to fix them.
## AI Disclosure
No
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Adds CI step to run MVCC TCL tests via a custom sqlite wrapper with
experimental flags, and narrows the MVCC test suite to known passing
tests.
>
> - **CI**:
> - Add `Test mvcc` step in `test-limbo` job to run `make test-compat-
mvcc` in `.github/workflows/rust.yml`.
> - **Makefile**:
> - Update `test-compat-mvcc` to use `SQLITE_EXEC=scripts/turso-mvcc-
sqlite3`.
> - **Test Runner Script** (`scripts/turso-mvcc-sqlite3`):
> - Enable experimental flags: `--experimental-mvcc`, `--experimental-
views`, `--experimental-strict`.
> - **Tests**:
> - Define `testing/all-mvcc.test` with a subset of passing test
files; comment out failing ones.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
ee48302e30. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4178
This PR brings back go driver to the turso monorepo and adds sync
support for the golang.
It depends on the pre-compiled binaries at
https://github.com/tursodatabase/turso-go-platform-libs repository
There is a CI which trigger build on the turso-go-platform-libs repo in
case when release tag is published
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#4085
npm trusted publishing requires npm 11.5.1+, but Node 20 ships with
npm 10.x. Also remove registry-url which was causing token-based auth
to be attempted instead of OIDC.
The useblacksmith/setup-node@v5 fork is archived and doesn't properly
support npm OIDC trusted publishing. Switch the publish job to use
actions/setup-node@v4 which correctly configures .npmrc for OIDC
authentication.
Remove NPM_TOKEN-based authentication in favor of npm's new OIDC
trusted publishing. This eliminates the need for long-lived tokens
and improves security.
Changes:
- Add registry-url to setup-node configuration as recommended by
GitHub's documentation for npm provenance publishing. This ensures
npm knows the target registry and creates consistent .npmrc setup.
- Remove NPM_TOKEN from .npmrc injection
- Use --provenance flag directly on npm publish
- Remove NPM_TOKEN and GITHUB_TOKEN env vars
- The workflow already had the required "id-token: write" permission.
- Run sims with external for loops (instead of --loop flag in sim) so that each iteration gets a separate seed.
This produces much easier-to-debug traces when there's a failure since we don't have to potentially run 10 iterations
of the same seed to get to the failure.
- Run with memory-io because it's 1. faster and 2. introduces async IO latency unlike our normal synchronous IO
backend.
Previously we didn't use GITHUB_TOKEN for anything. But now that PR
meta-data must be fetched with a extra GitHub API call, then PRs at
least will always nedd GITHUB_TOKEN.
Closes#3918
Previously we didn't use GITHUB_TOKEN for anything.
But now that PR meta-data must be fetched with a extra
GitHub API call, then PRs at least will always nedd GITHUB_TOKEN.
The Antithesis config image was not being uploaded during CI runs, only
the workload image. This caused experiment failures when the config
image expired from the registry after 6 months of inactivity.
## Purpose
- Deploy `tech.turso:turso:<version>` to maven central so that users can
easily use java bindings
- For example :
https://repo1.maven.org/maven2/io/github/seonwkim/turso/0.0.1/
## Requirements
- [x] Add the following github secrets.
- [x] MAVEN_CENTRAL_USERNAME
- [x] MAVEN_CENTRAL_PASSWORD
- [x] GPG_PRIVATE_KEY
- [x] GPG_PASSPHRASE
- [ ] Namespace `tech.turso` must be registered at maven central
- [ ] GPG key registration to key servers
- Notes
- Retrieve MAVEN_CENTRAL_USERNAME and MAVEN_CENTRAL_PASSWORD from
[maven central](https://central.sonatype.com/usertoken)
- GPG keys should be registered. You should distribute your keys to
designated(maven central supported) servers
- Refer to [GPG key related docs](https://central.sonatype.org/publ
ish/requirements/gpg/#distributing-your-public-key)
- Btw, I used `keyserver.ubuntu.com` key server while testing
### [Maven Central Username &
Password](https://central.sonatype.com/usertoken)
<img width="2878" height="1338" alt="image"
src="https://github.com/user-
attachments/assets/03e6f967-a7f6-46b8-aef5-d15772bd9eea" />
### [Maven Central
Namespace](https://central.sonatype.com/publishing/namespaces)
<img width="1424" height="456" alt="image" src="https://github.com/user-
attachments/assets/8c0f4f17-bf5a-4c6a-bc47-748d86cd1f1a" />
## Future Works
- Currently, we depend on gradle.properties to determine the version of
our dependency and it's cumbersome to always change the version
manually. Let's find a better solution.
Closes#3624
