Remove NPM_TOKEN-based authentication in favor of npm's new OIDC
trusted publishing. This eliminates the need for long-lived tokens
and improves security.
Changes:
- Add registry-url to setup-node configuration as recommended by
GitHub's documentation for npm provenance publishing. This ensures
npm knows the target registry and creates consistent .npmrc setup.
- Remove NPM_TOKEN from .npmrc injection
- Use --provenance flag directly on npm publish
- Remove NPM_TOKEN and GITHUB_TOKEN env vars
- The workflow already had the required "id-token: write" permission.
## Description
This PR simply removes 1 unneeded refcell, a couple unneeded clones and
makes naming consistent `Arc<Page>` -> `PageRef`
## Motivation and context
Did something similar in #4170
## AI Disclosure
No LLM's were used in the creation of this PR
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4172
## Description
<!--
Please include a summary of the changes and the related issue.
-->
i believe this reversion is incorrect, as we are not inserting into the
Limbo conn, and are opening a limbo conn and a sqlite conn into the same
DB file.
## Motivation and context
<!--
Please include relevant motivation and context.
Link relevant issues here.
-->
CI is slow as hell
## AI Disclosure
None
<!--
Please disclose if any LLM's were used in the creation of this PR and to
what extent,
to help maintainers properly review.
-->
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#4171
Make `checkpointed_txid_max_old` be an `Optional<NonZeroU64>` as we may
have never checkpointed a database before, meaning there is a
possibility of recovered rows being excluded if `begin_ts == 0` and
`checkpointed_txid_max_old == 0`
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4167
## Description
This PR removes some unneeded clones (mostly Arc clones, but still
clones) in the Pager
## AI disclosure
No LLM's were used in this PR
Reviewed-by: Pedro Muniz (@pedrocarlo)
Closes#4170
Recently maintainers have been self-disclosing LLM use in PR's, so I
don't see why we can't have a PR template to encourage everyone to do
so.
Reviewed-by: Pedro Muniz (@pedrocarlo)
Closes#4165
Some of the tests that were added pass only on this branch. I was
surprised to see that there didn't seem to be any TCL tests for
`json_insert()`.
## AI-generated description
When using JSON_INSERT with a path like '$.a.b.d' on an object like
'{"a": {"b": {"c": 5}}}', the function was incorrectly returning the
input unchanged instead of inserting the new key.
The root cause was that InsertNew mode was being applied to all path
segments. The fix uses Upsert mode for intermediate segments and only
applies InsertNew for the final segment.
🤖 Generated with [Claude Code](https://claude.com/claude-code), and
cleaned up by Mikaël
## AI Disclosure
This was written by Claude, and then I cleaned it up manually.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#4166
`ExecRows` trait should allow us to do something like this when testing:
```rust
let rows: Vec<(String,)> = conn.exec_rows("SELECT val FROM t ORDER BY val");
```
Which just makes your life easier overall so we don't have to constantly
repeat the `.step` loop
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#4164
When using JSON_INSERT with a path like '$.a.b.d' on an object like
'{"a": {"b": {"c": 5}}}', the function was incorrectly returning the
input unchanged instead of inserting the new key.
The root cause was that InsertNew mode was being applied to all path
segments. The fix uses Upsert mode for intermediate segments and only
applies InsertNew for the final segment.
🤖 Generated with [Claude Code](https://claude.com/claude-code), and cleaned up by Mikaël
This PR (mostly) finishes implementing support for `ANALYZE`
It also uses this newly available metadata to improve calculating the
join order.
### Example Queries:
Both the same query, different order:
<img width="757" height="928" alt="image" src="https://github.com/user-
attachments/assets/82edd3bc-ef33-4df0-833d-92106bf4c065" />
Previously, tursodb would have changed the build table when the query
was written with `users` on the RHS. Now that we have the metadata
available, we are able to determine that `products` should _always_ be
the build table for inner equijoin/hash join.
=======================
### AI disclosure
A lot of the emission code in `core/translate/analyze.rs` was written by
codex.
EDIT: Opus 4.5 was monumental in the cost based optimization work here.
That remains to be seen whether or not it succeeded XD
Closes#4141
Improve errors messages for sync engine:
- Error for unexpected metadata file format
```
sync engine operation failed: deserialization error: unexpected metadata file format, 'version' field must be present and have string type
```
- Error from the server:
```
sync engine operation failed: database sync engine error: remote server returned an error: status=401, body={"error":"Unauthorized: `unauthorized access attempt on database: empty JWT token`"}
```
Closes#4155
Sadly, due to how we use dual cursors, we cannot use optimization under
btree cursor to count rows without first checking if the row in btree is
valid. So this is a slow count implementation.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Adds a state-driven `count()` implementation that iterates via dual
cursors, validating B-Tree keys and tallying visible rows.
>
> - **Core (MVCC Cursor)**:
> - **Counting**:
> - Implement `count()` using a small state machine (`CountState`)
to iterate (`rewind` → `next`) and tally rows, ensuring B-Tree keys are
validated via existing dual-cursor logic.
> - **State Management**:
> - Add `CountState` enum and `count_state` field to
`MvccLazyCursor` to keep count logic isolated from other cursor states.
> - Initialize `count_state` in `new()`.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
356ea0869d. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4160
turso cloud generate urls with `libsql://` protocol by default
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Pedro Muniz (@pedrocarlo)
Closes#4159
On bootstrap just store the header but not flush it to disk. Only try to
flush it when we start an MVCC transaction. Also applied fix in
`OpenDup` where we should not wrap an ephemeral table with an MvCursor
Reviewed-by: Mikaël Francoeur (@LeMikaelF)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4151
CTEs now work correctly when combined with UNION, UNION ALL, INTERSECT,
and EXCEPT.
**Before:**
```sql
WITH t AS (SELECT 1 as x) SELECT * FROM t UNION ALL SELECT 2 as x
-- Error: Parse error: no such table: t
```
**After:**
```sql
WITH t AS (SELECT 1 as x) SELECT * FROM t UNION ALL SELECT 2 as x
-- Works correctly, returns rows (1) and (2)
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4123
This significantly simplifies using the Rust binding (and adds a more
complete example).
By adding prepare to Transaction (that just passes it onto the
Transactions connection) we can write simpler application code that is
passed a transaction. This allows multiple database update methods to be
flexibly grouped into transactions, even when they use prepared
statements.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4113
This PR introduces fuzz test which maintains 2 database files and
periodically switch the "engine" which process operations over this
files between sqlite and turso. The purpose of this fuzz test is to
ensure compatibility with on-disk data format and logic depending on it
between sqlite and turso.
With this test, few bugs were identified:
1. automatic indices order were incompatible with sqlite in turso
because we processed constraints first and columns later (so, for cases
like `CREATE TABLE t (x UNIQUE, y, UNIQUE (y))` the order of indices
were `y` and `x` while sqlite expected that first autoindex is over `x`
and second is over `y`
2. after collecting `unique_sets` we need to process them in exactly
same order for the same reason describe above
3. now all write queries require subtransaction journal because without
this (with previous extra condition `if
!self.table_references.is_empty()`) we incorrectly processed `CREATE
UNIQUE INDEX` queries in case when they failed with `CONSTRAINT` error
(because column is not unique)
4. Table access method decision fixed in order to not use index with
collation different from table column collation
We still have at least one more bug which can be identified by
uncommenting fuzz test configuration (see issue
https://github.com/tursodatabase/turso/issues/4154):
```
// temporary disable this action - because right now we still have bug with affinity for insertion to the indices
// 35..=55 => Action::InsertData, // ~20%
```
## AI disclosure
The fuzz test was mainly written by `openai/gpt-5` model with the
following prompt
<details>
[Code output="mod.rs" model="openai/gpt-5" language="rust"]
Write fuzz test which will simulate different data layout in SQLite and
switch connections between each other periodically.
The purpose of the test is to identify potential incompatibilities in
data layou between sqlite3 and turso.
From the layout perspective you MUST randomize following things:
1. UNIQUE constraint placement:
* Mix 3 different options:
```sql
CREATE TABLE t (x, y UNIQUE);
CREATE TABLE t (x, y, UNIQUE (y));
CREATE UNIQUE INDEX t_idx ON t (y);
```
2. UNIQUE constraint column order - it must be randomized and multiple
column constraints must be fuzzed
3. Create INDEX column order
4. Column collation
5. Sort order for index columns
6. Amount of columns
7. AUTOINCREMENT columns (they are affecting data on disk as they force
creation of sqlite_sequence table)
Test also must execute read queries in order to be able to capture
potential layout mismatch:
Use SELECT with random subset of columns from the table for that
purpose.
The test should be structured like this:
```rs
struct FuzzTestState {
tables: Vec<FuzzTestTable>,
indices: Vec<FuzzTestIndex>,
}
#[test]
pub fn test_data_layout_compatibility() {
const OUTER: usize = 100;
const INNER: usize = 100;
let (mut rng, seed) = rng_from_time_or_env();
tracing::info!("test_data_layout_compatibility seed: {}", seed);
let left = NamedTempFile::new().unwrap();
let right = NamedTempFile::new().unwrap();
let state = FuzzTestState::new();
for i in 0..OUTER {
let turso_path = if i % 2 == 0 { left.path() } else { right.path() };
let sqlite_path = if i % 2 == 1 { left.path() } else { right.path() };
let turso = TempDatabase::builder().with_db_path(turso_path).build();
let sqlite = TempDatabase::builder().with_db_path(sqlite_path).build();
for _ in 0..INNER {
// use state to pick random valid action for a database among the list:
// 1. create table
// 2. create index
// 3. select data from existing table
let sql = "...";
// execute sql against turso and sqlite and compare the generated rows
}
}
}
```
Use following snippet to understand style of tests and availble methods:
[Shell cmd="cat ./mod.rs | head -n 1000" /]
[/Code]
</details>
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4153
