## Description
closes https://github.com/tursodatabase/turso/issues/4142
<!--
Please include a summary of the changes and the related issue.
-->
## Motivation and context
compatibility, we were wrongly rewriting table qualified cols, also
added trigger.test to all.test and expect correct values in a test
<!--
Please include relevant motivation and context.
Link relevant issues here.
-->
## AI Disclosure
None
<!--
Please disclose if any LLM's were used in the creation of this PR and to
what extent,
to help maintainers properly review.
-->
Closes#4206
The translate_integrity_check function was missing a call to
add_pragma_result_column, causing num_columns() to return 0.
This made the Python bindings treat it as a non-row-returning
statement, finalizing it during execute() and leaving fetchone()
to return None instead of ("ok",).
Spotted by Antithesis.
## Beef
- Add `./scripts/run-sqlancer.sh` script to run
[SQLancer](https://github.com/sqlancer/sqlancer) using Turso's Java
bindings.
> SQLancer is a tool to automatically test Database Management Systems
(DBMSs) in order to find bugs in their implementation. That is, it finds
bugs in the code of the DBMS implementation, rather than in queries
written by the user. SQLancer has found hundreds of bugs in mature and
widely-known DBMSs.
- Fix some bugs that were already found by running it
## Reader's guide to the PR
- Commit by commit reviewing is probably best since the java bindings
changes, turso core bugfixes, and the sqlancer vibecode are all
separated into commits.
## AI Disclosure
Heavy Opus 4.5 vibecoding. I just started with `"This is Turso, the Rust
rewrite of SQlite. Let's investigate ways to run SQLancer against it"`,
and went from there.
I seriously have no idea if this is the least-effort way of doing it,
but it works, so I think that's a good enough start.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: Pedro Muniz (@pedrocarlo)
Closes#4180
Problem:
The existing DP-based join optimizer has O(2^n) complexity, which
causes large joins to basically not get past the planning phase.
Fix:
Add a greedy algorithm that runs in O(n²) time for >12 tables.
Details:
- Add compute_greedy_join_order() with hub score heuristic for
selecting the starting table. Tables referenced by many other
tables' constraints are preferred, enabling index lookups on
subsequent joins. This is especially good for star schema
queries.
- Add GREEDY_JOIN_THRESHOLD constant (12) for switchover point
- Add fuzz tests covering star schemas, chains, cliques up to 62
tables, and LEFT JOIN ordering invariants (RHS of a left join
cannot be reordered).
- Not all the tests necessarily assert that a query results in a
good plan (apart from star schemas), but all tests do assert
that we are _able_ to construct a plan (unlike before, where
even 32-way joins would grind to a halt).
AI usage:
- Pretty much all of this was a conversation between me and Opus 4.5.
I asked it to search the internet for practical solutions to the
problem and it suggested a simple greedy search as a low-complexity
solution and I thought it was a good idea for now.
This PR (mostly) finishes implementing support for `ANALYZE`
It also uses this newly available metadata to improve calculating the
join order.
### Example Queries:
Both the same query, different order:
<img width="757" height="928" alt="image" src="https://github.com/user-
attachments/assets/82edd3bc-ef33-4df0-833d-92106bf4c065" />
Previously, tursodb would have changed the build table when the query
was written with `users` on the RHS. Now that we have the metadata
available, we are able to determine that `products` should _always_ be
the build table for inner equijoin/hash join.
=======================
### AI disclosure
A lot of the emission code in `core/translate/analyze.rs` was written by
codex.
EDIT: Opus 4.5 was monumental in the cost based optimization work here.
That remains to be seen whether or not it succeeded XD
Closes#4141
CTEs now work correctly when combined with UNION, UNION ALL, INTERSECT,
and EXCEPT.
**Before:**
```sql
WITH t AS (SELECT 1 as x) SELECT * FROM t UNION ALL SELECT 2 as x
-- Error: Parse error: no such table: t
```
**After:**
```sql
WITH t AS (SELECT 1 as x) SELECT * FROM t UNION ALL SELECT 2 as x
-- Works correctly, returns rows (1) and (2)
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4123
these are not necessary as long as the cost model works well enough,
which it does after a few tweaks to selectivities from the commits
that precede this one.
Closes#4066Closes#4129
## Problem
Take e.g.
CREATE TABLE t(x); CREATE INDEX txdesc ON t(x desc); INSERT INTO t
values (1),(2),(3);
SELECT * FROM t WHERE x > NULL;
--
Our plan, like Sqlite, was to start iterating the descending index from
the beginning (Rewind) and stop once we hit a row where x is <= than
NULL using `IdxGe` instruction (GE in descending indexes means LE).
However, `IdxGe` and other similar instructions use a sort comparison
where NULL is less than numbers/strings etc, so this would incorrectly
not jump.
## Fix
Fix: we need to emit an explicit NULL check after rewinding.
## Tests
Added TCL tests + improved `index_scan_compound_key_fuzz` to have NULL
seek keys sometimes.
## AI disclosure
I started debugging this with Claude Code thinking this is a much deeper
corruption issue, but Opus 4.5 noticed immediately that we are returning
rows from a `x > NULL` comparison which should never happen. Hence, the
fix was then fairly simple.
Closes#4132
Take e.g.
CREATE TABLE t(x); CREATE INDEX txdesc ON t(x desc);
INSERT INTO t values (1),(2),(3);
SELECT * FROM t WHERE x > NULL;
--
Our plan, like Sqlite, was to start iterating the descending index
from the beginning (Rewind) and stop once we hit a row where x is
<= than NULL using `IdxGe` instruction (GE in descending indexes
means LE).
However, `IdxGe` and other similar instructions use a sort comparison
where NULL is less than numbers/strings etc, so this would incorrectly
not jump.
Fix: we need to emit an explicit NULL check after rewinding.
After this fix, I ran the fuzz test for more than an hour with no
issues.
Closes https://github.com/tursodatabase/turso/issues/4075
## AI disclosure
Claude wrote the implementation and tests from just a copy/paste of the
Github issue.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#4119
SQLite rejects `CREATE INDEX idx ON t(rowid)` with "no such column: rowid"
because rowid is a pseudo-column, not an actual column. Limbo was
incorrectly allowing this.
The fix removes the special exception for ROWID_STRS (rowid, _rowid_, oid)
in validate_index_expression(). Now these identifiers are only allowed
if they match an actual column name in the table (i.e., when shadowed).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1. Previously compound select would not start a transaction if
the rightmost subselect had no table references, e.g.
SELECT * FROM t UNION VALUES(1)
2. Previously the column names for the query were taken from the
rightmost subselect - instead, they should be taken from the
leftmost subselect.
