Commit graph

118 commits

Author SHA1 Message Date
Jussi Saurio
87f9cb71c7 sim: fix apply_snapshot for create table and create index
have to leave for train soon, so claude summary:

Changes in runner/env.rs:
  1. Renamed RowOperation to TxOperation
  2. Fixed the CreateTable variant to store the full Table instead of separate fields
  3. Added CreateIndex and DropIndex variants
  4. Added corresponding record_* methods to both TransactionTables and ShadowTablesMut
  5. Updated apply_snapshot() to handle CreateTable, CreateIndex, and DropIndex operations in order
  6. Removed the index sync loop at the end of apply_snapshot() since operations are now tracked properly

  Changes in model/mod.rs:
  1. Updated Shadow for Create to call record_create_table() before applying the change
  2. Updated Shadow for CreateIndex to call record_create_index() before applying the change
  3. Updated Shadow for DropIndex to call record_drop_index() before applying the change

  The core fix is that CreateTable and CreateIndex operations are now recorded as they happen, so when apply_snapshot() replays operations in order, subsequent operations like AlterTable find the table already created in committed_tables.
2025-12-20 07:51:24 +02:00
pedrocarlo
1c737cfac5 refactor simulator to use new helpers 2025-12-19 17:20:46 -03:00
Pekka Enberg
edd45ff7b8 Improve MVCC DX by dropping --experimental-mvcc flag
The DX is right now pretty terrible:

```
penberg@vonneumann turso % cargo run -- hello.db
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/tursodb hello.db`
Turso v0.4.0-pre.18
Enter ".help" for usage hints.
Did you know that Turso supports live materialized views? Type .manual materialized-views to learn more.
This software is in BETA, use caution with production data and ensure you have backups.
turso> PRAGMA journal_mode = 'experimental_mvcc';
  × Invalid argument supplied: MVCC is not enabled. Enable it with `--experimental-mvcc` flag in the CLI or by setting the MVCC option in `DatabaseOpts`

turso>
```

To add insult to the injury, many SDKs don't even have a way to enable
MVCC via database options. Therefore, let's remove the flag altogether.
2025-12-19 12:59:42 +02:00
Jussi Saurio
da0ec21580 fix/sim: all alter table ops must be recorded and applied in order
this didn't work:

BEGIN;
ALTER TABLE t ADD COLUMN foo;
DELETE FROM t WHERE bar != 5;
COMMIT;

None of the rows where bar != 5 were deleted because apply_snapshot()
was checking that the rows in the committed table were exactly equal
to the rows that were recorded, but since the recorded deletes contained
a NULL `foo` column, they never matched. This meant that the sim thought
it should still have all the rows that were deleted.

Fix:

like all the other operations, record add column / drop column too so
that they are applied in sequential order in apply_snapshot()

No explicit test for this - I ran into this in another branch of mine
whose seed doesn't reproduce on main (because I changed the simulator
in that branch).
2025-12-17 16:27:36 +02:00
Jussi Saurio
70eeae0d88 expect instead of if let 2025-12-10 08:52:55 +02:00
Jussi Saurio
7c3bfeed7c sim: fix applying transaction effects
the issue was that the sim was executing queries on
commit, instead of applying their effects. for example,
if transaction T1 did a `DELETE FROM t WHERE TRUE` it would
delete all the rows even if another mvcc transaction T2 had
inserted rows concurrently that should not be visible to
T1.
2025-12-09 13:03:53 +02:00
Jussi Saurio
7b8acc7c04 sim: rollback all shadow transactions on REOPEN_DATABASE 2025-12-09 11:32:34 +02:00
Jussi Saurio
826ca4d44d chore: remove experimental_indexes feature flags 2025-12-08 13:00:37 +02:00
Jussi Saurio
77ad9d87c5 sim: stop ignoring sql execution errors 2025-12-05 13:14:39 +02:00
pedrocarlo
085c108f0d Simulator: after freestanding DML statement, check the table's content 2025-12-03 10:48:46 -03:00
pedrocarlo
f09b73c768 remove Span, as interaction ID is enough to determine membership of a property 2025-11-17 11:45:02 -03:00
pedrocarlo
2aab33b714 find_interactions_range only check for interaction id to determine membership 2025-11-17 11:45:02 -03:00
pedrocarlo
af31e74d9f add depending tables to assertions to delete them if needed in shrinking 2025-11-17 11:45:02 -03:00
pedrocarlo
157a5cf10a - add Interaction Builder to simplify code for building an interaction.
Modify `generation/property.rs` to use the Builder
- add additional metadata to `Interaction` to give more context for
  shrinking and iterating over interactions that originated from the
  same interaction.
- add Iterator like utilities for `InteractionPlan` to facilitate
  iterating over interactions that came from the same property:
2025-11-17 11:45:02 -03:00
pedrocarlo
c088a653e6 move interaction stats to metrics 2025-11-17 11:45:02 -03:00
pedrocarlo
2fe39d40bb add Span and PropertyMetadata structs 2025-11-17 11:45:02 -03:00
pedrocarlo
a4f0f2364d disable Watch Mode until we can properly serialize interaction plan 2025-11-17 11:45:02 -03:00
pedrocarlo
8f50154db2 separate struct defining code from struct generation code. Also move Remaining to a metrics file 2025-11-17 11:44:20 -03:00
Mikaël Francoeur
d309e6ddb3
generate self-inserts with nested subqueries 2025-11-14 09:16:17 -05:00
Mikaël Francoeur
23d6080531
make FromClause recursive 2025-11-14 09:16:17 -05:00
Mikaël Francoeur
8ea038c00b
fix join 2025-11-14 09:16:17 -05:00
Pavan-Nambi
73cabf49f2
fmt sim 2025-10-18 18:46:20 +05:30
Pavan-Nambi
b473a1c3ca
Merge remote-tracking branch 'mikael/sim-pragma' into avcm 2025-10-18 16:41:39 +05:30
Jussi Saurio
ebc4ddb2a2
Merge 'Simulator: fix alter table shadowing to modify index column name ' from Pedro Muniz
Forgot to modify the column name referenced in the indexes when
shadowing

Reviewed-by: bit-aloo (@Shourya742)

Closes #3712
2025-10-14 07:25:29 +03:00
pedrocarlo
45567e6837 fix alter table shadowing to modify index column name on rename and alter 2025-10-13 14:02:26 -03:00
pedrocarlo
bfeccf6543 integrate DropIndex in query generator 2025-10-13 13:56:36 -03:00
pedrocarlo
b2e54d9816 add Drop Index to simulator model 2025-10-13 13:32:16 -03:00
pedrocarlo
d99e3f590f ALTER TABLE should be added to is_ddl 2025-10-13 02:34:43 -03:00
pedrocarlo
49e96afd39 generate ALTER COLUMN 2025-10-13 02:34:43 -03:00
pedrocarlo
9c2edbb8b7 create separate Index struct for sql generation 2025-10-13 02:04:15 -03:00
pedrocarlo
c072058e4b add Alter Table query generation in Sim 2025-10-13 02:04:15 -03:00
pedrocarlo
230755eb2e shadow for AlterTable 2025-10-13 02:04:15 -03:00
pedrocarlo
f593080c2a add Query::AlterTable variant 2025-10-10 11:08:04 -03:00
pedrocarlo
b6f94b2fa1 remove dead code in sim 2025-10-09 17:25:04 -03:00
Mikaël Francoeur
7aa91d7c24
add pragma autovacuum_mode to simulator 2025-10-09 17:28:48 +03:00
pedrocarlo
6d5443d4f0 add Query::Placeholder 2025-10-07 11:38:47 -03:00
pedrocarlo
4fc7be5042 as we have DROP table now, if we want to generate extensional queries eagerly, without affecting how we document interactions with MVCC, we need to travel forward in time and shadow queries eagerly so we can generate queries correctly. This involves cloning the tables unfortunately which is inneficient but correct 2025-10-07 11:38:47 -03:00
pedrocarlo
bb9c8dea4f rework interaction generation to only generate possible queries + do less allocations 2025-10-07 02:36:13 -03:00
pedrocarlo
1d1b09dc17 modify query generation to always sample from valid queries 2025-10-07 02:36:13 -03:00
pedrocarlo
13764e6b8e in mvcc if creating a ddl statement, first commit all connections 2025-09-27 23:52:57 -03:00
pedrocarlo
d3c2198a75 keep track of each pending query for the transaction and then apply the queries on commit 2025-09-27 23:52:57 -03:00
pedrocarlo
021d5d272a refactor shadowing code to take into account snapshot isolation 2025-09-20 12:01:07 -03:00
pedrocarlo
c4843d6a6e refactor SimulatorEnv to hold committed tables and separate connection tables for snapshot isolation 2025-09-20 11:52:15 -03:00
pedrocarlo
5f24ff6e45 fix non determinism by using IndexSet for order preserving iteration 2025-09-05 02:00:18 -03:00
TcMits
33a04fbaf7 resolve conflict 2025-09-02 17:30:10 +07:00
pedrocarlo
962666831b read Profile file from path or use predefined profiles 2025-08-30 11:31:52 -03:00
TcMits
4ddfdb2a62 finish 2025-08-27 14:58:35 +07:00
pedrocarlo
8010b7d0c7 make simulator use sql_generation crate as dependency 2025-08-25 22:59:31 -03:00
Jussi Saurio
6c17fa2a5e fix/sim: prevent sim from trying to create an existing table or index
We recently merged a change that panics the sim on parse errors, because
not doing so has masked many scenarios where the sim unintentionally
creates incorrect sql and we just ignore it.

We already have Property::DoubleCreateFailure to assert that the same table
cannot be created twice, so this should not hide any bugs.
2025-08-17 18:13:05 +03:00
pedrocarlo
2bc6edc3d4 introduce TableContext for the simulator to properly generate predicates for Joins 2025-08-15 23:44:00 +03:00