489 commits

Author	SHA1	Message	Date
Pekka Enberg	913b7ac600	core: Disable autovacuum by default ... People have discovered various bugs in autovacuum so let's disable it by default for now.	2025-11-02 12:09:21 +02:00
Pavan-Nambi	cf93360b9c	cleanup ... Co-authored-by: Mikaël Francoeur <mikael.francoeur@hotmail.com> cleanup Co-authored-by: Mikaël Francoeur <mikael.francoeur@hotmail.com>	2025-10-24 19:35:13 +05:30
Pavan-Nambi	8d0ae362da	Merge branch 'main' of github.com:tursodatabase/turso into avcm	2025-10-24 18:58:30 +05:30
Pavan-Nambi	dbd8b1b01d	change weight and add pragma_weight to querytypes	2025-10-24 18:10:10 +05:30
pedrocarlo	72baf48863	add random generation in simulator IO	2025-10-21 14:10:38 -03:00
Jussi Saurio	b67fabdd62	Fix git directory resolution in simulator to support worktrees ... sim cannot be run in a git worktree on main	2025-10-21 14:34:27 +03:00
Bob Peterson	b92f4cb9c4	Make Miri easier to run	2025-10-20 23:48:19 -05:00
Pavan-Nambi	73cabf49f2	fmt sim	2025-10-18 18:46:20 +05:30
Pavan-Nambi	b473a1c3ca	Merge remote-tracking branch 'mikael/sim-pragma' into avcm	2025-10-18 16:41:39 +05:30
Pekka Enberg	9a1bd2112d	Merge 'Run simulator under Miri' from Bob Peterson ... This adds support for running the simulator under Miri to detect UB. There are a few things to note about Miri and its limitations - It has limited `libc` coverage, so it's not really possible to have Miri help with `UringIO`/`UringFile` or `UnixIO`/`UnixFile`. That's a big gap ☹️ - It **can** work for `GenericIO`/`GenericFile`, which only uses `std` - It can't call external C libraries, so even using `sqlite` is out (hence adding `--disable-integrity-check` to the simulator for Miri use) - It runs on nightly, consequently there are a few new lints that don't exist on turso's pinned version of rustc Some questions I have about this MR - I made `GenericFile::{lock_file,unlock_file}` noops so I could use `GenericIO`. This isn't great, but if/when you update from Rust 1.88.0 to 1.89.0, `std::File::{lock,lock_shared,unlock}` will be stabilized and available. Should I note that as a TODO or something? - Previously, the sim runner shelled out to `git` to get stuff like the current git hash and the repo directory. For Miri, that's out, and so is `git2`. Unfortunately, `gix` is also out since it has a required dependency that uses inline assembly, which Miri doesn't like. I wrote a hacky shim that uses only std to look for `.git` and find the hash that HEAD is pointing to. It doesn't deal with stuff like packed-refs or the repo being a secondary one made with `git worktree`. I'm happy to support that, but wanted to hear from maintainers before doing more work. Two UB occurrences I already found: - `TursoRwLock::read` used `AtomicU64::compare_exchange_weak`, which is (evidently) [allowed to spuriously fail](https://doc.rust-lang.org/std/s ync/atomic/struct.AtomicU64.html#method.compare_exchange_weak) in exchange for perf. Miri forces this behavior, which triggers trivial read deadlocks even with zero readers/writers. I changed it to `compare_exchange`, but I'm not an atomics expert. - Uninitialized read in non-Unix `core::storage::buffer_pool::arena::alloc`. This is a simple one, resolved by using `std::alloc::alloc_zeroed` instead of `std::alloc::alloc` Moving forward, I'd be interested in potentially getting the tests to run in Miri, too. `tokio` looks like a good example of a project with partial coverage that runs it where they can. They have some extra test config to allow as many as possible to run under Miri, with appropriately scaled-down parameter values since Miri is super slow Closes #3720	2025-10-14 09:26:55 +03:00
Jussi Saurio	ebc4ddb2a2	Merge 'Simulator: fix alter table shadowing to modify index column name ' from Pedro Muniz ... Forgot to modify the column name referenced in the indexes when shadowing Reviewed-by: bit-aloo (@Shourya742) Closes #3712	2025-10-14 07:25:29 +03:00
Bob Peterson	4d843804b7	Add --disable-integrity-check option to simulator ... Miri can't execute sqlite via the FFI, so this needs to be configurable	2025-10-13 14:54:16 -05:00
Bob Peterson	3d4c10df40	Document using Miri to run the simulator	2025-10-13 14:54:16 -05:00
Bob Peterson	ce2f286df0	Replace git shell commands with std shims ... gix doesn't work here, since while it's pure Rust, it has a non-configurable dependency on crates using inline assembly, which Miri does not support. This commit is a bit of a hack, and only works in non-bare git repos without e.g packed-refs.	2025-10-13 14:54:16 -05:00
pedrocarlo	45567e6837	fix alter table shadowing to modify index column name on rename and alter	2025-10-13 14:02:26 -03:00
pedrocarlo	bfeccf6543	integrate DropIndex in query generator	2025-10-13 13:56:36 -03:00
pedrocarlo	b2e54d9816	add Drop Index to simulator model	2025-10-13 13:32:16 -03:00
Jussi Saurio	c12c1db275	Merge 'Simulator: persist files in sim memory IO for integrity check' from Pedro Muniz ... If we don't persist the files, rusqlite will open an empty file and perform integrity check on it. Reviewed-by: bit-aloo (@Shourya742) Closes #3676	2025-10-13 14:23:53 +03:00
pedrocarlo	773fa28063	workaround in sqlite for schema changes become visible to other connections	2025-10-13 02:34:43 -03:00
pedrocarlo	dca1137f81	rusqlite stop trying to get rows when we error with InvalidColumnIndex	2025-10-13 02:34:43 -03:00
pedrocarlo	d99e3f590f	ALTER TABLE should be added to is_ddl	2025-10-13 02:34:43 -03:00
pedrocarlo	5f65196115	fix load_bug	2025-10-13 02:34:43 -03:00
pedrocarlo	b6c5fee300	do not count certain interactions in the InteractionPlan and correctly report the length when shrinking	2025-10-13 02:34:43 -03:00
pedrocarlo	49e96afd39	generate ALTER COLUMN	2025-10-13 02:34:43 -03:00
pedrocarlo	a18a472685	add option to disable alter column for differential testing	2025-10-13 02:34:43 -03:00
pedrocarlo	703efaa724	adjust Properties to skip Alter Table in certain conditions	2025-10-13 02:34:43 -03:00
pedrocarlo	9c2edbb8b7	create separate Index struct for sql generation	2025-10-13 02:04:15 -03:00
pedrocarlo	c072058e4b	add Alter Table query generation in Sim	2025-10-13 02:04:15 -03:00
pedrocarlo	230755eb2e	shadow for AlterTable	2025-10-13 02:04:15 -03:00
pedrocarlo	fafbdbfa9d	persist files in sim memory io for integrity check	2025-10-11 15:03:22 -03:00
pedrocarlo	f593080c2a	add Query::AlterTable variant	2025-10-10 11:08:04 -03:00
pedrocarlo	b6f94b2fa1	remove dead code in sim	2025-10-09 17:25:04 -03:00
Mikaël Francoeur	7aa91d7c24	add pragma autovacuum_mode to simulator	2025-10-09 17:28:48 +03:00
pedrocarlo	f54b1132ca	ignore Property::AllTableHaveExpectedContent when counting stats, so we can generate more interesting interactions	2025-10-09 01:20:03 -03:00
pedrocarlo	300d918040	fix differential check for parse error	2025-10-07 15:03:50 -03:00
pedrocarlo	3b2583c540	adjust Interaction generation to take into account possibilty of PropertyDistribution to have 0 Weights	2025-10-07 14:17:37 -03:00
pedrocarlo	c578f7ba96	Faultless should produce any type of query, just not faulty	2025-10-07 14:15:51 -03:00
pedrocarlo	21fc8bae2a	Property::FaultyQuery and FsyncNoWait stored a list of tables to check the on the database. Again, the FaultyQuery could be a Drop Table which meant that we could be running a SELECT on an inexistent table. To solve this, just insert a Property that check all the tables in the db after a Faulty Property	2025-10-07 13:23:35 -03:00
pedrocarlo	6bad5d04ce	generate extensional queries when iterating over the next interaction, not when generating the property. This is necessary as the extensional queries can modify schema and thus could cause the next queries to fail because the DB enviroment context was not updated on generation time. Rule of thumb: queries should never be generated in bulk, always one a a time so the enviroment can be shadowed accordingly	2025-10-07 13:19:53 -03:00
pedrocarlo	6d5443d4f0	add Query::Placeholder	2025-10-07 11:38:47 -03:00
pedrocarlo	7eb504baef	certain properties cannot be generated if there are no tables in the current context	2025-10-07 11:38:47 -03:00
pedrocarlo	07cc1c548b	adjust query generation to avoid DROP for certain extensional queries	2025-10-07 11:38:47 -03:00
pedrocarlo	4fc7be5042	as we have DROP table now, if we want to generate extensional queries eagerly, without affecting how we document interactions with MVCC, we need to travel forward in time and shadow queries eagerly so we can generate queries correctly. This involves cloning the tables unfortunately which is inneficient but correct	2025-10-07 11:38:47 -03:00
pedrocarlo	3e8867c8f5	DropSelect property should only fail when error is not a parse error on the table name	2025-10-07 11:38:47 -03:00
pedrocarlo	7f93f64fc5	enable Drop statements	2025-10-07 11:38:47 -03:00
pedrocarlo	8b6456f843	do not allow Property::Queries to attempt to be generated	2025-10-07 02:36:14 -03:00
pedrocarlo	91da12390d	refactor property generation to use query distribution and avoid more ... memory allocations	2025-10-07 02:36:14 -03:00
pedrocarlo	b1c26505b8	adjust Rng generic to include ?Sized + introduce WeightedDistribution trait	2025-10-07 02:36:13 -03:00
pedrocarlo	a5845285be	remove unnecessary functions	2025-10-07 02:36:13 -03:00
pedrocarlo	bb9c8dea4f	rework interaction generation to only generate possible queries + do less allocations	2025-10-07 02:36:13 -03:00