Commit graph

108 commits

Author SHA1 Message Date
pedrocarlo
6dfd4700e9 simulator: disable all ansi encodings for debug print log file 2025-06-14 19:34:40 -03:00
Jussi Saurio
f7fa56ecfd Change integrity check failure to say 'simulation failed:'
Some checks failed
Run long fuzz tests on Btree / run-long-tests (push) Has been cancelled
Run long fuzz tests on Btree / simple-stress-test (push) Has been cancelled
our AWS nightly sim looks for a line 'simulation failed:' in the process
output to capture context for the github issues it creates.

if the sim passes otherwise but integrity check fails, then this log line
must also be present there, otherwise the aws sim will panic due to not finding
a suitable line.
2025-06-13 11:26:04 +03:00
pedrocarlo
83d72db901 better error messages + cleaner printing of file stats + tracing
instrumentation in `execute_interaction`
2025-06-11 11:32:45 -03:00
pedrocarlo
82538dbe8e add integrity_check to sim 2025-06-10 12:17:59 -03:00
pedrocarlo
f3ef60e1f1 use anyhow in simulator for lazily evaluated error context instead of eager evaluation with .or 2025-06-10 10:50:39 -03:00
Jussi Saurio
51e3a04668 simulator: add option to disable BugBase 2025-06-02 12:14:35 +03:00
Pere Diaz Bou
8ee7228c43 simulator: log to file too
most of the time debugging the log of a simulator run is spent reading
from a file for convinience. Let's just log to a file too.
2025-05-30 11:10:17 +02:00
Pere Diaz Bou
83db9be818 log seed creating plan 2025-05-30 11:04:19 +02:00
Pere Diaz Bou
d3495238c4 set default level info and log seed at startup 2025-05-30 11:02:05 +02:00
Pere Diaz Bou
8ec7e616b4 simulator: switch to tracing, run io.run_once and add update queries
* Without tracing crate we cannot log anything that happens in
limbo_core
* IO never ran in step loop inside simulator.
* Added update queries (which currently loop forever for some reason I'm
debugging).
2025-05-30 10:57:13 +02:00
alpaylan
e13b5bc698 fix min/max-tests bug 2025-04-11 15:33:32 -04:00
alpaylan
9d08693e8f add simulator subcommands 2025-04-11 14:10:12 -04:00
alpaylan
d4707fe391 add non-zero exit code in case of failures, remove the interactive initalization option in bug base for now, fix bugs in differential mode, add detailed information regarding runs to the bug base 2025-04-11 11:23:03 -04:00
alpaylan
64c2917e81 add bug base, refactor 2025-04-08 17:48:16 -04:00
Pekka Enberg
96175cccf7 cli: Add --experimental-mvcc option to enable MVCC 2025-03-06 10:16:42 +02:00
alpaylan
c133bbdd29 add differential testing against rusqlite 2025-02-11 14:13:14 -05:00
Aarni Koskela
eaea02c567 Fix a handful of typos 2025-02-09 18:08:29 +02:00
alpaylan
4362bc16a3 fix formatting 2025-02-08 09:37:08 -05:00
alpaylan
6308ce4544 fix the shrinking file and poison errors 2025-02-08 09:34:51 -05:00
Pekka Enberg
0d0906dce4 Merge 'simulator: implement --load and --watch flags' from Alperen Keleş
The current status of the PR is halfway. The new framing of simulation
runner where `setup_simulation` is separated from `run_simulation`
allows for injecting custom plans easily. The PR is currently missing
the functionality to update the `SimulatorEnv` ad hoc from the plan, as
the environment tables were typically created during the planning phase.
The next steps will be to implement a function `fn
mk_env(InteractionPlan, SimulatorEnv) -> SimulatorEnv`, add `--load`
flag to the CLI for loading a serialized plan file, making a
corresponding environment and running the simulation.
We can optionally combine this with a `--save` option, in which we keep
a seed-vault as part of limbo simulator, corresponding each seed with
its generated plan and save the time to regenerate existing seeds by
just loading them into memory. I am curious to hear thoughts on this?
Would the maintainers be open to adding such a seed-vault? Do you think
the saved time would be worth the complexity of the approach?

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #720
2025-01-26 08:52:58 +02:00
alpaylan
e476b9f697 implement watch mode
- add `--watch` flag
- start saving seeds in persistent storage
- make a separate version of execution functions that use `vector of interaction` instead of `InteractionPlan`
2025-01-18 23:54:03 +03:00
Jorge López
0d6e6a0aa4
syntactic changes: use Result::unwrap_or() instead of manual match 2025-01-18 19:20:11 +01:00
alpaylan
c30e2757b4 - implement '--load <PATH>' flag that loads an interaction plan and executes it instead of generating one from scratch
- save a json serialization of the generated plans to `<tempdir>/simulator.plan.json`
2025-01-17 22:04:55 +03:00
alpaylan
28cde537a8 this commit;
- makes interaction plans serializable
- fixes the shadowing bug where non-created tables were assumed to be created in the shadow tables map
- makes small changes to make clippy happy
- reorganizes simulation running flow to remove unnecessary plan regenerations while shrinking and double checking
2025-01-17 01:30:46 +03:00
alpaylan
ecb0f782ac Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-01-15 10:59:46 +03:00
Pekka Enberg
3c118db20d simulator: Welcome banner 2025-01-14 19:15:14 +02:00
Pekka Enberg
0c7ebd4df5 simulator: Enable info-level logging by default 2025-01-14 17:54:39 +02:00
Pekka Enberg
30a380cab1 simulator: Move more logging under trace level 2025-01-14 17:54:29 +02:00
alpaylan
fb937eff7b fix non-determinism bug arising from a call to thread_rng while picking
which row to check existence for in the result of the select query
2025-01-13 17:26:23 +03:00
alpaylan
43f6c34408 fix arbitrary_from ergonomics by removing the implicit reference in the trait signature 2025-01-13 14:43:12 +03:00
alpaylan
82fcc27a58 this commit fixes query generation;
- previous query generation method was faulty, producing wrong assertions
- this commit adds a new arbitrary_from implementation for predicates
- new implementation takes a table and a row, and produces a predicate that would evaluate to true for the row
this commit makes small changes to the main for increasing readability
2025-01-13 02:31:19 +03:00
alpaylan
191b586f05 this commit restructures the interaction generation in order to have
better counterexample minimization.

- it separates interaction plans from their state of execution
- it removes closures from the property definitions, encoding properties as an enum variant, and deriving the closures from the variants.
- it adds some naive counterexample minimization capabilities to the Limbo simulator and reduces the plan sizes considerably.
- it makes small changes to various points of the simulator for better error reporting, enhancing code readability, small fixes to handle previously missed cases
2025-01-11 02:20:22 +03:00
alpaylan
cc56276c3a add execution history to the simulator, the history records
three indexes(connection, interaction pointer, and secondary pointer)
that can uniquely identify the executed interaction at any point.
we will use the history for shrinking purposes.
2025-01-07 13:40:29 +03:00
alpaylan
daa77feea1 add assumptions to the interactions, where a failing assumption stops
the execution of the current property and switches to the next one.
2025-01-06 19:04:29 +03:00
alpaylan
2a4d461627 previously, interactions plans were comprised of a flat sequence of operations that
did not reflect the internal structure, in which they were actually concatenations of
properties, which are a coherent set of interactions that are meaningful by themselves.
this commit introduces this semantic layer into the data model by turning interaction plans
into a sequence of properties, which are a sequence of interactions
2025-01-06 18:16:34 +03:00
alpaylan
58f23983e1 minor changes, add maximum time bound to the simulator, fix bug in the table create shadowing 2024-12-30 00:36:43 -05:00
alpaylan
8e2a1e4289 Merge remote-tracking branch 'turso/main' 2024-12-29 16:08:35 -05:00
alpaylan
d3fee3b331 add empty line at the end of cargo.toml, add create counts to the interaction stats, turn the percentages into f64 2024-12-29 14:00:57 -05:00
PThorpe92
f6cd707544
Add clippy CI, fix or ignore warnings where appropriate 2024-12-29 10:25:41 -05:00
Pekka Enberg
f2ecebc357 Rename RowResult to StepResult
The name "row result" is confusing because it really *is* a result from
a step() call. The only difference is how a row is represented as we
return from VDBE or from a statement.

Therefore, rename RowResult to StepResult.
2024-12-27 10:20:41 +02:00
Alperen Keleş
7524cf052e
Merge branch 'tursodatabase:main' into main 2024-12-26 15:10:40 -05:00
alpaylan
003ad6cc64 allow failure assertions in the simulator, add creating the same table two times to the list of checked properties as a failure property example 2024-12-26 15:10:03 -05:00
Pekka Enberg
548f66e1cd Merge 'fix empty range error when 0 interactions are produced by creating at least 1 interaction' from Alperen Keleş
Fixes the panicking case in
https://github.com/tursodatabase/limbo/issues/548

Closes #549
2024-12-25 19:45:09 +02:00
alpaylan
e49ba4f982 fix empty range error when 0 interactions are produced by creating at least 1 interaction 2024-12-25 09:55:28 -05:00
Pekka Enberg
652283efc1 simulator: Kill dead code
...the old maybe_add_table() codepath as it is not used.
2024-12-25 10:41:05 +02:00
alpaylan
28ae691bf7 switch the seed, database path, and plan path prints to println instead of log::info 2024-12-25 03:04:57 -05:00
Pekka Enberg
49b235cc92 Merge 'core: wal transaction start' from Pere Diaz Bou
This pr adds support for multiple readers and a single writer with a
custom made lock called `LimboRwLock`. Basically there are 5 allowed
read locks which store the max frame allowed in that "snapshot" and any
reader will try to acquire the biggest one possible. Writer will just
try to lock the `write_lock` and if not successful, it will return busy.
The only checkpoint mode supported for now is `PASSIVE` but it should be
trivial to add more modes.
This needs testing, but I will do it in another PR. I just wanted to do
it in another PR.

Closes #544
2024-12-25 09:42:03 +02:00
Pere Diaz Bou
aed14117c9 core: transaction support 2024-12-24 18:04:30 +01:00
alpaylan
2186b3973b change the name of the simulator submodule into runner 2024-12-23 16:16:39 -05:00
alpaylan
4f07342fdc catch panics, add doublecheck 2024-12-22 23:25:35 -05:00