Commit graph

185 commits

Author SHA1 Message Date
Luca Muscat
a5ac1884c1 core: Stop treating the contents of a Value::Blob as a String
By encoding a Vec<u8> (vector of bytes), a lossy conversion from a
`Vec<u8>` to a `String` occurs. The lossy conversion leads to an
incorrect hex value to be displayed.

Avoid the lossy conversion and let the `hex` crate do its thing.
2025-06-16 21:10:17 +02:00
Pekka Enberg
db4945eada Merge 'Fix update queries to set n_changes ' from Kim Seon Woo
- `Update` query doesn't update `n_changes`. Let's make it work
- Add `InsertFlags` to add meta information related to insert operations
- For update query, add `UPDATE` flag
- Currently, the update query executes `Insn::Delete` and `Insn::Insert`
internally, it increases `n_change` by 2. So, for the update query,
let's skip increasing `n_change` for the `Insn::Insert`
https://github.com/tursodatabase/limbo/issues/1681

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1683
2025-06-16 16:30:20 +03:00
Pekka Enberg
4496a0d08a core: Clean up integrity_check()
Suggested by Jussi.
2025-06-16 14:46:36 +03:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Pekka Enberg
90c1e3fc06 Switch Connection to use Arc instead of Rc
Connection needs to be Arc so that bindings can wrap it with `Mutex` for
multi-threading.
2025-06-16 10:43:19 +03:00
meteorgan
b910fd0012 drop unused code in op_column 2025-06-15 00:50:16 +08:00
Jussi Saurio
30e4511d62 Merge 'NOT NULL constraint' from Anton Harniakou
Some checks failed
Python / sdist (push) Has been cancelled
Python / macos-arm64 (aarch64) (push) Has been cancelled
JavaScript / stable - x86_64-apple-darwin - node@20 (push) Has been cancelled
JavaScript / stable - x86_64-pc-windows-msvc - node@20 (push) Has been cancelled
JavaScript / stable - x86_64-unknown-linux-gnu - node@20 (push) Has been cancelled
Rust / cargo-fmt-check (push) Has been cancelled
Python / configure-strategy (push) Has been cancelled
Python / lint (push) Has been cancelled
Python / check-requirements (push) Has been cancelled
Python / linux (x86_64) (push) Has been cancelled
Python / macos-x86_64 (x86_64) (push) Has been cancelled
Rust / build-native (blacksmith-4vcpu-ubuntu-2404) (push) Has been cancelled
Rust / build-native (macos-latest) (push) Has been cancelled
Rust / build-native (windows-latest) (push) Has been cancelled
Rust / clippy (push) Has been cancelled
Rust / build-wasm (push) Has been cancelled
Rust / test-limbo (push) Has been cancelled
Rust / test-sqlite (push) Has been cancelled
Rust Benchmarks+Nyrkiö / bench (push) Has been cancelled
Rust Benchmarks+Nyrkiö / clickbench (push) Has been cancelled
Rust Benchmarks+Nyrkiö / tpc-h-criterion (push) Has been cancelled
Rust Benchmarks+Nyrkiö / tpc-h (push) Has been cancelled
JavaScript / Test bindings on x86_64-apple-darwin - node@18 (push) Has been cancelled
JavaScript / Test bindings on x86_64-apple-darwin - node@20 (push) Has been cancelled
JavaScript / Test bindings on Linux-x64-gnu - node@18 (push) Has been cancelled
JavaScript / Test bindings on Linux-x64-gnu - node@20 (push) Has been cancelled
JavaScript / Build universal macOS binary (push) Has been cancelled
JavaScript / Publish (push) Has been cancelled
Python / test (push) Has been cancelled
Python / Release (push) Has been cancelled
Implements basic support for `NOT NULL` contraint check.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1675
2025-06-13 14:25:24 +03:00
Jussi Saurio
3f90fad131 Merge 'Simulator Ast Generation + Simulator Unary Operator + Refactor to use limbo_core::Value in Simulator for massive code reuse' from Pedro Muniz
This PR is a Drop-In replacement to the Predicate defined in the
Simulator. Predicate is basically the same as our ast::Expr, but it
supports a small number of the SQL expression syntax. By creating a
NewType that wraps ast::Expr we can tap into our already mostly
correctly defined parser structs. This change will enable us to easily
add generation for more types of sql queries.
I also added an ArbitraryFrom impl for ast::Expr that can be used in a
freestyle way (for now) for differential testing.
This PR also aims to implement Unary Operator logic similar to the
Binary Operator logic we have for predicate. After this change we may
need to adjust the Logic for how some assertions are triggered.
<s>Sometimes the `Select-Select-Optimizer` property thinks that these
two queries should return the same thing:
```sql
SELECT (twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783) FROM twinkling_winstanley WHERE 1;

SELECT * FROM twinkling_winstanley WHERE twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783;
```
However after running the shrunk plan manually, the simulator was
incorrect in asserting that. Maybe this a bug a in the generation of
such a query? Not sure yet. </s>
<b>EDIT: The simulator was correctly catching a bug and I thought I was
the problem. The bug was in `exec_if` and I fixed it in this PR.</b>
I still need to expand the Unary Operator generation to other types of
predicates. For now, I just implemented it for `SimplePredicate` as I'm
trying to avoid to bloat even more this PR.
<b>EDIT: I decided to just have one PR open for all the changes I'm
making to make my life a bit easier and to avoid merge conflicts with my
own branches that I keep spawning for new code.</b>
PS: This should only be considered for merging after
https://github.com/tursodatabase/limbo/pull/1619 is merged. Then, I will
remove the draft status from this PR.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1674
2025-06-13 11:23:10 +03:00
Jussi Saurio
efed4bc07d Merge 'Complete ALTER TABLE implementation' from Levy A.
Resolves #895
- [x] `ALTER TABLE _ ADD _`
- [x] `ALTER TABLE _ DROP _`
- [x] `ALTER TABLE _ RENAME _ TO _`

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1527
2025-06-13 10:23:43 +03:00
Jussi Saurio
04c590f5a6 Add comment about re-entrancy of op_idx_insert() 2025-06-12 12:36:53 +03:00
Jussi Saurio
bdbd021bbb Fix large inserts to unique indexes hanging
We were incorrectly setting `moved_before` as `false` after checking
for unique constraint violation, but the reality is that after the
uniqueness check, we are already correctly positioned -- if no match
was found, the cursor is positioned at either:

a) no row, or
b) at the first entry that is greater than the key we are inserting.

This means we don't have to move anymore and can just insert.
2025-06-12 12:25:17 +03:00
Levy A.
43db84e6ea fix: broken rebase 2025-06-11 14:19:06 -03:00
Levy A.
49a6ddad97 wip 2025-06-11 14:19:04 -03:00
Levy A.
15e0cab8d8 refactor+fix: precompute default values from schema 2025-06-11 14:18:39 -03:00
Levy A.
7638b0dab7 fix: use default value on empty columns added via ALTER TABLE 2025-06-11 14:18:19 -03:00
Pere Diaz Bou
3f3e91927f fmt 2025-06-11 19:02:23 +02:00
Pere Diaz Bou
d3c646378a Cell coverage checker
We check cells and freeblocks do not overlap and the fragmentation is
correct.
2025-06-11 16:50:30 +02:00
pedrocarlo
cf34e21ba9 fix for bug caught in simulator regarding exec_if 2025-06-11 11:32:45 -03:00
Pere Diaz Bou
9383ba207d introduce integrity_check pragma 2025-06-11 11:14:29 +02:00
Krishna Vishal
1c6a65ded4 Change seek op match from unit variants to struct variants. 2025-06-11 00:44:07 +05:30
krishvishal
5a1da026e6 Unify comparison function to reduce code duplication 2025-06-11 00:33:47 +05:30
krishvishal
5837f7329f clean up 2025-06-11 00:33:47 +05:30
krishvishal
d13abad4b1 Handle Blob type together with Text type in op_seek 2025-06-11 00:33:47 +05:30
krishvishal
faa9aedbae Add affinity based type coercion to comparison ops 2025-06-11 00:33:47 +05:30
krishvishal
3b2980c7c0 Fix op_seek to handle affinity coercion 2025-06-11 00:33:05 +05:30
krishvishal
e68293a1d1 Add affinity conversion to op_gt, op_le, op_lt, op_eq, op_ne 2025-06-11 00:33:05 +05:30
krishvishal
30ccbe46c7 Added apply_numeric_affinity function to handle string conversion
to integer. Exising functions' behavior is tailored to `CAST` ops.
SQLite has different behavior when it comes to handling string to
`integer` conversion in CAST vs predicate ops.
2025-06-11 00:33:05 +05:30
krishvishal
20e6e73057 Fix affinity in op_seek when there's string integer comparison 2025-06-11 00:33:03 +05:30
krishvishal
2171c5f4e1 Added apply_affinity_char to fix bugs in SeekRowId 2025-06-11 00:31:09 +05:30
Jussi Saurio
07c947b47d use GE for idx delete seek, although it doesnt really matter 2025-06-10 14:16:26 +03:00
Jussi Saurio
976c2f72ef OpIdxDeleteState needs another state to be re-entrant 2025-06-10 14:16:26 +03:00
Jussi Saurio
a5aeff9a3d Fix index insert accidentally double-inserting after balance 2025-06-10 14:16:26 +03:00
Jussi Saurio
844461d20b update and delete fixes 2025-06-10 14:16:26 +03:00
Jussi Saurio
d81f5f67bd insert spaghetti fixes 2025-06-10 14:16:26 +03:00
Jussi Saurio
1b4bef9c7c Fix op_idx_delete infinite seeking loop 2025-06-10 14:16:26 +03:00
Jussi Saurio
2bac140d73 Remove SeekOp::EQ and encode eq_only in LE&GE - needed for iteration direction aware equality seeks 2025-06-10 14:16:26 +03:00
Pere Diaz Bou
681df9b1eb fix get record 2025-06-10 14:16:26 +03:00
Pere Diaz Bou
77b6896eae implement lazy record and rowid in cursor
This also comments save_context for now
2025-06-10 14:16:26 +03:00
pedrocarlo
a9ed8dd288 namespace exec_min and exec_max to Value for reuse in simulator 2025-06-09 11:59:44 -03:00
pedrocarlo
39f85ffd03 namespace exec_like to Value 2025-06-09 11:39:55 -03:00
pedrocarlo
6c95a88533 namespace many functions to Value 2025-06-09 11:38:15 -03:00
Jussi Saurio
2075e5f3eb Fix UPDATE always inserting only nulls into non-unique indexes 2025-06-09 08:51:23 +03:00
김선우
0193e1525b Nit 2025-06-07 17:51:24 +09:00
김선우
f52f881bc8 Remove unnecessary comment 2025-06-07 17:44:47 +09:00
김선우
a9c096bb01 Skip increasing n_changes for insn::Insert when it's UPDATE query 2025-06-07 17:23:23 +09:00
Zaid Humayun
e994adfb40 Persisting database header and pointer map page to cache
This commit ensures that the metadata in the database header and the pointer map pages allocated are correctly persisted to the page cache. This was not being done earlier.
2025-06-06 23:14:25 +05:30
Zaid Humayun
1f5025541c addresses comment https://github.com/tursodatabase/limbo/pull/1600#discussion_r2115796655 by @jussisaurio
this commit ensures that ptrmap operations return a CursorResult so operation can be suspended & later retried
2025-06-06 23:14:25 +05:30
Zaid Humayun
5827a33517 Beginnings of AUTOVACUUM
This commit introduces AUTOVACUUM to Limbo. It introduces the concept of ptrmap pages and also adds some additional instructions that are required to make AUTOVACUUM PRAGMA work
2025-06-06 23:14:22 +05:30
Anton Harniakou
a83ee73f20 Correct not null error message 2025-06-06 12:56:14 +03:00
Anton Harniakou
fb86476525 Implement basic not null constraint checks 2025-06-05 19:02:31 +03:00