Commit graph

147 commits

Author SHA1 Message Date
Pekka Enberg
db4945eada Merge 'Fix update queries to set n_changes ' from Kim Seon Woo
- `Update` query doesn't update `n_changes`. Let's make it work
- Add `InsertFlags` to add meta information related to insert operations
- For update query, add `UPDATE` flag
- Currently, the update query executes `Insn::Delete` and `Insn::Insert`
internally, it increases `n_change` by 2. So, for the update query,
let's skip increasing `n_change` for the `Insn::Insert`
https://github.com/tursodatabase/limbo/issues/1681

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1683
2025-06-16 16:30:20 +03:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Jussi Saurio
30e4511d62 Merge 'NOT NULL constraint' from Anton Harniakou
Some checks failed
Python / sdist (push) Has been cancelled
Python / macos-arm64 (aarch64) (push) Has been cancelled
JavaScript / stable - x86_64-apple-darwin - node@20 (push) Has been cancelled
JavaScript / stable - x86_64-pc-windows-msvc - node@20 (push) Has been cancelled
JavaScript / stable - x86_64-unknown-linux-gnu - node@20 (push) Has been cancelled
Rust / cargo-fmt-check (push) Has been cancelled
Python / configure-strategy (push) Has been cancelled
Python / lint (push) Has been cancelled
Python / check-requirements (push) Has been cancelled
Python / linux (x86_64) (push) Has been cancelled
Python / macos-x86_64 (x86_64) (push) Has been cancelled
Rust / build-native (blacksmith-4vcpu-ubuntu-2404) (push) Has been cancelled
Rust / build-native (macos-latest) (push) Has been cancelled
Rust / build-native (windows-latest) (push) Has been cancelled
Rust / clippy (push) Has been cancelled
Rust / build-wasm (push) Has been cancelled
Rust / test-limbo (push) Has been cancelled
Rust / test-sqlite (push) Has been cancelled
Rust Benchmarks+Nyrkiö / bench (push) Has been cancelled
Rust Benchmarks+Nyrkiö / clickbench (push) Has been cancelled
Rust Benchmarks+Nyrkiö / tpc-h-criterion (push) Has been cancelled
Rust Benchmarks+Nyrkiö / tpc-h (push) Has been cancelled
JavaScript / Test bindings on x86_64-apple-darwin - node@18 (push) Has been cancelled
JavaScript / Test bindings on x86_64-apple-darwin - node@20 (push) Has been cancelled
JavaScript / Test bindings on Linux-x64-gnu - node@18 (push) Has been cancelled
JavaScript / Test bindings on Linux-x64-gnu - node@20 (push) Has been cancelled
JavaScript / Build universal macOS binary (push) Has been cancelled
JavaScript / Publish (push) Has been cancelled
Python / test (push) Has been cancelled
Python / Release (push) Has been cancelled
Implements basic support for `NOT NULL` contraint check.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1675
2025-06-13 14:25:24 +03:00
Anton Harniakou
25d0584b19 Fix typo 2025-06-13 11:57:39 +03:00
Levy A.
15e0cab8d8 refactor+fix: precompute default values from schema 2025-06-11 14:18:39 -03:00
Levy A.
de2ac89ad2 feat: complete ALTER TABLE implementation 2025-06-11 14:17:36 -03:00
Pere Diaz Bou
9383ba207d introduce integrity_check pragma 2025-06-11 11:14:29 +02:00
Jussi Saurio
2bac140d73 Remove SeekOp::EQ and encode eq_only in LE&GE - needed for iteration direction aware equality seeks 2025-06-10 14:16:26 +03:00
김선우
a9c096bb01 Skip increasing n_changes for insn::Insert when it's UPDATE query 2025-06-07 17:23:23 +09:00
Zaid Humayun
5827a33517 Beginnings of AUTOVACUUM
This commit introduces AUTOVACUUM to Limbo. It introduces the concept of ptrmap pages and also adds some additional instructions that are required to make AUTOVACUUM PRAGMA work
2025-06-06 23:14:22 +05:30
Anton Harniakou
fb86476525 Implement basic not null constraint checks 2025-06-05 19:02:31 +03:00
pedrocarlo
c9c73f2497 fix explain panicking on None CursorKey 2025-05-31 01:19:26 -03:00
Jussi Saurio
77ce4780d9 Fix ProgramBuilder::cursor_ref not having unique keys
Currently we have this:

program.alloc_cursor_id(Option<String>, CursorType)`

where the String is the table's name or alias ('users' or 'u' in
the query).

This is problematic because this can happen:

`SELECT * FROM t WHERE EXISTS (SELECT * FROM t)`

There are two cursors, both with identifier 't'. This causes a bug
where the program will use the same cursor for both the main query
and the subquery, since they are keyed by 't'.

Instead introduce `CursorKey`, which is a combination of:

1. `TableInternalId`, and
2. index name (Option<String> -- in case of index cursors.

This should provide key uniqueness for cursors:

`SELECT * FROM t WHERE EXISTS (SELECT * FROM t)`

here the first 't' will have a different `TableInternalId` than the
second `t`, so there is no clash.
2025-05-29 00:59:24 +03:00
Jussi Saurio
ad0f2bb399 Merge 'Small VDBE insn tweaks' from Jussi Saurio
1. allow calling op_null with Insn::BeginSubrtn
    - BeginSubrtn is identical to Null, but named differently so that
its use in context is clearer
2. Insn::Return: add possibility to fallthrough on non-integer values as
per sqlite spec

Closes #1588
2025-05-27 20:19:31 +03:00
meteorgan
d9d3a5ecbb Use the SetCookie opcode to implement user_version pragma 2025-05-28 00:31:11 +08:00
Jussi Saurio
70965f4b28 Insn::Return: add possibility to fallthrough on non-integer values as per sqlite spec 2025-05-27 19:09:10 +03:00
pedrocarlo
e3fd1e589e support using a INSERT SELECT that references the same table in both statements 2025-05-25 19:15:28 -03:00
Piotr Rzysko
9c1dca72db Introduce VTable
This allows storing table arguments parsed in the VTabModule::create
method.
2025-05-21 08:33:17 +02:00
Pekka Enberg
e102cd0be5 Merge 'Add support for DISTINCT aggregate functions' from Jussi Saurio
Reviewable commit by commit. CI failures are not related.
Adds support for e.g. `select first_name, sum(distinct age),
count(distinct age), avg(distinct age) from users group by 1`
Implementation details:
- Creates an ephemeral index per distinct aggregate, and jumps over the
accumulation step if a duplicate is found

Closes #1507
2025-05-20 13:58:57 +03:00
pedrocarlo
4a3119786e refactor BtreeCursor and Sorter to accept Vec of collations 2025-05-19 15:22:55 -03:00
pedrocarlo
5bd47d7462 post rebase adjustments to accomodate new instructions that were created before the merge conflicts 2025-05-19 15:22:15 -03:00
pedrocarlo
bf1fe9e0b3 Actually fixed group by and order by collation 2025-05-19 15:22:15 -03:00
pedrocarlo
0df6c87f07 Fixed Group By collation 2025-05-19 15:22:14 -03:00
pedrocarlo
a818b6924c Removed repeated binary expression translation. Adjusted the set_collation to capture additional context of whether it was set by a Collate expression or not. Added some tests to prove those modifications were necessary. 2025-05-19 15:22:14 -03:00
pedrocarlo
d0a63429a6 Naive implementation of collate for queries. Not implemented for column constraints 2025-05-19 15:22:14 -03:00
Jussi Saurio
8d66347729 vdbe: add Insn::Found 2025-05-17 15:33:55 +03:00
Pekka Enberg
e3f71259d8 Rename OwnedValue -> Value
We have not had enough merge conflicts for a while so let's do a
tree-wide rename.
2025-05-15 09:59:46 +03:00
pedrocarlo
3aaf4206b7 altered constraint tests to create bad update statements. Tests caught a bug where I was copying the wrong values from the registers 2025-05-14 13:30:39 -03:00
pedrocarlo
6457d7675a instruction emitted should be correct, but having an infinite loop bug 2025-05-14 13:30:39 -03:00
pedrocarlo
5f2216cf8e modify explain for MakeRecord to show index name 2025-05-14 13:30:39 -03:00
pedrocarlo
5bae32fe3f modified OpenWrite to include index or table name in explain 2025-05-14 13:30:39 -03:00
pedrocarlo
b69debb8c0 create count opcode 2025-05-10 22:23:01 -03:00
Pekka Enberg
14ef25ebb8 Merge 'Add drop index' from Anton Harniakou
This commit adds suport for DROP INDEX.
Bytecode produced by this commit differs from SQLITE's bytecode, main
reason we don't do autovacuum or repacking of pages like SQLITE does.
Closes #1280

Closes #1444
2025-05-10 08:04:39 +03:00
Pekka Enberg
be1621e099 Merge 'EXPLAIN should show a comment for the Insert opcode' from Anton Harniakou
After this commit EXPLAIN should show a comment for `Insert`.
```
limbo> explain insert into t (age, name, id) values (20, 'max', 1);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     9     0                    0   Start at 9
1     OpenWrite          0     2     0                    0
2     Integer            1     2     0                    0   r[2]=1
3     String8            0     3     0     max            0   r[3]='max'
4     Integer            20    4     0                    0   r[4]=20
5     NewRowId           0     1     0                    0
6     MakeRecord         2     3     5                    0   r[5]=mkrec(r[2..4])
7     Insert             0     5     1     t              0   intkey=r[1] data=r[5]
8     Halt               0     0     0                    0
9     Transaction        0     1     0                    0   write=true
10    Goto               0     1     0                    0

```

Closes #1452
2025-05-10 07:59:36 +03:00
Levy A.
023a116b0d feat: initial implementation of ALTER TABLE
only supporting renaming tables
2025-05-08 09:24:56 -03:00
Anton Harniakou
d74df2473e EXPLAIN should show a comment for the Insert opcode 2025-05-05 10:54:59 +03:00
Anton Harniakou
a971bce353 Show explanation for the NewRowid opcode 2025-05-04 14:14:19 +03:00
Anton Harniakou
2432e0561e Fix clippy warnings 2025-05-04 13:02:54 +03:00
Anton Harniakou
6c8eef2fac Support DROP INDEX
This commit adds suport for DROP INDEX.
Bytecode produced by this commit differs from SQLITE's bytecode, main
reason we don't do autovacuum or repacking of pages like SQLITE does.
2025-05-04 12:13:16 +03:00
Jussi Saurio
306e097950 Merge 'Fix bug: we cant remove order by terms from the head of the list' from Jussi Saurio
we had an incorrect optimization in `eliminate_orderby_like_groupby()`
where it could remove e.g. the first term of the ORDER BY if it matched
the first GROUP BY term and the result set was naturally ordered by that
term. this is invalid. see e.g.:
```sql
main branch - BAD: removes the `ORDER BY id` term because the results are naturally ordered by id.
However, this results in sorting the entire thing by last name only!

limbo> select id, last_name, count(1) from users GROUP BY 1,2 order by id, last_name desc limit 3;
┌──────┬───────────┬───────────┐
│ id   │ last_name │ count (1) │
├──────┼───────────┼───────────┤
│ 6235 │ Zuniga    │         1 │
├──────┼───────────┼───────────┤
│ 8043 │ Zuniga    │         1 │
├──────┼───────────┼───────────┤
│  944 │ Zimmerman │         1 │
└──────┴───────────┴───────────┘

after fix - GOOD:

limbo> select id, last_name, count(1) from users GROUP BY 1,2 order by id, last_name desc limit 3;
┌────┬───────────┬───────────┐
│ id │ last_name │ count (1) │
├────┼───────────┼───────────┤
│  1 │ Foster    │         1 │
├────┼───────────┼───────────┤
│  2 │ Salazar   │         1 │
├────┼───────────┼───────────┤
│  3 │ Perry     │         1 │
└────┴───────────┴───────────┘

I also refactored sorters to always use the ast `SortOrder` instead of boolean vectors, and use the `compare_immutable()` utility we use inside btrees too.

Closes #1365
2025-05-03 12:48:08 +03:00
Pere Diaz Bou
63a94e7c62 Merge 'Emit IdxDelete instruction and some fixes on seek after deletion' from Pere Diaz Bou
Previously `DELETE FROM ...` only emitted deletes for main table, but
this is incorrect as we want to remove entries from index tables as
well.

Closes #1383
2025-04-28 09:13:54 +03:00
Pere Diaz Bou
b7970a286d implement IdxDelete
clippy

revert op_idx_ge changes

fmt

fmt again

rever op_idx_gt changes
2025-04-24 16:23:34 +02:00
Jussi Saurio
3798b4aa8b use SortOrder in sorters always 2025-04-24 10:34:06 +03:00
pedrocarlo
b6036cc79d Primary key constraint working 2025-04-23 16:44:13 -03:00
Jussi Saurio
13a703d636 Merge 'Add BeginSubrtn, NotFound and Affinity bytecodes' from Diego Reis
I'm working on an optimization of `WHERE .. IN (..)` statements that
requires these bytecodes.
```sh
sqlite> explain select * from users where first_name in ('alice', 'bob', 'charlie');
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     35    0                    0   Start at 35
1     OpenRead       0     2     0     10             0   root=2 iDb=0; users
2     Rewind         0     34    0                    0
3       Noop           0     0     0                    0   begin IN expr
4       BeginSubrtn    0     1     0                    0   r[1]=NULL <---- Here
5         Once           0     17    0                    0
6         OpenEphemeral  1     1     0     k(1,B)         0   nColumn=1; RHS of IN operator
7         String8        0     2     0     alice          0   r[2]='alice'
8         MakeRecord     2     1     3     B              0   r[3]=mkrec(r[2])
9         IdxInsert      1     3     2     1              0   key=r[3]
10        String8        0     2     0     bob            0   r[2]='bob'
11        MakeRecord     2     1     3     B              0   r[3]=mkrec(r[2])
12        IdxInsert      1     3     2     1              0   key=r[3]
13        String8        0     2     0     charlie        0   r[2]='charlie'
14        MakeRecord     2     1     3     B              0   r[3]=mkrec(r[2])
15        IdxInsert      1     3     2     1              0   key=r[3]
16        NullRow        1     0     0                    0
17      Return         1     5     1                    0
18      Column         0     1     4                    0   r[4]= cursor 0 column 1
19      IsNull         4     33    0                    0   if r[4]==NULL goto 33
20      Affinity       4     1     0     B              0   affinity(r[4]) <---- Here
21      NotFound       1     33    4     1              0   key=r[4]; end IN expr <---- Here
22      Rowid          0     5     0                    0   r[5]=users.rowid
23      Column         0     1     6                    0   r[6]= cursor 0 column 1
24      Column         0     2     7                    0   r[7]= cursor 0 column 2
25      Column         0     3     8                    0   r[8]= cursor 0 column 3
26      Column         0     4     9                    0   r[9]= cursor 0 column 4
27      Column         0     5     10                   0   r[10]= cursor 0 column 5
28      Column         0     6     11                   0   r[11]= cursor 0 column 6
29      Column         0     7     12                   0   r[12]= cursor 0 column 7
30      Column         0     8     13                   0   r[13]= cursor 0 column 8
31      Column         0     9     14                   0   r[14]= cursor 0 column 9
32      ResultRow      5     10    0                    0   output=r[5..14]
33    Next           0     3     0                    1
34    Halt           0     0     0                    0
35    Transaction    0     0     3     0              1   usesStmtJournal=0
36    Goto           0     1     0                    0
```
EDIT: [Found](https://sqlite.org/opcode.html#Found) and
[NoConflict](https://sqlite.org/opcode.html#NoConflict) can be easily
derived from `NotFound` but I wanted to be concise, I could do it in
another PR :)

Closes #1345
2025-04-15 20:25:55 +03:00
Diego Reis
58efb90467 core: Add Affinity bytecode
Apply affinities to a range of P2 registers starting with P1.

P4 is a string that is P2 characters long. The N-th character of the string indicates the column affinity that should be used for the N-th memory cell in the range.
2025-04-15 09:52:04 -03:00
Diego Reis
c5161311fc core/vdbe: Add NotFound bytecode
If P4==0 then register P3 holds a blob constructed by MakeRecord. If P4>0 then register P3 is the first of P4 registers that form an unpacked record.

Cursor P1 is on an index btree. If the record identified by P3 and P4 is not the prefix of any entry in P1 then a jump is made to P2. If P1 does contain an entry whose prefix matches the P3/P4 record then control falls through to the next instruction and P1 is left pointing at the matching entry.

This operation leaves the cursor in a state where it cannot be advanced in either direction. In other words, the Next and Prev opcodes do not work after this operation.
2025-04-15 09:52:04 -03:00
Diego Reis
825aeb3f83 core/vdbe: Add BeginSubrtn bytecode
Basically it does the very same thing of Null, but has a different name to differentiate its usage.
2025-04-15 09:52:04 -03:00
Jussi Saurio
e299a0e77e vdbe: add Insn::IdxRowId 2025-04-15 15:13:18 +03:00
Jussi Saurio
5a38b38e71 Merge 'Feature: VDestroy for Dropping Virtual Tables' from Pedro Muniz
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1274
2025-04-15 14:34:30 +03:00