Commit graph

661 commits

Author SHA1 Message Date
Pekka Enberg
9303262dfd OwnedValue -> Value 2025-05-15 11:43:23 +03:00
Pere Diaz Bou
ef65275bda Merge 'test page_free_array' from Pere Diaz Bou
Simply add a fuzz test to test free_array works as intended

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1480
2025-05-15 10:19:09 +02:00
Pekka Enberg
e3f71259d8 Rename OwnedValue -> Value
We have not had enough merge conflicts for a while so let's do a
tree-wide rename.
2025-05-15 09:59:46 +03:00
pedrocarlo
9fc9415b20 use Jussi's code to avoid cloning immutable record 2025-05-14 13:30:39 -03:00
pedrocarlo
72cc0fcdcb fixes and comments 2025-05-14 13:30:39 -03:00
pedrocarlo
b2615d7739 add CursorValidState and only save context in delete when rebalancing 2025-05-14 13:30:39 -03:00
pedrocarlo
814508981c fixing more rebase issues and cleaning up code. Save cursor context when calling delete for later use when needed 2025-05-14 13:30:39 -03:00
pedrocarlo
05f4ca28cc btree rewind and next fix. Keep track of rowids seen to avoid infinite loop 2025-05-14 13:30:39 -03:00
pedrocarlo
c146877344 add sqlite debug cli for nix. Fix cursor delete panic. Add tracing for cell indices in btree 2025-05-14 13:30:39 -03:00
Pere Diaz Bou
5025655f75 debug assertions 2025-05-14 13:53:51 +02:00
Pere Diaz Bou
c3432e1cc3 fmt 2025-05-14 13:23:12 +02:00
Pere Diaz Bou
13a120530e
comment shift_cells_left
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-14 13:20:11 +02:00
Pere Diaz Bou
0d8a94d49d
expect parse seed in rng_from_time_or_env
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-14 13:19:20 +02:00
Pere Diaz Bou
34c74e34d9 test page_free_array
Simply add a fuzz test to test free_array works as intended
2025-05-14 12:48:59 +02:00
krishvishal
f6e814c113 Refactor read_write_payload_to_page into separate read and write
functions
2025-05-14 13:03:23 +05:30
krishvishal
fb9fcb7887 Remove prints and change overflow test blob to be 10 pages long. 2025-05-13 21:19:49 +05:30
krishvishal
c424e830cd Add safety documentation 2025-05-13 16:57:44 +05:30
krishvishal
416c88d67d Add comments to the code. 2025-05-13 16:51:37 +05:30
krishvishal
6628757c12 Added unit test to check case of read-write payload with overflow 2025-05-13 16:36:46 +05:30
krishvishal
93ad65e512 Extend read_write_payload_with_offset to handle offset into
overflow pages.
2025-05-13 15:28:58 +05:30
krishvishal
cffb731e4c Fix read_write_payload_with_offset and add a test that verifies
you can read and write from a column.
2025-05-12 22:55:11 +05:30
Jussi Saurio
2b6b09d435 Merge 'btree: Coalesce free blocks in page_free_array()' from Mohamed Hossam
Coalesce adjacent free blocks during `page_free_array()` in
`core/storage/btree`.
Instead of immediately passing free cells to `free_cell_range()`, buffer
up to 10 free cells and try to merge adjacent free blocks. Break on the
first merge to avoid time complexity, `free_cell_range()` coalesces
blocks afterwards anyways. This follows SQLite's [`pageFreeArray()`](htt
ps://github.com/sqlite/sqlite/blob/d7324103b196c572a98724a5658970b4000b8
c39/src/btree.c#L7729) implementation.
Removed this TODO:
```rust
fn page_free_array( . . )
    .
    .
    // TODO: implement fancy smart free block coalescing procedure instead of dumb free to
    // then defragment
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1448
2025-05-12 16:52:11 +03:00
m0hossam
70b9438f80 Add comments 2025-05-12 15:58:36 +03:00
krishvishal
f6cc2c3cd8 Initial implementation of read_write_payload_with_offset
This is a port of `accessPayload` in `btree.c`. This port gives
us the ability to read and write from a payload at a set offset an
-d amount. This will be used in the upcoming PRs that will add
incremental IO for blobs.
2025-05-12 15:31:22 +05:30
pedrocarlo
977d09fd36 small fixes 2025-05-10 22:23:01 -03:00
pedrocarlo
342bf51c88 remove state machine for count 2025-05-10 22:23:01 -03:00
pedrocarlo
655ceeca45 correct count implementation 2025-05-10 22:23:01 -03:00
m0hossam
aefce20460 Try to coalesce free blocks before freeing 2025-05-04 22:14:51 +03:00
PThorpe92
d4cf8367ba
Wrap return_if_locked in balance non root in debug assertion cfg 2025-05-02 10:55:00 -04:00
PThorpe92
f025f7e91e
Fix panic on async io due to reading locked page 2025-05-02 10:55:00 -04:00
Jussi Saurio
6096cfb3d8 Merge 'Add PRAGMA schema_version' from Anton Harniakou
This PR adds `PRAGMA schema_version` to get the value of the schema-
version integer at offset 40 in the database header.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1427
2025-05-01 10:50:02 +03:00
Jussi Saurio
a525feb7ad Merge 'Fix: allow page_size=65536' from meteorgan
Since `page_size` in `DatabaseHeader` can be 1 representing 65526 bytes,
it can't be used it directly.  Additionally, we should use `u32` instead
of `u16` or `usize` in other contexts.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1411
2025-05-01 10:46:19 +03:00
Anton Harniakou
525b7fdbaa Add PRAGMA schema_version 2025-04-30 09:41:04 +03:00
Pere Diaz Bou
a30241ca91 Add state machine for op_idx_delete + DeleteState simplification
DeleteState had a bit too many unnecessary states so I removed them.
Usually we care about having a different state when I/O is triggered
requiring a state to be stored for later.

Furthermore, there was a bug with op_idx_delete where if balance is
triggered, op_idx_delete wouldn't be re-entrant. So a state machine was
added to prevent that from happening.
2025-04-29 14:58:20 +03:00
meteorgan
d1a50f8a69 skip unneccessary conversion 2025-04-28 16:13:07 +08:00
meteorgan
d2dce740f7 fix some issues about page_size 2025-04-28 16:13:07 +08:00
Jussi Saurio
3459c1f7dd Merge 'btree/tablebtree_move_to: micro-optimizations' from Jussi Saurio
```bash
jussi@Jussis-MacBook-Pro limbo % git co main && cargo build --bin limbo --release && hyperfine --shell=none --warmup 5 './target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = '\''FURNITURE'\'' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('\''1995-03-29'\'' as datetime) and l_shipdate > cast('\''1995-03-29'\'' as datetime);"'

...

Benchmark 1: ./target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = 'FURNITURE' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('1995-03-29' as datetime) and l_shipdate > cast('1995-03-29' as datetime);"
  Time (mean ± σ):      2.104 s ±  0.006 s    [User: 1.952 s, System: 0.151 s]
  Range (min … max):    2.094 s …  2.115 s    10 runs

jussi@Jussis-MacBook-Pro limbo % git co move-to-micro-opt && cargo build --bin limbo --release && hyperfine --shell=none --warmup 5 './target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = '\''FURNITURE'\'' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('\''1995-03-29'\'' as datetime) and l_shipdate > cast('\''1995-03-29'\'' as datetime);"'

...

Benchmark 1: ./target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = 'FURNITURE' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('1995-03-29' as datetime) and l_shipdate > cast('1995-03-29' as datetime);"
  Time (mean ± σ):      1.883 s ±  0.012 s    [User: 1.733 s, System: 0.146 s]
  Range (min … max):    1.866 s …  1.908 s    10 runs
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1408
2025-04-28 10:27:59 +03:00
Pere Diaz Bou
63a94e7c62 Merge 'Emit IdxDelete instruction and some fixes on seek after deletion' from Pere Diaz Bou
Previously `DELETE FROM ...` only emitted deletes for main table, but
this is incorrect as we want to remove entries from index tables as
well.

Closes #1383
2025-04-28 09:13:54 +03:00
Jussi Saurio
46d45a6bf4 don't recompute cell_count 2025-04-26 14:50:42 +03:00
Jussi Saurio
75c6678a06 sqlite3_ondisk: use debug asserts for cell_table_interior_read... funcs 2025-04-26 14:50:42 +03:00
Jussi Saurio
ac1bc17ea4 btree/tablebtree_seek: remove some more useless calls to set_cell_index() 2025-04-26 13:41:30 +03:00
Jussi Saurio
5060f1a1fa don't use integer division in binary search halfpoint calculation 2025-04-26 12:38:15 +03:00
Jussi Saurio
23ae5a27c4 btree/tablebtree_move_to: micro-optimizations 2025-04-26 11:32:24 +03:00
Jussi Saurio
3553d05c32 Merge 'Give name to hard-coded page_size values' from Anton Harniakou
Related to #1379
I guess there are more hard-coded values.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1404
2025-04-25 22:03:43 +03:00
Anton Harniakou
dd7c0ad1c8 Give name to hard-coded page_size values 2025-04-25 14:15:15 +03:00
Jussi Saurio
7921d7c2e0 types: refactor serialtype again to make it faster 2025-04-24 17:28:31 +03:00
Jussi Saurio
04adf8242a faster validate 2025-04-24 16:05:12 +03:00
Jussi Saurio
af6a783f4d core/types: remove duplicate serialtype implementation 2025-04-24 15:38:47 +03:00
Jussi Saurio
dc3e97887f Merge 'replace vec with array in btree balancing' from Lâm Hoàng Phúc
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1354
2025-04-24 11:22:07 +03:00
Jussi Saurio
af703110f8 btree: remove extra iter_dir argument that can be derived from seek_op 2025-04-23 17:38:48 +03:00