Commit graph

661 commits

Author SHA1 Message Date
Pere Diaz Bou
1e5239a164 use correct min/max frames 2024-12-13 14:06:29 +01:00
Pere Diaz Bou
b43e8e46f6 impl sync/send for cache 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
97dd95abea core: change Rc<RefCell<Page>> to Arc<Page>
This includes an inner struct in Page wrapped with Unsafe cell to access
it. This is done intentionally because concurrency control of pages is
handled by pages and not by the page itself.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
c816186326 fmt 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
3fda2d09b9 Extract multi threaded part from WalFile to WalFileShared
Since we expect to ensure thread safety between multiple threads in the
future, we extract what is important to be shared between multiple
connections with regards to WAL.

This is WIP so I just put whatever feels like important behind a RwLock
but expect this to change to Atomics in the future as needed. Maybe even
these locks might disappear because they will be better served with
transaction locks.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
a4297702bd extract page cache to be multi threaded 2024-12-13 13:09:13 +01:00
krishvishal
7e2928a5f1 Feature: last_insert_rowid()
- Changed `Cursor` trait to be able to get access to `root_page`
- SQLite only updates last_insert_rowid for non-schema inserts. So we check if the `InsertAwait` is not for `root_page` before
  updating rowid
2024-12-09 22:48:42 +05:30
Lauri Virtanen
afeb1cbe74
Clippy warning fixes 2024-11-24 20:24:47 +02:00
Lauri Virtanen
a7100d8e9b
Autofix clippy issues with cargo fix --clippy 2024-11-24 20:24:47 +02:00
Pekka Enberg
aa4dd5c8e7 Merge 'wal: checksums' from Pere Diaz Bou
Implemeted checksums so that sqlite3 is able to read our WAL. This also
helps with future work on proper recovery of WAL.
Create some frames with CREATE TABLE and kill the process so that there
is no checkpoint.
```
Limbo v0.0.6
Enter ".help" for usage hints.
limbo> create table x(x);
limbo> [1]    15910 killed     cargo run xlimbo.db
```
Now sqlite3 is able to recover from this WAL created in limbo:
```
sqlite3 xlimbo.db
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE x (x);
```

Closes #413
2024-11-22 13:21:01 +02:00
jussisaurio
f945795ae6 consistent naming 2024-11-21 20:25:51 +02:00
jussisaurio
d8eb4be424 better, less cool names 2024-11-21 20:23:53 +02:00
jussisaurio
3002f2a552 io macro 2024-11-21 20:23:23 +02:00
jussisaurio
6dc427fc9b Merge 'impl order by desc' from lemonwx
Hi all,
I attempted to implement the `order by desc`, which mainly consists of
the following two parts:
1. for cases where the primary key can be utilized, I implemented
`push_scan_direction` to push the scan direction down to the `Scan`.
2. for cases where the primary key cannot be utilized, I re-implemented
the sorting in `Sorter::rewind` based on whether it's desc or asc.
there is also some related work to be done, such as sort using secondary
index, I will attempt to do next.
Please help review this and let me know how I can improve it, thanks in
advance

Reviewed-by: Pere Diaz Bou <pere-altea@hotmail.com>

Closes #376
2024-11-21 16:37:38 +02:00
Pere Diaz Bou
05a05cfc63 endian naming changes 2024-11-21 11:55:36 +01:00
Pere Diaz Bou
6c14c50d41 wal: checksums
Implemeted checksums so that sqlite3 is able to read our WAL. This also
helps with future work on proper recovery of WAL.

Create some frames with CREATE TABLE and kill the process so that there
is no checkpoint.
```
Limbo v0.0.6
Enter ".help" for usage hints.
limbo> create table x(x);
limbo> [1]    15910 killed     cargo run xlimbo.db
```

Now sqlite3 is able to recover from this WAL created in limbo:

```
sqlite3 xlimbo.db
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE x (x);
```
2024-11-20 20:12:32 +01:00
Pekka Enberg
de40442677 core: Remove unused import from btree.rs 2024-11-20 11:55:49 +02:00
Pere Diaz Bou
51ad827f1f fmt 2024-11-19 17:56:24 +01:00
Pere Diaz Bou
f5a1f7c800 various fixes in btree
* read_u8 now takes self.offset into account
* shift cell pointers left on balance_root with offset > 0
* fix wrong writes to page in degragment_page
2024-11-19 17:15:19 +01:00
limeng.1
a188bcd287 fix 2024-11-19 16:45:17 +08:00
limeng.1
5e9e701417 resolve comments 2024-11-19 11:39:08 +08:00
limeng.1
c51750fad5 fix lint 2024-11-19 11:39:08 +08:00
limeng.1
1cb1d16c08 resolve comments 2024-11-19 11:39:08 +08:00
limeng.1
8cca659052 impl order by desc 2024-11-19 11:39:07 +08:00
Pere Diaz Bou
db343ac5ea fix page1 balancing 2024-11-18 16:25:06 +01:00
Pere Diaz Bou
4aa5612334 fix schema query 2024-11-18 14:06:50 +01:00
Pere Diaz Bou
9f72655e30 tree_create -> btree_create 2024-11-18 10:31:47 +01:00
Pere Diaz Bou
090615b289 create btree table + parse schema 2024-11-16 16:24:28 +01:00
Pere Diaz Bou
66e20df309 fix some offset usages in btree 2024-11-16 16:23:06 +01:00
Pere Diaz Bou
6cd0f03643 core: create databases from limbo 2024-11-15 12:09:07 +01:00
Pere Diaz Bou
87c80b2a07 use core wal file for wasm 2024-11-14 17:54:20 +01:00
Pere Diaz Bou
c46bd63b5a core: drop mutex on contents
There is no need to have mutexes on buffers, we will introduce mutexes
if we want later on a file level to introduce serializability.
2024-11-13 19:13:15 +01:00
Pere Diaz Bou
2a787aedb5 core: extract page stack and document a bit 2024-11-13 17:13:30 +01:00
Pere Diaz Bou
e2276c2e9d O_DIRECT disable on WAL 2024-11-13 13:45:42 +00:00
Pere Diaz Bou
94a45eab9e checkpoint inflight 2024-11-13 12:04:54 +00:00
Pere Diaz Bou
cbfb45e55b more trace logs 2024-11-13 11:31:22 +00:00
Pere Diaz Bou
87957bf4f2 checkpoint log state 2024-11-13 11:24:07 +00:00
Pere Diaz Bou
78118bae81 fmt 2024-11-13 09:34:36 +00:00
Pere Diaz Bou
eb4105282b btree: fix btree traversal 2024-11-12 23:10:38 +00:00
Pere Diaz Bou
8d1d2d36cc core/linux: aligned wal header read 2024-11-12 17:53:59 +00:00
Pere Diaz Bou
48f0e72e14 checkpoint on drop connection 2024-11-12 17:03:30 +01:00
Pere Diaz Bou
807496a146 core: load pages if not loaded
Since pages are now tracked with a single centralized structure, it is
possible for a page to have been unloaded a be kept in memory for
metadata purposes like going back in the stack.

Using the example of going to a parent page which was unloaded for
whatever reason: in this case we need to check if it's loaded, if not,
we load it. Locked still means the same thing, loaded just means the
contents of a page are present in memory and if they are present, they
must be in cache.
2024-11-12 14:44:25 +01:00
Pere Diaz Bou
cd2b61d838 btree: cursor with lineal stack structure
Removed MemPage from the code in favor of an array to encode the stack
of the cursor. This is both simpler and better in terms of memory
access.
2024-11-12 10:21:22 +01:00
Pere Diaz Bou
ce8378793e btree: run until finished or IO + bug fixes 2024-11-11 17:03:15 +01:00
Pere Diaz Bou
74974d07e7 clippy fix 2024-11-08 11:14:29 +01:00
Pere Diaz Bou
e5836df160 write state machine
Returning on ongoing IO requires to save the current state, therefore a
state machine makes sense for write operations. This information is
encoded in WriteInfo/WriteState.
2024-11-08 11:10:49 +01:00
Pere Diaz Bou
ecfcd8835f rename get_page -> get_current_page 2024-11-07 15:37:21 +01:00
Pere Diaz Bou
0b46648c0e write wal header on init 2024-11-07 13:29:57 +01:00
Pere Diaz Bou
eb8c462c5f fix io submission on cacheflush 2024-11-06 16:25:42 +00:00
Pere Diaz Bou
8eb3c89227 wasm,sim fixes 2024-11-05 15:41:30 +01:00