Commit graph

93 commits

Author SHA1 Message Date
Pekka Enberg
90c1e3fc06 Switch Connection to use Arc instead of Rc
Connection needs to be Arc so that bindings can wrap it with `Mutex` for
multi-threading.
2025-06-16 10:43:19 +03:00
Pekka Enberg
05df548b10 core/io: Add wait_for_completion() to I/O dispatcher 2025-05-27 13:47:40 +03:00
Pekka Enberg
eca9a5b703 core/io: Switch to Arc<Completion> 2025-05-27 11:28:49 +03:00
Owen Delahoy
c31f2b10b7 fix: implement generate_random_number for wasm binding 2025-05-22 20:12:48 +09:00
Pekka Enberg
e3f71259d8 Rename OwnedValue -> Value
We have not had enough merge conflicts for a while so let's do a
tree-wide rename.
2025-05-15 09:59:46 +03:00
meteorgan
6a860e75b8 fix cargo clippy 2025-04-25 22:06:44 +08:00
meteorgan
f464d15f8b refactor database open_file and open 2025-04-25 21:45:18 +08:00
Diego Reis
d9bf383507 core/io: Untie MemoryIO's lifetime of the IO layer 2025-04-13 11:10:06 -03:00
Diego Reis
79f8b83cbe Fix dumb clippy errors 2025-04-13 11:10:06 -03:00
Avinash Sajjanshetty
3543e83b91 Impl Clock trait in bindings 2025-04-06 23:34:15 +05:30
Pere Diaz Bou
9291f60722 Introduce Register struct
OwnedValue has become a powerhouse of madness, mainly because I decided
to do it like that when I first introduced AggContext. I decided it was
enough and I introduced a `Register` struct that contains `OwnedValue`,
`Record` and `Aggregation`, this way we don't use `OwnedValue` for
everything make everyone's life harder.

This is the next step towards making ImmutableRecords the default
because I want to remove unnecessary allocations. Right now we clone
OwnedValues when we generate a record more than needed.
2025-03-27 17:53:02 +01:00
Pekka Enberg
df6af6ed79 core: Rename FileStorage to DatabaseFile 2025-03-25 11:15:16 +02:00
Pere Diaz Bou
00ab3d1c0c Fix ordering and implement Deref 2025-03-17 10:22:42 +01:00
Pere Diaz Bou
20f5ade95e Experiment with a custom Lock for database header 2025-03-17 10:21:34 +01:00
Pekka Enberg
96175cccf7 cli: Add --experimental-mvcc option to enable MVCC 2025-03-06 10:16:42 +02:00
Pere Diaz Bou
e4a8ee5402 move load extensions to Connection
Extensions are loaded per connection and not per database as per SQLite
behaviour. This also helps with removing locks.
2025-03-05 14:07:48 +01:00
Pere Diaz Bou
8daf7666d1 Make database Sync + Send 2025-03-05 14:07:48 +01:00
Pekka Enberg
936ae307b7 core: Kill value type
We currently have two value types, `Value` and `OwnedValue`. The
original thinking was that `Value` is external type and `OwnedValue` is
internal type. However, this just results in unnecessary transformation
between the types as data crosses the Limbo library boundary.

Let's just follow SQLite here and consolidate on a single value type
(where `sqlite3_value` is just an alias for the internal `Mem` type).
The way this will eventually work is that we can have bunch of
pre-allocated `OwnedValue` objects in `ProgramState` and basically
return a reference to them all the way to the application itself, which
extracts the actual value.
2025-02-26 10:57:45 +02:00
Pekka Enberg
0638550be7 Merge 'Remove unnecessary reference counting from completion I/O callbacks' from Preston Thorpe
I am on a bit of a mission to revisit a lot of the ref counting, this
was an easy first win.
It seems to be a linear path of function calls or hashmaps which can own
the completions directly, no cloning needed.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #912
2025-02-10 12:11:30 +02:00
Tiago Ribeiro
4b67e67b31
Update wasm bindings to use the new retrieval methods to access Record values. 2025-02-10 00:27:51 -07:00
PThorpe92
75898027a0
Remove unnecessary reference counting from completion io callbacks 2025-02-08 08:20:19 -05:00
Pekka Enberg
c210821100 core: Move result row to ProgramState
Move result row to `ProgramState` to mimic what SQLite does where `Vdbe`
struct has a `pResultRow` member. This makes it easier to deal with result
lifetime, but more importantly, eventually lazily parse values at the edges of
the API.
2025-02-06 11:52:26 +02:00
Levy A.
9df0b01689 refactor: lower ownership requirement 2025-02-03 16:52:42 -03:00
sonhmai
75f0cf9e20 chore: make clippy happy 2025-01-20 13:29:23 +07:00
Elijah Morgan
f20a2d1334 Reorg to single package
Handle commonjs and esm module entry points

tests works
2025-01-16 21:29:19 -05:00
Elijah Morgan
02e4104932 reorg package structure
src moved under web/ to make it cleaner
build does less moving of files, mostly just moves the wasm-pack
into dist for node and web
2025-01-16 21:29:19 -05:00
Pekka Enberg
a369329988 Merge 'Add OPFS support to Wasm bindings' from Elijah Morgan
Lots of cleanup still left to do. Draft PR for adding support for OPFS
for WASM build (add support for limbo in browser).
Overall explanation of the architecture: this follows the sqlite wasm
architecture for OPFS.
main <> (limbo-worker.js) limbo (VFS - opfs.js) <> opfs-sync-proxy
The main thread loads limbo-worker.js which bootstraps the opfs.js and
opfs-sync-proxy.js and then launches the limbo-wasm.js.
At that point it can be used with worker.postmessage and
worker.onmessage interactions from the main thread.
The VFS provided by opfs.js provides a sync API by offloading async
operations to opfs-sync-proxy.js. This is done through SharedArrayBuffer
and Atomic.wait() to make the actual async operations appear synchronous
for limbo.
resolves #531

Closes #594
2025-01-07 10:36:18 +02:00
Pekka Enberg
daee7f8458 s/RowResult/StepResult/ 2025-01-05 20:24:26 +02:00
Pekka Enberg
5681603750 Merge 'Make iterate() lazily evaluated on wasm' from Diego Reis
#514
Introduces a new feature for lazy evaluation in the
`Statement.raw().iterate()` method and includes related changes in both
the test and implementation files. The most important changes include
adding a test case for lazy evaluation, creating a `RowIterator` struct,
and modifying the `iterate` method to use this new struct.
Everything seems to works fine, but suggestions on code improvement and
test use cases are welcoming.

Closes #527
2025-01-05 20:23:06 +02:00
Elijah Morgan
d51ca2150c Cleanup logging, move html files
General cleanup of cruft, deleted dead code, moved html files to clean
up dir.
2025-01-03 14:38:34 -05:00
Elijah Morgan
4bda7803c3 Add build for nodejs or web
Change how VFS gets loaded based on feature flags
2025-01-01 16:54:31 -05:00
Elijah Morgan
058ca89561 feat add basic opfs support and tests 2025-01-01 10:31:26 -05:00
PThorpe92
f6cd707544
Add clippy CI, fix or ignore warnings where appropriate 2024-12-29 10:25:41 -05:00
adamnemecek
97647ff056 Clean up code to use Self
Closes #556
2024-12-29 10:07:38 +02:00
Pekka Enberg
f2ecebc357 Rename RowResult to StepResult
The name "row result" is confusing because it really *is* a result from
a step() call. The only difference is how a row is represented as we
return from VDBE or from a statement.

Therefore, rename RowResult to StepResult.
2024-12-27 10:20:41 +02:00
Diego Reis
67c6409412 Add handling to busy row on RowIterator::next() 2024-12-26 15:40:04 -03:00
Diego Reis
d7be6f91cd Fix formating and pass on tests 2024-12-26 15:40:04 -03:00
Diego Reis
979115ee59 Remove unused variable on RowIterator 2024-12-26 15:40:04 -03:00
Diego Reis
77c2b6f1a0 Make iterate() lazily evaluated on wasm 2024-12-26 15:40:04 -03:00
Pere Diaz Bou
aed14117c9 core: transaction support 2024-12-24 18:04:30 +01:00
Ziyak Jehangir
99d1b0e5a3
cleanup: replace &(*x) with x.as_ref() for smart pointer derefs 2024-12-19 19:36:04 +05:30
Pekka Enberg
e93ac38e55 Add statement interruption support
This adds an interrupt() method to Statement that allows apps to
interrupt a running statement. Please note that this is different from
`sqlite3_interrupt()` which interrupts all ongoing operations in a
database. Although we want to support that too, per statement interrupt
is much more useful to apps.
2024-12-19 12:30:32 +02:00
Pere Diaz Bou
39a75147d4 Page cache by page_number and frame_number
Since page cache is now shared by default, we need to cache pages by
page number and something else. I chose to go with max_frame of
connection, because this connection will have a max_frame set until from
the start of a transaction until the end of it.

With key pairs of (pgno, max_frame) we make sure each connection is
caching based on the snapshot it is at as two different connections
might have the same pageno being using but a different frame. If both
have same max_frame then they will share same page.
2024-12-13 21:57:27 +01:00
Pere Diaz Bou
3fda2d09b9 Extract multi threaded part from WalFile to WalFileShared
Since we expect to ensure thread safety between multiple threads in the
future, we extract what is important to be shared between multiple
connections with regards to WAL.

This is WIP so I just put whatever feels like important behind a RwLock
but expect this to change to Atomics in the future as needed. Maybe even
these locks might disappear because they will be better served with
transaction locks.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
a4297702bd extract page cache to be multi threaded 2024-12-13 13:09:13 +01:00
Pekka Enberg
36f9565910 Merge 'feat(wasm): add get and iterate func' from Jean Arhancet
Add `get` and `iterate` functions to the wasm module

Closes #421
2024-12-09 17:28:43 +02:00
JeanArhancet
5693cd1ae0 feat(wasm): add get and iterate func 2024-11-29 21:48:20 +01:00
Lauri Virtanen
a7100d8e9b
Autofix clippy issues with cargo fix --clippy 2024-11-24 20:24:47 +02:00
Pekka Enberg
bfce49c7eb bindings/wasm: Fix conversion to JsValue 2024-11-20 11:54:34 +02:00
Pekka Enberg
b65d0a72ba bindings/wasm: Implement Statement.raw() 2024-11-20 11:47:28 +02:00