limbo/PERF.md

Performance Testing

Mobibench

1. Clone the source repository of Mobibench fork for Limbo:

git clone git@github.com:penberg/Mobibench.git

2. Change LIBS in shell/Makefile to point to your Limbo source repository.

3. Build Mobibench:

cd shell && make

4. Run Mobibench:

./mobibench -p <benchmark-directory> -n 1000 -d 0 -j 4

Clickbench

We have a modified version of the Clickbench benchmark script that can be run with:

make clickbench

This will build Limbo in release mode, create a database, and run the benchmarks with a small subset of the Clickbench dataset. It will run the queries for both Limbo and SQLite, and print the results.

Comparing VFS's/IO Back-ends (io_uring | syscall)

make bench-vfs SQL="select * from users;" N=500

The naive script will build and run limbo in release mode and execute the given SQL (against a copy of the testing/testing.db file) N times with each vfs. This is not meant to be a definitive or thorough performance benchmark but serves to compare the two.

TPC-H

1. Clone the Taratool TPC-H benchmarking tool:

git clone git@github.com:tarantool/tpch.git

2. Patch the benchmark runner script:

diff --git a/bench_queries.sh b/bench_queries.sh
index 6b894f9..c808e9a 100755
--- a/bench_queries.sh
+++ b/bench_queries.sh
@@ -4,7 +4,7 @@ function check_q {
        local query=queries/$*.sql
        (
                echo $query
-               time ( sqlite3 TPC-H.db < $query  > /dev/null )
+               time ( ../../limbo/target/release/limbo -m list TPC-H.db < $query  > /dev/null )
        )
 }