RxMoDb - Database v.0.1.0

Mops package: https://mops.one/rxmodb
Old thread: https://forum.dfinity.org/t/motoko-database/20759

Setup

I’ll keep things simple here and not get into details explaining how you set up your documents.
Example setup https://github.com/infu/RxMoDb/blob/main/test/hero.mo

import Hero "./hero";

stable let hero_store = Hero.init();
let hero = Hero.use(hero_store);

Usage

Insert or Update (if using PK)

hero.db.insert({
        id=123;
        updatedAt= 1;
        score= 3;
        name= "B";
        level= 1;
        skills= ["Fireball","Blizzard","Thunder"];
        deleted= true
    });

Delete by PK

hero.pk.delete(123);

Index find

hero.score.find(0, ^0, #fwd, 10);
// [{id=123; name="B" ...}, ...]

PK get

hero.pk.get(123);
// ?{id=123; name="B" ...}

Under the hood, it has a plugin architecture. The only data structure that is fixed is the main storage using Vector. This way we can have fast idx:Nat → Document access. Indexes have customizable key type and point to that idx:Nat.

Utilizing the plugins a developer can decide what data structures to use. Currently, there are only PrimaryKey:Btree and Index:Btree plugins. (Anyone can make them).

Going to get more tests done. So far I was able to insert 10mil documents with 2 indexes.

// Motoko compiler 0.9.3 --incremental-gc
// 10mil documents (took 66m 44.7s to insert in local replica)
// 2 indexes. Nat64 -> Nat
// upgrades ok

  "documents": "10 000 000",
  "memory_size": "3 758 096 384",
  "max_live_size": "0",
  "heap_size": "3 644 987 000",
  "total_allocation": "3 644 987 304",
  "reclaimed": "0"

Thanks to @icme for BTree, @timo for Vector, @claudio for helping figure things out @skilesare for helping with design @luc-blaeser for Incremental Gc @ZenVoich for mops and the test framework

17 Likes

@infu moves motoko development forward by years every time he gets a hunch and spends a weekend on something. Amazing work!

Special thinks to @timo for the amazing optimized libraries that have been coming up lately. With a bit more of this we’re going to have some amazing tools on our hands.

11 Likes

@infu - would love to chat with you some more about how this library works. What’s the best way to get in contact? Shall I DM you here or is there another way you’d prefer? :slight_smile:

1 Like