Motoko stable memory in 2022

(@Manu asked someone to follow up on this)

The library is marked experimental because its rather easy to shoot yourself in the foot. In particular, without coordination, separate libraries that import ExperimentalStableMemory can easily wind up trashing each others’ memory.

That said, I you can build on this if you are careful. The library will at most be replaced by something roughly similar but with better isolation guarantees.

Regarding the recent extension of stable memory limits from 8GB to 32GB:

The library already use 64-bit addresses so supports the 32GB stable memory limit out of the box.
However, in order to do that, you do need to tell the compiler how many stable memory pages (at most) to dedicate to ExperimentalStableMemory.mo using the --max-stable-pages <n> compiler flag. (With dfx, you can set this with the optional “args” string property of a motoko canister in the dfx.json file - this contains additional command line arguments to pass to the moc compiler during a build.)

By default, the compiler allows at most 65536 (64K) pages (4GB), reserving the remainder (previously 4GB, but now 28GB) for Motoko stable variable storage.

From https://github.com/dfinity/motoko-base/blob/master/src/ExperimentalStableMemory.mo:

Memory is allocated, using* grow(pages) *, sequentially and on demand, in units of 64KiB pages, starting with 0 allocated pages. New pages are zero initialized. Growth is capped by a soft limit on page count controlled by compile-time flag --max-stable-pages <n> (the default is 65536, or 4GiB).

NB: The IC’s actual stable memory size (ic0.stable_size) may exceed the page size reported by Motoko function* size() . This (and the cap on growth) are to accommodate Motoko’s stable variables. Applications that plan to use Motoko stable variables sparingly or not at all can increase --max-stable-pages as desired, approaching the IC maximum (currently 8GiB). All applications should reserve at least one page for stable variable data, even when no stable variables are used.

A tutorial sample using ExperimentalStableMemory is here (thought it doesn’t mention the compiler flag):

https://internetcomputer.org/docs/current/developer-docs/build/cdks/motoko-dfinity/stablememory/

(An example on passing (other) command line args to moc via dfx.json is here Stable-types build error moving to 0.9.3 and later versions of the SDK - #35 by claudio)

9 Likes