Stable memory and heap on canisters

Developers Getting Started
1

I’m not quite sure I understand how much storage is available on canisters and how it’s managed.

Here is what I don’t understand:
To my understanding, I store something as a stable variable that will be stored in the canisters stable memory. If now I enquire how much memory my canister has used, has it doubled? (once in the heap and once in the stable memory)? Does this mean I only have 2GiB heap memory for upgrading?
If my canister uses Motoko’s ExperimentalStableMemory, how does the behavior change?
When does the limit of 4GiB stop when I still have 64GiB as stable memory?

Thanks in advance!

1 Like
2

Hey Roxs!

The Motoko stable variables use stable memory while upgrade to persist the data. There are several factors to consider here:

  • The bytes in stable memory originating from stable variables are appended to the end of the stable memory (which is controlled by ExperimentalStableMemory).
  • What is the footprint of your stable variables? Motoko can write out in a streaming fashion, but cannot (currently) stream in, so the bytes are copied into the 4GB heap area and deserialised from there. This can constitute that the same data is present three times before the temporary buffer is garbage-collected (in heap, in a serialised buffer in heap and in stable memory). It also implies that the stable variables (in serialised form) are optimistically limited to ca. 2GB. Other factors, such as cycle consumption, may restrict this further.
  • After the stable variables are re-loaded in the post_upgrade hook, the stable memory bytes storing the stable variables are overwritten with zeros. This is to minimise stable memory cost, as shrinking is not possible.
  • The update roundtrip of stable variable data is not preserving sharing. A blob referenced from two locations before an upgrade will be present twice in the heap after upgrade, thus occupying more space. This is a known deficiency and being worked on.
  • A canister (or the controller of it in recent Motoko versions) can query the number of bytes occupied by own stable variables.

Hope this helps, feel free to drill deeper if you are still in doubt.

2 Likes