Memory Allocation Explained

Developers
1

Hi everyone! I’m Adam and I recently started at DFINITY on the execution team. We thought there might be interest in a post describing how memory allocation works so here’s an overview. Hope it’s helpful!

Canister Memory

One advantage of the Internet Computer’s canister smart contracts over other smart contracts is that canisters can cheaply store data in addition to code. Here we’ll discuss how developers can reserve storage space for their canisters and how canisters are charged for the storage they use.

IC Storage Basics

IC canisters can store data in multiple ways:

  1. By allocating memory for the data on the wasm heap (as would happen using standard Rust or Motoko data structures).
  2. By storing the data in stable memory (via explicit calls to the system API to write to stable memory).
  3. By storing static data in the wasm binary itself (through, e.g. static objects or globals).

Canisters are periodically charged cycles for any memory they are using (including the memory used to store the wasm binary itself) and all kinds of memory usage are charged at the same rate.

Allocation

Developers have the option of choosing whether to reserve memory for their canisters or have them run on a best-effort basis.

  1. Reserved: If memory is reserved for a canister then the subnet will guarantee that the reserved amount of memory is free for the canister to use. However the canister will be charged for the full reserved amount, even if only a portion of it is used and the canister will not be able to use more than the reserved amount.
  2. Best-effort: If the canister is running on a best-effort basis then it is only charged for the actual memory in use, but there is the risk that the canister will not be able to run if it tries to expand its memory usage when there is not enough free space for it on the subnet.

The choice of whether to reserve memory for a canister or not and how much memory to reserve should weigh the cost of reserving additional memory against the risk of the canister being unable to run.

Viewing and Updating Settings

Developers can view the memory allocation settings for canisters with the dfx canister status command:

$ dfx canister status hello

Canister status call result for hello.

Status: Running

Controller: rwlgt-iiaaa-aaaaa-aaaaa-cai

Memory allocation: 0

Compute allocation: 0

Freezing threshold: 2_592_000

Memory Size: Nat(360284)

Balance: 4_000_000_000_000 Cycles

Module hash: 0xb500708ce662300606d92bbab682ee373968233179a2b72d43a350107396b13b

Note that a memory allocation of 0 indicates that the canister is running on a best-effort basis. The memory currently in use by the canister can also be seen.

When creating a canister, the memory allocation can be set using the --memory-allocation option of dfx canister create. The memory allocation for an existing canister can be changed using the dfx canister update-settings command with the --memory-allocation option.

Costs

The current cost for memory usage on the IC is 127,000 cycles per GiB per second. This comes out to less than $6 per GiB per year. Full details of IC costs can be found in the docs and they may be changed in the future by an NNS proposal and vote. Note that there are also costs associated with ingress messages, so in addition to the storage costs there would be some additional charge for uploading data to a canister.

Limits

Currently canisters are limited to 12 GiB total memory usage. In addition, the wasm heap is limited to 4 GiB and stable memory is limited to 8 GiB.

20 Likes
3

A relation work about ic storage explaination(in chinese):

Welcome to contribute

2 Likes
4

Thanks for the overview.

I’m curious about the implementation of stable memory. The WASM heap implementation is provided by wasmtime, so that’s less of a mystery.

But I’ve heard stable memory is shared by all canisters in a subnet. Is stable memory ultimately mapped to disk on a node (or all nodes, to be precise), or does it exist in RAM only? How does IC ensure that two different canisters on the same subnet don’t write to the same stable memory address, assuming that stable memory is indeed a shared resource?

Thanks!

5

The stable memory of a canister is isolated from all other canisters and really a stop gap until the VM supports the Wasm multi-memory proposal.

At least, that was the original intention.

Stable memory is intended as a mechanism for preserving data across upgrades (which the wasm memory will not survive or can not meaningfully survive since the compiler may not guarantee compatible memory organization/use across upgrades.)

1 Like
6

Interesting, I didn’t know it was isolated. I distinctly remember reading that there a hard 300 GB cap on all stable memory in a subnet. Maybe I read wrong.

Dumb question: is stable memory actually persisted to a node’s SSD or is it all in RAM (assuming 1/3+ of nodes don’t go down, or something like that)?

7

There may well be a cap on total stable memory, but that doesn’t imply that canisters can read/write each other’s stable memory. That would also be a pretty strong departure from the actor model of shared nothing concurrency.

I don’t know more about the implementation, but someone else will hopefully answer.

2 Likes
8

@jzxchiang Yes there is a cap on the total memory a subnet can use and stable memory contributes to that, but each canister only has access to its own stable memory.

The state of a canister’s stable memory is persisted to disk when we take checkpoints, and in between checkpoints it is kept in memory.

@ulan had a community conversation which describes this in detail: Community Conversations | Optimizing The Memory System - YouTube. That’s specifically covering the wasm heap, but stable memory is stored in essentially the same way (the difference is just in how the canister accesses it).

3 Likes