PSA: Upcoming Wasm memory limit may break your canisters

Developers
8

The goal is to make it possible to successfully run the pre_upgrade() hook of the canister. Whether 200MB is sufficient or not depends on the canister. Usually, canisters that don’t use the stable memory as the primary storage do something like this:

let buffer = serialize(state);
write_to_stable_memory(buffer);

If the canister has reached 3.8GiB, then it is likely the serialized state is larger than 200MB and allocation of buffer would fail.

There is a tradeoff in choosing the default value:

  • if it is too low, then many canisters will hit the limit even if they wouldn’t reach 4GiB.
  • if it is too high, then the limit doesn’t help to prevent the disaster for canisters that would reach 4GiB.

Another thing to consider:

  • if the default is set to 3GiB, then developer who would prefer 3.8GiB can always increase the limit to 3.8GiB when they see the error and thus get the same behavior as if the default was set to 3.8GiB.
  • if the default is set to 3.8GiB, then developer who would prefer 3GiB cannot do anything when they see the error.

Also, I remember proposing the canister_on_low_heap_memory hook for canisters in Canister Lifecycle Hooks
Would a higher “safe” limit plus adding a heap memory lifecycle hook suffice?

These two feature solve different problems. The goal of wasm_memory_limit is to make developers aware of out-of-memory problem. If the developer already knows about that problem and even implements the low-memory hook, then they can set wasm_memory_limit to any value they see fit (like they would set the freezing threshold to protect against going out of cycles).

If the community feels strongly about setting the default value higher, then we could go with 3.5GiB. I think going higher than then would defeat the purpose of wasm_memory_limit.

This means canister heaps will no longer be limited by 32-bit wasm and can store as much data as the subnet allows them to support.

The existing 32-bit canisters will still be limited by 32-bit memory. In order to transition from 32-bit to 64-bit, the developer would need to upgrade the canister and migrate their code and data (which is not trivial). I expect many production canisters to remain 32-bit even if 64-bit is available.

1 Like