Motoko canister memory size increased after upgrade

A few people reported that they actually observed heap usage in Rust canisters doubling after the first upgrade.
I think it depends on the serialization format and the way developers use stable memory.

When we decode stable memory after an upgrade, we need to first copy a portion of the data into a fresh memory buffer, and then interleave parsing and buffer filling. We cannot free the buffer until the parsing is done, so the expected memory consumption after the upgrade is size(buffer) + size(data structures).

If the buffer management is naive and the serialized data is copied into the main memory in full before the parsing begins, our simple formula predicts that the heap usage roughly doubles.

A quick look at the current Rust CDK implementation reveals that the buffer management used by stable_restore is indeed naive: storage.rs - source

1 Like