Wasm_memory_persistence backwards compatibility

Updating the IC types in TypeScript leads to having to resolve a breaking change given that wasm_memory_persistence is a mandatory optional types.

I found an entry about it in this CHANGELOG and a brief explanation in the documentation but, nowhere it is explained how to migrate - i.e. what’s the backwards compatibility value to use?

// Mode used to be typed as
mode: {reinstall: null} | {upgrade: [] | [{skip_pre_upgrade: [] | [boolean]}]} | {install: null};

// New type
mode: {reinstall: null}
  | {
      upgrade:
        | []
        | [
            {
              wasm_memory_persistence: [] | [{keep: null} | {replace: null}];
              skip_pre_upgrade: [] | [boolean];
            }
          ];
    }
  | {install: null};

// So given the following
mode: {upgrade: [{skip_pre_upgrade: [false]}]}

// What should be the backwards compatible parameter?
mode: {upgrade: [{skip_pre_upgrade: [false], wasm_memory_persistence: [????????????]}]}

Side note: wasm_memory_persistence relates to what is commonly referred as “heap” memory?

I’d recommend you to use wasm_memory_persistence: [] assuming this corresponds to wasm_memory_persistence being set to null.

wasm_memory_persistence relates to what is commonly referred as “heap” memory

indeed

Thanks. So [] equals keep or replace? I assumed keep but, never too sure.

[] corresponds to replace which is the behavior you’d get before the persistence feature was introduced (the heap memory would be reset as the canister would by default not be able to make sense of the old heap memory, e.g., created by code compiled by a different version of the compiler)

I’m a bit confused now. If I understand correctly, the default behavior remains the same, the heap memory is wiped during a canister upgrade. This corresponds to the [replace] or [] option.

If the above is correct, what’s the use case for the keep option? Is it the opposite, where the heap memory is preserved? Is there an explanation somewhere that details when to use each behavior (I mean in practice, in real life)? Also, is there a sample repository available assuming pre and post upgrade should be coded differently?

If the above is correct, what’s the use case for the keep option? Is it the opposite, where the heap memory is preserved?

The main use case is Motoko orthogonal persistence which wants to reuse the heap memory of the old canister after a canister upgrade and thus the heap memory must be preserved across the upgrade.

Is there an explanation somewhere that details when to use each behavior (I mean in practice, in real life)?

You should use this for Motoko canisters which expose a custom section with the name enhanced-orthogonal-persistence. Note that the execution environment checks for that, too.

Also, is there a sample repository available assuming pre and post upgrade should be coded differently?

It isn’t just about a pre- and post-upgrade. It requires careful support from the language runtime which is what the Motoko orthogonal persistence feature is about.

Thanks for the details. I think I’ve got it now. It would be helpful to include this type of information in the CHANGELOG.

As for me, to maintain backwards compatibility and answer the main question of the thread, I’ll migrate to wasm_memory_persistence: [{replace: null}].

Could you please check with the Motoko team what implications this would have for Motoko orthogonal persistence. @luc-blaeser

Please feel free the open a PR with your suggested changes.

A suggestion for the future CHANGELOGs.

Thanks for reaching out, @peterparker

wasm_memory_persistence is indeed a Motoko-specific feature, that is used by enhanced orthogonal persistence. More information can be found here: Enhanced Orthogonal Persistence (Complete Integration) by luc-blaeser · Pull Request #4488 · dfinity/motoko · GitHub
We plan to beta-release it soon, but still need to wait for some support of other components, e.g. dfx.

I strongly recommend to use wasm_memory_persistence: [] which is the default behavior. As mentioned, this is dropping Wasm main memory (including the heap). This setting makes sense for all non-Motoko canisters.
For Motoko, the IC checks whether the keep is accidentally missing with that configuration, so reminds you to use keep when needed.

However, if you specify wasm_memory_persistence: [{replace: null}] you will lose the persistent data of future Motoko canisters, as it forces to erase main memory which is the new place where Motoko keeps the persistent data.

I am happy to provide more information or answer questions.

Thanks for the detailed explanation, Luc, it’s interesting and appreciated!

I don’t use Motoko. Relying on the default value in Juno’s tooling feels less future-proof than ensuring the expected behavior is met by being verbose. That’s why I mentionned I’ll use the the explicit argument.