Motoko stable memory in 2022

Quick question. Let’s say I have a canister with 2GB of user data. I change the data type and now have to copy all the data in postupgrade in order to upgrade the canister without wiping the data. Would I be able to upgrade? What if I have 3GB of user data?

1 Like

As someone who is writing documentation for Motoko, I would love to know how to present the use of stable vars and stable memory to programmers.

Right now it isn’t clear what best practices should be.

One idea I had while tinkering with a stable memory lib for my project, is to implement some kind of ‘Memory Boot Record’ in stable memory and initiate code from there.

I’m not sure

2 Likes

It is a great question. The “stable” keyword is currently confusing as it only has tangential relevance to “stable memory”. My understanding is that @matthewhammer is working on generalizing “stable memory” in a similar manner to “stable”, including garbage collection. I’m not sure how this will affect the vocabulary, but it is unlikely we go backward. “stable” should probably have been “managed” or some something else. Perhaps we get a breaking change at some point.

4 Likes

The MVP will only be a small generalization over what is now called ExperimentalStableMemory, giving independent, dynamically allocatable Regions (there is now only one global stable memory region that can be grown, like a pre-allocated, shared Region in the new terminology).

In particular, the new API will still be:

  • low level, with operations like grow and loadBlob and storeBlob taking offsets into a Region’s big array of bytes.
  • compatible with stable vars, another more “high level” way to represent stable data that is orthogonal, but uses the same IC-level machinery under the hood (namely, stable memory space).

Eventually, we want the Region type to be collected by GC, but that’s not part of the MVP.

In terms of the larger story about stable data in Motoko, there’s expectations that eventually, the IC execution layer will generalize stable memory so that it can be as inexpensive and plentiful as ordinary canister memory is today.

If and when that shift happens, lots of things will likely change in the Motoko GC story, and the Rust stable memory story too. I don’t have a timeline for that shift, but it seems inevitable given enough time for things to evolve, as it seems like the most faithful way to realize the “orthogonal persistence” promise. cc @claudio @luc-blaeser

6 Likes

In addition to the Motoko doc on stable variables and upgrades there is this example you might find useful:

2 Likes

Is the following a good dfx.json example on how to update stable memory?

{
  "version": 1,
  "dfx": "0.13.1",
  "canisters": {
    "hello": {
      "type": "motoko",
      "main": "src/hello/main.mo",
      "args": "-v --max-stable-pages 786432"
    }
  },
  "defaults": {
    "build": {
      "packtool": "",
      "args": ""
    }
  },
  "networks": {
    "local": {
      "bind": "127.0.0.1:8000",
      "type": "ephemeral"
    }
  }
}

Then we just need to use “mo:base/ExperimentalStableMemory” to store correct?

1 Like