Increased Canister Smart Contract Memory

Currently, the wasm heap is capped at 4GiB so we cannot expand it further. Once wasmtime stabilises support for wasm64, then we could allow heaps to grow beyond 4GiB as well. However, then we also have to worry about how does a canister with a ton of heap upgrades itself (because we currently wipe the heap on upgrades). So supporting wasm64 will open other issues that also need to be addressed.

2 Likes

Extending 64bit stable memory is just first step in the process. After this more work would be needed. At a high level, one idea to use the expanded stable memory with just 4GiB of wasm heap is to do something like the following:

  • When a canister starts executing a message, it will first have to identify where in the stable memory the relevant block of data resides.
  • It will then copy over the relevant block of data from stable memory to heap.
  • It can now operate on the data on the heap.
  • Once done, it can now copy the updated data back to the stable memory.

Identifying where in stable memory the relevant data lies will involve building some sort of index in the wasm heap or a HashMap structure that can operate over the stable memory API. This part is not being actively worked on currently by the team. We expect that this may not necessarily need system support and could be built purely in the application space.

5 Likes

Note that even already now, the Internet Identity canister does precisely that: Keep all relevant data in stable memory, and use the heap only as scratch space. Makes upgrades simpler.

I guess expanding stable memory beyond 4GB means that the Internet Identity service does not need to scale out for space reasons any time soon. (Maybe for throughput.). @kritzcreek will be happy to hear that :slight_smile:

5 Likes

Indeed, that is where we got the inspiration.

I just updated the Summary at the top of this thread with more info, including relevant folks.

3 Likes

Thanks, is there any documents about stable memory and heap of canister? what do the devs need to do to use the different memory style?

Maybe vars used stable keyword will be storaged in stable memory.

2 Likes

The way the two memories are used differs depending on whether you are writing in Rust or Motoko.

CC: @claudio for motoko documentation.

In Rust, you can use the heap as a normal wasm heap and the stable memory via the API as defined in cdk-rs/stable.rs at main · dfinity/cdk-rs · GitHub. Note that this will be extended when the 64bit stable memory extension lands.

1 Like

Motoko indeed uses stable memory as the backing store for stable variables.

Before an upgrade, the stable variables are serialized from wasm memory to stable memory. After an upgrade, the stable variables are deserialized from stable memory into wasm memory. Motoko doesn’t read or write to stable memory during execution outside of upgrades.

This solution, though simple, is not ideal as the (de)serialization steps can both run out of cycles for large data structures, preventing upgrade when they do.

We are currently exploring support and an API for (almost) raw access to (32-bit) stable memory that we hope to extend to 64-bit stable memory in the future. The idea to allow both high-level Motoko stable variables and (abstract) stable memory to co-exist in a single canister so that canisters that prefer to maintain their data in stable memory throughout execution (to avoid the problem with exhausting cycles on upgrade) can does so if they wish, at the conceptual cost of using a much lower-level API.

4 Likes

Hi all, a brief update on this feature.

We have had some discussions about what is the best way to introduce new APIs on the IC. We want to expose new APIs to let developers experiment with them and to solicit feedback. It is possible that based on the feedback we get, it turns out that the API needs some adjustments. And if developers have deployed canisters using the API, then making adjustments will be difficult. So we are planning on coming up with a process that allows us introduce APIs in an experimental phase. I’ll make a separate post about the process after some of the ongoing discussions have concluded to solicit feedback on this process.

I imagine that the community will not want to block the release of this feature / API till we have figured out what the best way to do experimental APIs is yet. So, the current plan is that when we release this API, we will document it as being in experimental phase and ask developers not to use it on any critical canisters yet. And if based feedback, some adjustments to the API are needed, then the existing canisters using it will be impacted. We will of course discuss and announce impending changes so that the everybody has an opportunity to upgrade their canisters. We will also have to figure out what to do if someone removes the controller from their canister and then they cannot be upgraded (hopefully no one does that :crossed_fingers:)

I apologise in advance for any difficulties this might cause you. And I hope that you can appreciate that introducing a process to iterate on APIs will allow us all to build a better IC without a lot of cruft. Thank you for understanding.

7 Likes

DFINITY engineer here. In the initial roll out we will limit the stable memory size to 8GB. We will increase it in future release after gathering feedback.

4 Likes

So should increasing the Wasm heap be another roadmap proposal entirely? Increasing stable memory is a good step, I hope that increasing the heap won’t be thought of as less useful or put off for too long. Copying over chunks of stable memory into the heap is still going to be a complication for library authors and developers to deal with, and having a 300 GB heap would be ideal.

2 Likes

@lastmjs : absolutely! Please do not think that we will stop after shipping this feature. This is just a small step in helping developers access tons of storage in an ergonomic way. We absolutely realise that asking developers to copy chunks of memory back and forth from stable memory to heap is complicated, froth with footguns, and also inefficient! We are actively discussing designs in this space and I will keep you guys posted on developments here.

6 Likes

Sounds just swell, thanks!

1 Like

Interesting idea to let canisters directly access stable memory during runtime outside of upgrades.

If they can do that, then what’s the need for a wasm heap? Can’t canisters use stable memory for all computation in that case, including scratch space? Is it slower to access the wasm heap versus stable memory via this hypothetical API? Does it cost more to store data in stable memory?

Thanks!

2 Likes

That was the vision all along: that wasm code has multiple memories they can use, with different lifetimes, and all accessed via Wasm memory instructions, so fast.

The stable memory API through expensive bulk system API function calls was just a make shift. Unfortunately the necessary webassembly extension (multiple memories) didn’t make it in time into the Wasm tools we use, so we are stuck with this transitional API for now. But in principle you are absolutely right.

3 Likes

Hm, when you mean “expensive bulk system API function calls”, do you mean that storing data in stable memory is more expensive than in non-stable memory? (If not because of any differences in storage cycle cost but instead due to the additional computation required to transfer data in and out of stable memory in pre- and post-upgrade functions?)

1 Like

In terms of actual running time, normal memory is accessed using Wasm memory instructions which are compiled to memory instructions, so that’s native speed fast. Accessing stable memory with the current API requires function calls into the host, bounds checking, explicit copying etc., so definitely more expensive. Whether that cost is currently reflected also in cycle costs (which you probably care more about than running time) I don’t know, but we can assume that cycle costs will be refined to match real costs eventually.

In the original vision, building on Wasm multi memory support, stable memory would be accessed the same as the main memory, and would likely have the same access cost. One wouldn’t speak of “main” memory at that point any more, as there can be any number of memories, of varying lifetimes (per message, maybe per call, per module version, permanent).

It might help if the people at DFINITY could share the “Refining persistence” document by Andreas Rossberg, which explains this better.

8 Likes

I would love access to said document

It seems the code for 64 bit stable memory is live! The IC runs this revision:

~/dfinity/ic $ curl https://ic0.app/api/v2/status|cbor2yaml.rb|grep impl_version
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   331  100   331    0     0    565      0 --:--:-- --:--:-- --:--:--   564
  impl_version: 27e1eadbcbe90abfe56d9c8dfd39e1a78e52c624

which, according to the code dump repo history, is dumped as 89446f5

~/dfinity/ic $ git log --oneline
35dd8f9 Update from revision 8a5b9a2e1468dfb286c77084a9b3597b9e3993b5
e362530 Update from revision d8dabe1cb0bb1e60a11e5bb8033d1615fefb252f
89446f5 Update from revision 27e1eadbcbe90abfe56d9c8dfd39e1a78e52c624

and indeed it’s there ic/system_api.rs at 89446f5a04f053040b4863eab5458446d925ed0e · dfinity/ic · GitHub.

So early adopters can probably start playing around with it, even if it didn’t make it to The Internet Computer Interface Specification :: Internet Computer or GitHub - dfinity/ic-hs: A haskell toolbox for the Internet Computer yet.

It would be helpful to the outsiders if the code dumps would include a changelog (maybe auto-generated from the internal repo?), to easier see what changes to look out for.

5 Likes

@nomeata : no the code is not actually live yet. These are stub implementations. See ic/system_state_accessor_direct.rs at 89446f5a04f053040b4863eab5458446d925ed0e · dfinity/ic · GitHub.

Some more of the implementation will get pushed out in the next upgrade. However, we are also planning on doing an NNS proposal where we will ask the community to vote on whether or not this feature should be enabled or not. Only if the vote passes, then we will push the final changes that will enable the feature.

5 Likes