Increased Canister Smart Contract Memory

Interesting idea to let canisters directly access stable memory during runtime outside of upgrades.

If they can do that, then what’s the need for a wasm heap? Can’t canisters use stable memory for all computation in that case, including scratch space? Is it slower to access the wasm heap versus stable memory via this hypothetical API? Does it cost more to store data in stable memory?

Thanks!

2 Likes

That was the vision all along: that wasm code has multiple memories they can use, with different lifetimes, and all accessed via Wasm memory instructions, so fast.

The stable memory API through expensive bulk system API function calls was just a make shift. Unfortunately the necessary webassembly extension (multiple memories) didn’t make it in time into the Wasm tools we use, so we are stuck with this transitional API for now. But in principle you are absolutely right.

3 Likes

Hm, when you mean “expensive bulk system API function calls”, do you mean that storing data in stable memory is more expensive than in non-stable memory? (If not because of any differences in storage cycle cost but instead due to the additional computation required to transfer data in and out of stable memory in pre- and post-upgrade functions?)

1 Like

In terms of actual running time, normal memory is accessed using Wasm memory instructions which are compiled to memory instructions, so that’s native speed fast. Accessing stable memory with the current API requires function calls into the host, bounds checking, explicit copying etc., so definitely more expensive. Whether that cost is currently reflected also in cycle costs (which you probably care more about than running time) I don’t know, but we can assume that cycle costs will be refined to match real costs eventually.

In the original vision, building on Wasm multi memory support, stable memory would be accessed the same as the main memory, and would likely have the same access cost. One wouldn’t speak of “main” memory at that point any more, as there can be any number of memories, of varying lifetimes (per message, maybe per call, per module version, permanent).

It might help if the people at DFINITY could share the “Refining persistence” document by Andreas Rossberg, which explains this better.

8 Likes

I would love access to said document

It seems the code for 64 bit stable memory is live! The IC runs this revision:

~/dfinity/ic $ curl https://ic0.app/api/v2/status|cbor2yaml.rb|grep impl_version
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   331  100   331    0     0    565      0 --:--:-- --:--:-- --:--:--   564
  impl_version: 27e1eadbcbe90abfe56d9c8dfd39e1a78e52c624

which, according to the code dump repo history, is dumped as 89446f5

~/dfinity/ic $ git log --oneline
35dd8f9 Update from revision 8a5b9a2e1468dfb286c77084a9b3597b9e3993b5
e362530 Update from revision d8dabe1cb0bb1e60a11e5bb8033d1615fefb252f
89446f5 Update from revision 27e1eadbcbe90abfe56d9c8dfd39e1a78e52c624

and indeed it’s there ic/system_api.rs at 89446f5a04f053040b4863eab5458446d925ed0e · dfinity/ic · GitHub.

So early adopters can probably start playing around with it, even if it didn’t make it to The Internet Computer Interface Specification :: Internet Computer or GitHub - dfinity/ic-hs: A haskell toolbox for the Internet Computer yet.

It would be helpful to the outsiders if the code dumps would include a changelog (maybe auto-generated from the internal repo?), to easier see what changes to look out for.

5 Likes

@nomeata : no the code is not actually live yet. These are stub implementations. See ic/system_state_accessor_direct.rs at 89446f5a04f053040b4863eab5458446d925ed0e · dfinity/ic · GitHub.

Some more of the implementation will get pushed out in the next upgrade. However, we are also planning on doing an NNS proposal where we will ask the community to vote on whether or not this feature should be enabled or not. Only if the vote passes, then we will push the final changes that will enable the feature.

5 Likes

What would be the ETA of getting these system APIs into Motoko once they are live?

1 Like

cc: @claudio . Maybe he can share his thoughts.

1 Like

Hey Folks!

This project is our guinea pig project for what public roadmap projects will look like, and the next 7 days will have a lot of activity surrounding it.

  1. Today, August 25 - Community Conversation on Increased Canister Storage

  2. Thursday, August 26 - Draft proposal posted on the forum for review. We will post to the developer forum thread on the project a markdown doc (hosted on Github repo for NNS proposals) that explains the project design and intent so people can see what the project is about and how we intend to achieve the desired goal.

  3. Wednesday, Sep 1 - This is the nuanced part: I will submit to the NNS a proposal (without a binary that upgrades the IC) asking the wider IC community to vote on whether the Foundation and wider community should continue working, and ultimately deploy, the work for increasing smart contract canister storage. We are essentially using the NNS’s voting mechanisms to let the community express themselves.

  4. Friday, Sep 3 - NNS Proposal has a 48-hour expiration so the decision will be completed by then.

There will be two possible outcomes from this:

a. Proposal passes - if it passes, then the project team will submit an NNS proposal to actually upgrade the IC. In the case of Increased Canister Storage, this will happen relatively quickly since a lot of the groundwork had been done so the actual changes will be light. This is an exception. Many times in the future, if a proposal to “work on this version of the plan” passes, it may take weeks or months to have an implementation ready.

b. Proposal fails - I think this is very unlikely, but worth considering that this means the community does NOT want this change. In which case, we will not continue to work on this plan, but potentially go back to the drawing board and discuss it further with the community

Please note: in order to let the community truly express themselves, the DFINITY Foundation will vote as late as possible. We know lots of people’s neurons follow the DFINITY Foundation so we want to let the community time to breathe.

7 Likes

Our Community Conversation on Increased Canister Storage with @akhilesh.singhania goes live in a few min:

Join the conversation to be able to participate in our live Q&A :tada:

3 Likes

Will DFINITY or the ICA be “banned” from voting on this proposal? How can we ensure the vote actually represents the will of the community if very large players vote?

Not sure if you saw this in my note above.

1 Like

I don’t want to sound more opinionated or “final” than i actually intend so I think I should write the facts and the intents:

The facts

  • there is no mechanism to ban anyone from voting
  • I think this is a very popular proposal

The Intent

  • the foundation intends to get as much community feedback as possible
  • the foundation intends to Vote last (with the knowledge of what the community voted), but it does intend to vote (as member of community)
  • this is very much an interactive and iterative process so If there are any unintended consequences of this model, we will iterate. Not set in stone. Maybe for more controversial proposals, we can try something different.
5 Likes

As we discussed earlier, we are posting our plan for Increased Canister Storage. You can read our intent, design decisions trade-offs, caveats, and rollout plan.

As Diego mentioned, the current plan is that on Wednesday September 1, Diego will submit an NNS proposal for the community to vote whether the final version of the plan should be implemented.

Please give us feedback, so we can improve and give the IC the best possible plan to increase smart contract canister stable memory.

NNS Proposal: Add 64-bit stable memory API

Main authors: @ulan , @akhilesh.singhania

Objective

Currently canisters are effectively limited to 4GiB of storage. This is because stable memory uses the 32-bit addressing scheme and when a canister is upgraded, its Wasm memory is wiped.

A number of applications can benefit from access to additional storage without having to partitioned into multiple canisters. The goal of this proposal is to introduce a stable memory API that allows canisters to address more than 4GiB of memory allowing canisters to have more storage bound [eventually] only by the actual capacity of the subnet.

The goal of this proposal is to increase the amount of memory that canisters can access [eventually] bound only by the actual capacity of the subnet. Since, the Memory64 proposal is not standardized yet and its implementation in Wasmtime is not production ready yet, this proposal enables the increase by introducing a new stable memory API.

Background

When the stable memory was first designed and implemented, the Memory64 proposal was not standardized yet and only recently it was implemented in Wasmtime but the implementation is not production ready yet. Further, the multiple memories proposal is still in Phase 3. With this in mind, stable memory was designed with a 32-bit addressing scheme so that it could eventually be replaced by Wasm native features.

Supporting 64-bit Wasm memory presents a number of performance challenges for the Internet Computer. Currently Wasmtime can efficiently eliminate bounds checks for 32-bit memory accesses by using guard pages. A similar optimization is not possible for 64-bit memory accesses making them more expensive. Besides that more optimization work around memory.grow() needs to be done before the implementation is production ready.

Since bringing the Memory64 support to IC may take a while and some canisters need large memory now, we propose an extension of the stable memory API to 64-bit instead.

Proposal

We propose to add four new functions to the System API that mirror the existing 32-bit functions:

  • ic0.stable64_write: (offset: i64, src: i64, size: i64) -> ()
    • Copies the Wasm memory region specified by src and size to the stable memory starting at the given offset. Note that this API uses 64-bit addressing for the Wasm memory even though at the moment the Wasm memory only supports 32-bit addressing. This is done to keep the possibility of supporting 64-bit Wasm memory open in the future.
  • ic0.stable64_read: (dst: i64, offset: i64, size: i64) -> ()
    • Copies the stable memory region specified by offset and size to the Wasm memory starting at the given address dst.
  • ic0.stable64_size: () -> (page_count: i64)
    • Returns the number of 64KiB pages in the stable memory as a 64-bit integer. Note that it would be possible for this function to continue to return a 32-bit integer as the 32-bit version of the API does. With the page size of 64KiB, a 32-bit integer could address up to 256 TiB which could be sufficient for a very long time. However, it was felt that there should be a clear distinction between the 32-bit and 64-bit versions of the API and having this API return a 64-bit integer should not have any negative impact.
  • ic0.stable64_grow: (additional_pages: i64) -> (old_page_count: i64)
    • Tries to grow the memory by new_pages many pages containing zeroes. If successful, returns the previous size of the memory (in pages). Otherwise, returns -1.

As the specification repo of the Internet Computer is not open source yet, please see the proposed diff here.

Backwards compatibility

In order to ensure smooth transition and upgrade we allow canisters to use the 32-bit and 64-bit versions interchangeably up to 4GiB. In other words, both versions operate on the same stable memory. As soons as the size of the stable memory grows beyond 4GiB the 32-bit versions cease to work. Calling them will result in a trap.

Risks

The main risk is canisters mixing the 32-bit and 64-bit functions after the stable memory grows beyond 4GiB. We somewhat mitigate the risk by ensuring that 32-bit functions will trap in such a case so that the canister execution stops instead of continuing with a wrong result.

Alternatives Considered

  1. One alternative is to introduce a completely new 64-bit stable memory that is disjoint from the existing 32-bit memory. While it is a cleaner design, it would complicate canister upgrades because canisters would need to copy the existing state from the 32-bit stable memory to 64-bit stable memory.
  2. Another alternative we considered was to say that once a canister uses the 64-bit version of the API then using any of the 32-bit API will result in a trap. We felt that this would allow canisters to more easily detect problems in switching between APIs however we felt that this may overly complicate the implementation.
  3. Another alternative is to wait until the Memory64 and Multiple Memories proposals are production ready. Then the stable memory can be represented as one of the multiple 64-bit memories removing the need for the new System API.

Rollout plan

When introducing a new functionality to any production environment, there are two types of risks that should be managed:

  • Regardless of all the testing done before the rollout, there is still a risk of the new functionality introducing some bugs in production.
  • Regardless of all the initial feedback gathered about the new API, using it in production may reveal some shortcomings requiring adjustments to the API impacting the canisters that already depend on the API.

Hence, the rollout plan for this proposal would be the following:

  • Even though the new API allows canisters to address the entire capacity of the subnet, the stable memory of a given canister will initially be capped at 8GiB.
  • The API will be marked as experimental and use of the API in essential canisters will be strongly discouraged. This way the community can feel encouraged to make and accept future proposals to break existing canisters using the API in order to not support deprecated APIs thereby keeping the API as clean and easy to understand as possible.
  • After we gain confidence in the API and in its implementation, over subsequent NNS proposals, we will gradually increase the size of the stable memory and mark it as no longer being experimental.
14 Likes

There was a question about how we will test this feature before we deploy it to production. I will write a small text here about how we do testing in general. I will also ask the testing team to write a more detailed post to provide more details.

  • As we use Rust, we rely heavily on its unit testing framework for writing unit tests
  • Then we have a e2e integration testing framework where we can bring up a simplified version of the Internet Computer on a single machine and validate a number of properties.
  • Next we have the reference implementation of the Internet Computer which is used to validate a number of additional properties.
  • Next we have a number of testnets where we deploy fully functioning Internet Computers to further test in an environment as close to production as possible.

We have a CI / CD pipeline that runs various tests using the above mechanisms. We run some tests on each PR as it is merged and we run some tests on an hourly basis, some on a nightly basis, etc.

7 Likes

Thanks! All sensible and well done. I endorse this proposal! (I should probably get a neuron that people can follow if they like my endorsements.)

I think an even better argument for returning i64 here is that the memory.size Wasm instruction will return an i64 in the memory64 extension, and the stable memory API should stay close to the wasm instructions it stands in for.

I like the word “yet” in this sentence.

Your prose sounds as if accessing the first 4GB will trap if the memory is large than 4GB, which I think is good. But the diff adds the check

if offset+size > 2^32 then Trap

This probably should say

if |es.wasm_state.stable_mem| > 2^32 then Trap

in stable_read, stable_write and stable_size.

I expect soon there will be a demand for a way for canisters to find out how much memory they can use (besides trying). But I don’t mind that not being part of the API, as there is no such functionalities for real WebAssembly memories, and sticking close to the native Wasm experience is probably useful.

3 Likes

Thank you for providing this information. I’d really like to learn more about the testnet environment. Hopefully there will be a lot of detail on this topic in the test team’s post.

1 Like

Foundation actually has a dedicated testing team so we have asked them to help document and write more about this for folks. thanks for the feedback!

3 Likes

pack it up, we are good to ship then :wink:

You totally should!

2 Likes