Wasm-Native Stable Memory
Objective
The goal of introducing Wasm-native stable memory is to improve the performance of stable reads and writes by letting these operations directly access stable memory in the same way Wasm loads and stores currently access the Wasm heap. This will make direct use of stable memory more practical and it will not require canister developers to make any changes to how they use stable memory.
Background
What is Stable Memory?
Stable memory is an array of bytes that canisters can use and is persisted across canister upgrades. Canisters with smaller amounts of data can use stable memory to persist data across updates by serializing objects from the Wasm heap into stable memory before an update and deserializing it back to the Wasm heap after the update. But larger canisters may permanently keep their data in stable memory and only move pieces of it to the Wasm heap when needed.
Stable memory is accessed through the stable64_read and stable64_write system API calls which copy a slice of data between stable memory and the Wasm heap.
Why is Stable Memory slower than the Wasm Heap?
The stable memory system API functions are currently implemented by Rust code which is registered with the Wasmtime engine when it executes a message in a canister. This means that each read/write to stable memory requires the canister to call into the Wasmtime engine, which then executes the Rust function to handle the read/write (see diagram [1]). On the other hand, when a canister performs a read/write to the Wasm heap, the Wasmtime engine knows where the Wasm heap resides in RAM so it can compile the canister’s Wasm instructions into a single assembly load or store instruction.
This proposal will make the performance of stable memory accesses closer to the performance of Wasm heap accesses.
Benefits
One current disadvantage of storing canister data directly in stable memory is that accessing it can be significantly slower than accessing the Wasm heap. This feature would significantly improve the performance of stable reads and writes (likely by 2x or more for certain workloads) to bring them closer to the performance of Wasm heap reads and writes [2]. This would make it easier for canisters to store the majority of their data directly in stable memory and keep the Wasm heap for “scratch space”. This pattern has several advantages:
- Canisters can store more data: Stable memory can currently hold up to 32 GiB as opposed to 4 GiB for the Wasm heap.
- Upgrades are faster: Upgrading no longer requires serializing and deserializing the entire Wasm heap.
- Upgrades are safer: Removing the (de)serializing steps can make upgrade code simpler which decreases the likelihood of bugs or hitting message instruction limits.
Furthermore, introducing the ability for canisters to use multiple memories opens doors for several possible future improvements in the area of BigData:
- Canisters may eventually be able to have several different memories with different properties (e.g. many small memories that are just as fast as the wasm heap, or memories with different lifetimes).
- Tracking memory accesses could be done within the canister itself (in an additional memory) which would be faster than our current segfault handler implementation.
- We could introduce the ability for additional memories to be passed between canisters.
Proposal
Wasm-native stable memory would use the Wasm multi-memory and memory64 features to define a second 64-bit memory which the canister can directly access (diagram [3]). By “directly access” we mean that stable API calls will be converted to Wasm memory.copy instructions which Wasmtime can compile to standard assembly load/stores. This will be much faster than the layers of function calls currently needed to go through the Wasmtime engine. In particular, the main changes to be made are:
- Enable the multi-memory and memory64 features in Wasmtime and implement those features in our Wasm parser and code generator.
- Modify canister instrumentation to inject the second 64-bit memory for stable memory and replace stable memory API calls with the corresponding
memory.copyinstructions. - Modify the existing segfault signal handler to be aware that there are now two memories each canister will be directly accessing and properly handle the two cases.
Note that canisters will not directly use the multi-memory or memory64 features - they will only be used through code that is inserted during the instrumentation step. But it may happen that a later proposal offers a way to give canisters direct access to these features.
Risks
The Wasm multi-memory feature is not yet standardized and has not yet been implemented in most Wasm engines. This means that we may waste engineering time if modifications are made to the spec while it is being standardized. It is also possible that the multi-memory feature is ultimately rejected, in which case this proposal would need to be rolled back. But that wouldn’t break any canisters because canisters will not directly use the multi-memory feature.
The memory64 feature is also not standardized, but it has already been implemented in Chrome and Firefox so it is likely to be standardized without any changes.
What are we asking the community
- Review comments, ask questions, give feedback
- Vote accept or reject on NNS Motion
- Participate in technical discussions as the motion moves forward
[1] Canisters currently access the Wasm heap directly, but cannot access stable memory directly.
+-------------------+ +-------------+
| | Direct | |
| Canister +----------------> Wasm Heap |
| | Read/Write | |
+----------+--------+----------------+-------------+
| | |
| | Wasmtime |
| | |
+----------+---------------------------------------+
| | |
| | System API Rust Implementation |
| | |
+----------v---------+-----------------------------+
| |
| Stable Memory |
| |
+--------------------+
[2] Stable reads/writes would still not be quite as fast as Wasm heap reads/writes because the Wasm heap only has 32-bits of addressable memory which allows Wasmtime to do some neat tricks to avoid checking that the memory accesses are in-bounds. The stable memory would need to use 64-bit addresses to allow it to grow beyond 4GiB and that means Wasmtime needs to insert a bounds check before each access.
[3] Wasm-native stable memory would allow the canister to directly access stable memory.
+---------------+ +----------+ +-----------+
| | Direct | | Direct | |
| Stable Memory <---------------+ Canister +---------------> Wasm Heap |
| | Read/Write | | Read/Write | |
+---------------+---------------+----------+---------------+-----------+
| |
| Wasmtime |
| |
+----------------------------------------------------------------------+
| |
| System API Rust Implementation |
| |
+----------------------------------------------------------------------+
