Enhanced orthogonal persistence for Motoko is a great step forward, but is there any similar functionality planned for Rust that could be called Stable Heap?
I am currently trying to build Stable Graph in Rust.
It uses ic-stable-structures::Memory to implement a graph structure with mutable compressed sparse rows in a canister.
The algorithm is fast, but the drawbacks are frequent deserialization when traversing nodes and edges with properties, and the maximum number of Memory required is seven.
If the code can be made more Rust-native, these drawbacks may be eliminated.
Hi @wiyota,
This isn’t something that would really be possible with Rust. It’s possible in Motoko because the team is committing to a heap layout that future versions of Motoko will still be able to handle. The Rust language doesn’t aim for any kind of stability of the heap layout across compiler versions so there’s no way to ensure that the new version of a Rust canister can use the heap left by an older one.
You are absolutely right. My current plan is to update both directly accessed heap and stable memory for backup for some parts that need fast access, such as Vec for virtual pointer swapping, to improve efficiency, but I will look into developing on Motoko.
Would it work as long as the compiler version remained the same?
If so that could still be useful. A canister would be pegged to a certain compiler version and maybe some other settings but could take advantage of the excellent developer experience of true orthogonal persistence.
It may be possible to be more flexible by using some non-stable structures in heap memory normally, and storing/retrieving to/from StableVec only when the compiler version is changed.
If this could be done, it would take advantage of Rust’s speed and existing assets without runtime overhead.
Even with a fixed compiler version, I don’t see how this could work. First, I don’t even think Rust promises that the layout will be the same even with the same compiler version. Probably in practice it doesn’t vary within the same optimization level, but I’m not even sure of that.
Also there’s no way to dynamically determine what types are stored in the heap. For example, if you had a u32 somewhere in the heap and deploy a new version of you canister with this value being a f32, then the new version will just reinterpret the old bytes incorrectly.