Rust: using stable storage as main storage - Example/Pattern

Developers
1

I’ve read the article here by @roman-kashitsyn

and the Community Conversation here:

where he advises this here:

Consider using stable memory as your main storage.

But I’ve been unable to find a simple example that goes over doing this and breaks down the steps with explanations.

I’ve also tried to follow along to the code for the Internet Identity canister here that is supposedly doing this but unable to grok it entirely:

Can someone capable with Rust provide a simple example of doing this that can be generalized for any custom struct or data structure?

2 Likes
2

Pssst, hey!

Tutorials are on their way!

3 Likes
3

Hi @senior.joinu

Thank you so much for this.

I went through the documentation for it and had a couple of questions:

  • In the first example as shown below
    image
    we can directly set to and retrieve from stable memory using the get_var and set_var functions provided. Will this support all the collections from the standard library or is it only limited to specific data structures?

  • In the third example as shown below

    image
    image812×301 9.85 KB

    how does the calling code determine the from and to values to be passed? Is there some adjoining index mechanism that is available to the calling code that can help it determine what to pass?

  • Are there plans to create stable variants of other data structures such as a BTreeMap or a Set?

  • How does one handle changes to the schema? If I were to store a User struct with name and age in a SHashMap today and then later decide to add another field gender to the User struct later, how would I migrate the existing collection?

Again, thank you for sharing your expertise. Highly grateful for the insights :slight_smile:

4

Hi @saikatdas0790

Any structure that implements CandidType trait will work fine. So any standard type that implements it should also work out of the box. For those types, which don’t implement this trait (as BinaryHeap, for example) you may just want to make a wrapper type that will implement it.

Oh, no. These “from” and “to” variables are just regular code. There is no notion of pagination in ic-stable-collections.

It is possible to add a support for HashSet pretty quickly. But BTrees seem quite tricky for me to implement, so it would take some time. But, yes, BTree-based collections are the first in the queue.

There are options. You can define your struct as a versioned box beforehand. E.g.

struct UserV1 {
  name: String;
}

struct UserV2 {
  name: String;
  phone_number: String;
}

struct User {
  version: u16;
  data: Vec<u8>;
}

Then you can serialize UserV{X} into the data field of User struct and type a correct version number.

Also, you can just introduce a new collection instance for each new version.

struct UserV1 {
  name: String;
}

struct UserV2 {
  name: String;
  phone_number: String;
}

let users_v1 = SHashMap::<ID, UserV1>::new();
let users_v2 = SHashMap::<ID, UserV2>::new();

But in general, you should be able to use any other data migration technique that you’re using right now with common collections.

2 Likes
5

Thank you so much for the response and the explanations.

I will definitely give this a try and provide any feedback that I have.

Cheers

1 Like
6

Hi @senior.joinu

I tried using it and ran into a snag. I created an issue for it here

Let me know if there’s any other details I can provide. Thank you for the awesome lib :slight_smile:

1 Like
7

Hey. Thanks for trying it out and opening the issue.

Yes, for now you need a nightly version of Rust.
I’ll see what I can do in order to make it work with stable!

1 Like