Permanent data in a stable memory, not really permanent

Developers Language Support
1

Please, tell me I am wrong :slight_smile:
I am almost done developing my application on ICP, and for a reason, I change one type by adding new member, I have already functionality to migrate data from HEAP to stable memory and copy back after the update. I got message when I was deploying the new Backend canister, saying that I am going to loose data, I proceded with yes, and all data I had was lost, the good thing is that, the data is just test data, I am scarred of what would happen when I am in poroduction? Is there any strategy to save persistant data and be sure it wont be lost like a usual database? Or do I need to go traditional web2.0 way to save data, in the case the whole ICP development will be pointless. Please help

1 Like
2

My answer below assumes you’re using Motoko. I’ll let someone else who’s more familiar with the Rust options (i.e. stable data structures, etc.) provide a follow-up.

If you’re using Motoko, I’d recommend reading this tutorial Stable variables and upgrade methods | Internet Computer, for which the stable keyword does the work of serialization/deserialization under the hood for you instead of needing to write your own preupgrade and postupgrade to perform this serialization.

If you need to migrate a stable data variable or data structure (say your data schema has evolved), then you need to write the code that migrates that stable data structure. Data migrations are probably the biggest headache on the IC, but there are various data migration patterns that have been developed by the Motoko community.

3

Thanks for the answer! I must use preupgrade and postupgrade, because I have the heap data in a Hashmap which cannot be stable, so when I redeploy I have to move the data into a temp stable list and migrate back to the Hashmap. I think I understood your suggestion, when schema is changed, I need to manually migrate the data, simply copying the data somewhere, and copy back after the migration.

I

1 Like
4

There are plenty of stable data structure collections that have been developed by the community.

Here’s a few that I’d recommend, all that use MOPS - on-chain package manager for Motoko! for easy 3rd-party Motoko library integration.

In general, I’d recommend heap-based stable collections over ones that directly manipulate stable memory since the Motoko team is currently working on improvements to the language and compiler that will provide a “stable heap” on top of the DFINITY foundation’s current work to bring wasm64 to canisters.

Here are a few data structures you might use often/find helpful:

You can find tons of Motoko packages at https://mops.one/

Btw, a few years ago, I had a similar question to you and asked it here → Clarification on Stable Types with Examples. That thread has some information on how you can write your own stable collection if you’re at all interested.

1 Like