if you code in rust and use stable-structures GitHub - dfinity/stable-structures: A collection of data structures for fearless canister upgrades. you can already download the SM and put it back. The only issue AFAIK is that if there’s an update call when download/restore happens the data can get corrupted.
I know @ielashi is working on some sort of locking mechanics but he can share more details.
We are considering a feature on ic-stable-structures to take snapshots of stable structures, which would be part of the library, be completely in user-space, and not require protocol support. That feature has not been scoped yet, but it’s certainly an option to introduce in the future if there’s enough demand for it.
Hi, canister backups and snapshots are a valuable feature for writing canisters.
The canister_tools rust library can create snapshots of the global-variables in the heap for backup and restore purposes. Controllers can create snapshots, download in chunks, and even upload a snapshot in chunks and then load the uploaded snapshot back onto the global variable. At the same time making a simple upgrade strategy for global-heap-variables. The user can choose the serialization format for the snapshots, default is bincode. The library works with the virtual memories feature of the ic-stable-structures library so stable-structures can be used alongside global-variables in the heap. The library also creates public controller methods for reading and writing the virtual stable-memories.
As we develop Azle and Kybra, especially considering their beta status, and especially when dealing with StableBTreeMap, I feel very uncomfortable as developers seem to be on their own with their StableBTreeMap upgrades. I’m always afraid something terrible will go wrong because of a mistake in Azle or just a misunderstanding of StableBTreeMap versions under-the-hood.
Seems like snapshots would give me a lot of peace of mind, as love canister upgrades would not be fatal. Of course backups are already possible, but IMO one of the worst problems with extra-protocol backups are that you then have to entrust the data, which could be sensitive, to some controller’s machine.
Hey @ielashi
Any updates on this?
Hi @Gabriel,
We are currently in the process of implementing this feature. Soon, we will release a version of the SDK for developers who would like to test and experiment with the feature. We will keep you updated and let you know once the SDK release is available.
Hi, I was wondering when this backup/restore feature goes live, do I need to clean the data in the current canister to use this feature?
Hi @e274426380 , could you please elaborate on what you mean by 'cleaning the canister’s data '? Are you referring to any specific actions that need to be taken on your end to utilize this feature effectively?
This feature, which is under implementation, will provide canisters a functionality ingrained in the system. Controllers can choose to take a snapshot of the canister data or restore previously saved data to the canister’s state. Once the feature is available, any canister can use it as is.
I learned from canister-tool that if you want to use canister-tool to backup restore data, you need to install the canister-tool method in pre-upgrade first, so this process is necessary to clean up the originally saved data when you use canister-tool for the first time.
So I’m worried that the upcoming backup system will have the same problem.
@Alexandra Is there a timeframe when this is expected to be released to the public?
Sorry for the delay. The last code bits were just merged. We’re enabling the feature on a test subnet and in dfx. We’ll update you once the feature is ready for testing…
Exciting Features.
For a long time, every new project party needed to put some effort into creating their own backup system over and over again.
Hey folks,
We’re happy to announce that the Canister Snapshots are available in the latest dfx beta. To install it run:
dfxvm install 0.23.0-beta.3
Earlier next week (Sep 2-3) the feature will be rolled out to the 26 app subnets, and (fingers crossed) in two weeks (Sep 9-10) to the rest of the subnets (but NNS).
The spec changes and the documentation are still work in progress, but there is an example: examples/rust/canister-snapshots at master · dfinity/examples · GitHub
I was wondering if this new feature will be integrated with the SNS workflow? Will SNS canisters (root, ledger, index, etc.) be automatically backed up upon upgrade through a proposal? Likewise, will the same apply to dApps controlled by SNSes? Or will snapshots and proposal upgrades always be separate?
This is just a question that crossed my mind, so I wanted to share it. Curiosity question.
Thanks for the feature. Are there any restrictions on the snapshots (e.g., quantity/size)?
I was wondering if this new feature will be integrated with the SNS workflow?
I’m not aware of any concrete plans, but perhaps @lara knows more.
Are there any restrictions on the snapshots (e.g., quantity/size)?
Currently, the number of snapshots per canister is limited to one. This limit will be definitely increased.
In general, snapshots take subnet memory, so other memory-related limits apply. For instance, the total size of all snapshots is limited by the subnet capacity (700GB).
Currently there is nothing on the SNS roadmap regarding snapshots.
For the dapp canisters, this would maybe already be done via generic proposals.
Which exact use case would you want to cover?
Suppose I modify the data structure causing a data conflict and need to clean up the data to deploy this update.
So can I use the backup system to import the previous data in the new data structure after I have finished modifying the data structure?