Canister backup and restore [Community Consideration]

Loading a canister snapshot reverts the canister’s Wasm module, heap, stable memory etc to their previous states. This means that the canister’s data structures will be restored to their state at the time the snapshot was taken.

I would like to know if the Foundation has any ideas to develop relevant tools?
For a long time, every developer has had to deal with the hard choice of having to clean up the corresponding data when the data structures conflict.
And to cope with such a situation, every developer needs to repeatedly develop a backup tool.

Currently, ICP offers two approaches to handle this:

  1. Serializing/deserializing data to/from stable memory.
  2. Using stable structures persist across upgrades.

In future iterations, we may implement the ability to download and upload Canister Snapshots. This could be also a step towards addressing your request.

Unfortunately, data compatibility is a complex issue, and it’s unlikely that there will be a solution that satisfies everyone…

3 Likes

I think this would be a very flexible backup system if it supported downloading and uploading snapshots.
At least we can build a backup system based on the snapshot system.
Instead of rebuilding the backup system from scratch.
I hope the Foundation understands that the matter of resolving data structure conflicts that lead to data clearing is a common problem for developers just getting started in IC, and that every developer needs to spend effort to solve this problem.
Every developer would like to have a tool to solve the death spiral of building wheels over and over again.

2 Likes

For example, rolling back the upgrades of dApps controlled by an SNS—rolling back upgrades that would break the dApps after the execution of generic proposals.

Similarly, this applies to upgrades of SNS canisters (root, index, etc.).

I’m just assuming that someday, an SNS upgrade, even through a proposal, could result in a broken canister. Therefore, it crossed my mind that leveraging this backup functionality could be interesting for the SNS tooling as well.

1 Like

Yes, I also had that in mind. I would appreciate more technical updates like these and fewer changes that interfere with SNS DAO governance.

I just read that the latest version of dfx supports this feature, so I was wondering if the spec has been finalized?

The spec PR is considered final. If you’re asking “when will it be merged/released”, this is blocked to the feature being fully rolled out to production. So, I expect it to happen in the next couple of weeks.

I asked about both, thanks for the answer. It’s good to know they are considered final.

Regarding “when it is merged” this is a requirement to implement the feature in @dfinity/ic-management, as the declarations and the generation of their types are automated on a weekly basis with CI jobs. I’ll keep an eye on that job next couple of weeks then.

1 Like

Hey, I have some follow-up questions:

  1. Do you know when it will be possible to create more than one backup at a time?
  2. It would be great if you could add the timestamp as an additional field indicating when the backup was created.
  3. What is the reason we can’t download the backup blobs locally?
  4. I noticed that for two different canisters with the same WASM installed, we get exactly the same IDs back. Is that intentional?

Do you know when it will be possible to create more than one backup at a time?
What is the reason we can’t download the backup blobs locally?

These two are both extensions that we haven’t prioritized yet. It would be nice to hear about your use cases and understand what is the need to help us prioritize better.

It would be great if you could add the timestamp as an additional field indicating when the backup was created.

This should already be there. Please check the small example dapp. In step 8 when you list the snapshots you can see the timestamp.

I noticed that for two different canisters with the same WASM installed, we get exactly the same IDs back. Is that intentional?

This does not sound right. The snapshot id depends on both the canister id and a counter per canister. Do you have the list of steps you took? Was this locally or on mainnet?

Thanks for getting back to me, you missed question 3 :slight_smile:

I haven’t tried on mainnet this, only local.

Canister R creates canister G and U on the fly. U creates canister G1,G2…Gn as well on the fly. Added some data in both G1 and G2 and then ran the command to create backups for both.

I can share a repo later on with the code.

I don’t think I did. It’s bundled with 1 :slight_smile:

This would be helpful to understand what’s going on.

Hey,

I think I missed the timestamp.

Ideally we want to be able to have multiple backups so we can test previous versions of the game. Also downloading the blobs we would be able to test in multiple enviroments with actual real data from mainnet so that would super helpful for us.