Effortless SNS Updates

TLDR; We propose a better process for SNS upgrades. (1) Each SNS stores a target version that can be changed via a new type of AdvanceSnsTargetVersion proposals. (2) The SNS is responsible for performing upgrades in the background, until the target version is reached. (3) Users can monitor and audit upgrades of their SNS via a single new endpoint.


Dear SNS communities,

Our team is working on an exciting new feature that will simplify keeping the SNS framework canisters up to date.

Background

The SNS framework consists of multiple canisters (Governance, Ledger, Swap, etc.) that are regularly upgraded with new features and security enhancements. SNS upgrades come in the form of canister WASMs that are published (via NNS proposals) to the SNS-W canister. Each SNS is responsible for pulling the next available upgrade from SNS-W (via an SNS proposal called UpgradeSnsToNextVersion).

What’s currently the problem?

Currently, to upgrade multiple SNS canisters, the SNS community needs to approve each SNS upgrade by an individual UpgradeSnsToNextVersion proposal. While the SNS framework consists of multiple canisters (Governance, Ledger, Swap, etc.), a single proposal UpgradeSnsToNextVersion upgrades just one SNS canister. Since SNS canisters are always upgraded in order, upgrading the whole SNS to the latest version might require multiple proposals.

A lot of manual work is required from those submitting and voting on upgrade proposals, especially since adopting upgrade proposals should be timed to avoid races, i.e., concurrent proposal failures, and the need to re-submit more upgrade proposals. Simply put, it is hard to understand what voters vote on.

We propose to simplify the process by allowing SNS communities to send just one proposal to convey the intent to upgrade their SNS to the newest available version. To this end, we propose to introduce a new SNS proposal type called AdvanceSnsTargetVersion. Submitting such proposals will be trivial: similarly to the current UpgradeSnsToNextVersion proposals, no input arguments are needed. If such a proposal is adopted, the SNS will update its target, i.e., the version this SNS should eventually reach (as opposed to actually performing any canister upgrades). The SNS will then automatically perform the required upgrades of its SNS canisters in the background, ensuring upgrades are done in the right order, until the version of this SNS reaches its target version. This background process would not require any additional proposals or manual work. Users will be able to monitor and audit this process using a single new SNS governance endpoint.

Example scenario
  1. The SNS receives the newest available SNS version (v3) from SNS-W.

  2. Initial information from the upgrade journal:

    • Current version is v1 (this is the currently running version of this SNS).
    • Target version is also v1 (this is the version the SNS has committed to reach).
    • Newest version is v3 (i.e., two upgrades are possible from the p.o.v. of SNS-W, where the SNS upgrades are published).
  3. Community submits an AdvanceTargetVersion proposal.

    • The SNS sets the new target to the newest version, v3 (at the time of proposal submission).
  4. The community adopts the AdvanceTargetVersion proposal. It is then executed, updating the target to v3.

  5. The SNS performs upgrades via background tasks, until the target is reached.

  6. Users monitor the progress via the upgrade journal.

Next steps

We plan to propose the SNS governance wasm that enables AdvanceSnsTargetVersion proposals as part of the normal release process.

If this feature is adopted by the NNS community and when SNS projects receive this new wasm, the legacy proposal type UpgradeSnsToNextVersion will still work as before for some time. This ensures that a battle tested upgrade mechanism is in place for upgrading SNSs (in the unlikely event that of a bug in AdvanceSnsTargetVersion that will not be caught in code reviews and testing) and that integrations have some time to update if needed. Eventually, we will propose to obsolete UpgradeSnsToNextVersion proposals.

If you have any questions or suggestions in the context of this design proposal, please let us know in this thread.

16 Likes

Cool!!
We had to manually perform 28 upgrade proposals over a few hours.

2 Likes

Awesome feature. Can there be an optional parameter in the new proposal type: opt target, that lets the proposal specify the target version instead of using newest available version?

2 Likes

Can there be an optional parameter in the new proposal type: opt target, that lets the proposal specify the target version instead of using newest available version?

Yes, that’s the plan. Glad you hear the proposal sounds useful to you!

1 Like

As someone currently exploring how to upgrade an SNS, this sounds like exactly the improvement we need! I have two questions:

  • Is there a target release date?
  • Could this new method assist in upgrading an older SNS, or will I still need to submit multiple proposals one last time to reach the WASM implementation for that method, if that applies to my situation?
2 Likes

Great questions, @Seb.

  1. Improving SNS upgrades is a high priority for the team. There is no target release date yet, but I’m going to update this thread when I know more.
  2. If the community adopts this feature, the corresponding SNS Governance WASM would be published in the SNS-W canister. Then, each SNS would first need to be upgraded using the legacy feature (i.e., adopt a sufficient number of UpgradeSnsToNextVersion proposals), and only then would the new feature (AdvanceSnsTargetVersion) become available. In other words, there would be one last SNS upgrade marathon.
2 Likes

Any efficiency gains are, of course, welcome, especially when that efficiency is gained in work that does not directly impact the user (kinda).

2 Likes

The proposal to add this feature is now live:
https://dashboard.internetcomputer.org/proposal/134342

DFINITY plans to vote on this proposal the upcoming Monday, December 2.

1 Like

Important update & action required

TL;DR Now is the perfect time to upgrade your SNS to the latest version.

With https://dashboard.internetcomputer.org/proposal/134342 adopted, the new proposal type AdvanceSnsTargetVersion, as announced in this thread, is now available to all SNSs. This includes Levi’s request to have an optional field (called new_target : opt SnsVersion to optionally specify the new target (if unset, the latest available version at proposal submission time will be automatically selected for the new target).

As always, it’s in the hands of the SNS communities to upgrade their SNS to actually obtain the new feature (unsurprisingly yet ironically, one needs manual effort before one obtains a greater degree of automation for the future).

Pending Upgrades Breakdown

  1. The following SNSs have already reached the latest version and already benefit from simplified operations. Congratulations!
    • Catalyze, DecideAI DAO, Dragginz, EstateDAO, ICPanda DAO, OpenChat, Sneed, YRAL
  2. The following SNSs are very close to the latest version. One last push!
    • ICPSwap, Motoko, Neutrinite, Seers AI, ICGhost, DOGMI, ICLighthouse DAO, ----, KongSwap
  3. The last group is more than 10 upgrades away; these projects would definitely benefit from having a more automated upgrade process.
    • Gold DAO, ORIGYN, Kinic, OpenFPL, ELNA AI, WaterNeuron, SONIC, BOOM DAO, ICPCC DAO LLC, ICVC, Nuance, TRAX, Yuku DAO

Rip off the band-aid! :adhesive_bandage:

5 Likes

Awesome!
Rip off the band-aid! :adhesive_bandage:

Good news :partying_face:

The new AdvanceSnsTargetVersion proposals are now available on most SNSs.

The reduced amount of manual work that is now required to keep an SNS up to date will hopefully inspire the rest to sprint through the remaining upgrade steps to get this feature, as outlined above.

For example, check out the first such proposal that has already been adopted by the WaterNeuron SNS.

2 Likes

Hey @aterga. I’d like to get your feedback on the Advance SNS Target Version proposal that I submitted recently for the ALICE SNS. Proposal 11 passed and was executed, but only step 1 was completed. Starting at step 2 in the payload, the version of the canisters were not updated. It’s been over a week. If there are any errors that happened, then I don’t know where to go to find them. I just know that the current hash does not match the hash shown in the payload for any of the canisters except Root. I noticed this a few hours after the proposal executed, but now a week later it still hasn’t changed. Of course, now there is another SNS update that needs to happen, but the first one didn’t complete. Do you know why this would occur? What needs to happen to move forward? Should I just submit the proposal again or do you need time to investigate?

For reference, I used the toolkit to submit ALICE proposal 11 and WTN proposal 2045 consecutively. Here are links to the canisters for the ALICE SNS and WaterNeuron SNS for easy access. The WTN proposal 2045 passed and was fully executed. All canisters were updated according to the upgrades steps in the payload of the proposal.

By the way, it’s a little clunky to have to click on each canister in order to find the current module hash for each canister in the SNS. Is there any way the dashboard can be modified to display a table of canisters and current module hash so this can be verified at a glance? If there is an existing solution that shows all SNS canisters and current module hash then please advise where I can find it. Perhaps this is a question for @Dylan or others on the dashboard team.

cc @rem.codes @bobdotfun

Hi @wpb, thanks for reporting this issue.

Proposal 11 passed and was executed, but only step 1 was completed.

Summary. This proposal worked as intended, simply setting the target version for Alice SNS. However, reaching that version required a sequence of upgrades involving an upgrade of the SNS Governance canister, which has failed due to an inconsistency detected in the SNS parameters. This inconsistency had been introduced by an earlier proposal: Increase Ledger Fee from 1 ALICE to 5 ALICE, making the ledger fee greater than the 3 ALICE minimum neuron stake.

Recommendation.

  1. Change the SNS parameters to satisfy the constraint neuron_minimum_stake_e8s > transaction_fee_e8s (note the strict inequality). This could be done, e.g., by sending out another Manage Nervous System Parameters proposal with neuron_minimum_stake_e8s: Some(600000000).
  2. After that gets executed, submit another Advance SNS Target Version proposal, since the target version has been unset after the (2nd) upgrade step failure.

Details

Form Alice SNS Root’s canister logs:

{
  "file": "rs/sns/root/canister/canister.rs",
  "line": 245,
  "severity": "Error",
  "timestamp": 1738570002 // Mon Feb 03 2025 08:06:42 GMT+0000
  "message": "change_canister: Canister change failed: Attempt to call install_code with request ChangeCanisterRequest { ... } failed with code CanisterError:
    Error from Canister oa5dz-haaaa-aaaaq-aaegq-cai:
      Canister called `ic0.trap` with message:
        Panicked at
          'Cannot start canister, because the deserialized GovernanceProto is invalid in some way:
            \"NervousSystemParameters.neuron_minimum_stake_e8s (300000000) must be greater than NervousSystemParameters.transaction_fee_e8s (300000000)\"',
    rs/sns/governance/canister/canister.rs:218:69..."
}

When Alice was created, transaction_fee_e8s was set to 1 ALICE, and then it was raised to 5 ALICE in https://dashboard.internetcomputer.org/sns/oh4fn-kyaaa-aaaaq-aaega-cai/proposal/9. Please don’t be confused by the value 300000000 appearing twice in the log event, the error text simply contains a typo :

if neuron_minimum_stake_e8s <= transaction_fee_e8s {
    Err(format!(
        "NervousSystemParameters.neuron_minimum_stake_e8s ({}) must be greater than \
        NervousSystemParameters.transaction_fee_e8s ({})",
        neuron_minimum_stake_e8s, neuron_minimum_stake_e8s
    ))
}

should be

if neuron_minimum_stake_e8s <= transaction_fee_e8s {
    Err(format!(
        "NervousSystemParameters.neuron_minimum_stake_e8s ({}) must be greater than \
        NervousSystemParameters.transaction_fee_e8s ({})",
        neuron_minimum_stake_e8s, transaction_fee_e8s
    ))
}

So, neuron_minimum_stake_e8s is currently below transaction_fee_e8s, which violates the SNS Governance rules. This isn’t a new rule — it has been enforced for all SNS launches since 2022. However, there wasn’t a strong enough check in place to prevent the SNS from changing its individual parameters to reach a state that is deemed illegal by this validation; we’ll look into improving this in one of the upcoming SNS upgrades.

2 Likes

It’s not part of the Dashboard, but each SNS has an upgrade journal endpoint (linked to from Advance SNS Target Version proposals). This endpoint contains perhaps too much data for regular users, but power users can read the current SNS version and the pending ones (if any) at a glance, including each SNS framework canister’s module hash.

For example, WaterNeuron’s upgrade journal has the following two events at the end:

  {
    "timestamp_seconds": 1738792072,
    "event": {
      "UpgradeStarted": {
        "current_version": {
          "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
          "governance_wasm_hash": "5c43913c77f922a21f54b3422abf7cb43d369677e7668b7c8f91a429acd5c864",
          "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
          "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
          "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
          "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
        },
        "expected_version": {
          "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
          "governance_wasm_hash": "5c43913c77f922a21f54b3422abf7cb43d369677e7668b7c8f91a429acd5c864",
          "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
          "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
          "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
          "archive_wasm_hash": "2b0970a84976bc2eb9591b68d44501566937994fa5594972f8aac9c8b058672f"
        },
        "reason": {
          "BehindTargetVersion": {}
}}}},
  {
    "timestamp_seconds": 1738792095,
    "event": {
      "UpgradeOutcome": {
        "human_readable": "Upgrade marked successful at 2025-02-05 21:48:15 UTC.",
        "status": {
          "Success": {}
}}}}

So, e.g., the last expected_version to which there was UpgradeOutcome.status Success is the current version.

In contrast, the last event in Alice SNS’s upgrade journal is:

  {
    "timestamp_seconds": 1738608947,
    "event": {
      "UpgradeStepsRefreshed": {
        "upgrade_steps": {
          "versions": [
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "bd936ef6bb878df87856a0b0c46034a242a88b7f1eeff5439daf6278febca6b7",
              "swap_wasm_hash": "8313ac22d2ef0a0c1290a85b47f235cfa24ca2c96d095b8dbed5502483b9cd18",
              "index_wasm_hash": "2adc74fe5667f26ea4c4006309d99b1dfa71787aa43a5c168cb08ec725677996",
              "ledger_wasm_hash": "25071c2c55ad4571293e00d8e277f442aec7aed88109743ac52df3125209ff45",
              "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "5e0547725b033e689f6c7381ca9555fbf09e5ed884178c8286b72e6b2c09e863",
              "swap_wasm_hash": "8313ac22d2ef0a0c1290a85b47f235cfa24ca2c96d095b8dbed5502483b9cd18",
              "index_wasm_hash": "2adc74fe5667f26ea4c4006309d99b1dfa71787aa43a5c168cb08ec725677996",
              "ledger_wasm_hash": "25071c2c55ad4571293e00d8e277f442aec7aed88109743ac52df3125209ff45",
              "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "5e0547725b033e689f6c7381ca9555fbf09e5ed884178c8286b72e6b2c09e863",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "2adc74fe5667f26ea4c4006309d99b1dfa71787aa43a5c168cb08ec725677996",
              "ledger_wasm_hash": "25071c2c55ad4571293e00d8e277f442aec7aed88109743ac52df3125209ff45",
              "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "5c43913c77f922a21f54b3422abf7cb43d369677e7668b7c8f91a429acd5c864",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "2adc74fe5667f26ea4c4006309d99b1dfa71787aa43a5c168cb08ec725677996",
              "ledger_wasm_hash": "25071c2c55ad4571293e00d8e277f442aec7aed88109743ac52df3125209ff45",
              "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "5c43913c77f922a21f54b3422abf7cb43d369677e7668b7c8f91a429acd5c864",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
              "ledger_wasm_hash": "25071c2c55ad4571293e00d8e277f442aec7aed88109743ac52df3125209ff45",
              "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "5c43913c77f922a21f54b3422abf7cb43d369677e7668b7c8f91a429acd5c864",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
              "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
              "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "5c43913c77f922a21f54b3422abf7cb43d369677e7668b7c8f91a429acd5c864",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
              "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
              "archive_wasm_hash": "2b0970a84976bc2eb9591b68d44501566937994fa5594972f8aac9c8b058672f"
            },
            {
              "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
              "governance_wasm_hash": "487940ab827c929471dbd128fcb1c7d5dfe2f3f56d9e489e0f3b68122b56a94f",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
              "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
              "archive_wasm_hash": "2b0970a84976bc2eb9591b68d44501566937994fa5594972f8aac9c8b058672f"
            },
            {
              "root_wasm_hash": "9c129437f868e54a2dc3c72438cd7977925d089f9982c35d0370eaa71d5e3a6d",
              "governance_wasm_hash": "487940ab827c929471dbd128fcb1c7d5dfe2f3f56d9e489e0f3b68122b56a94f",
              "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
              "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
              "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
              "archive_wasm_hash": "2b0970a84976bc2eb9591b68d44501566937994fa5594972f8aac9c8b058672f"
}]}}}}

This indicates that Alice SNS is currently at version

{
    "root_wasm_hash": "dc243135057d13c48f71d2f0a4b8f5fc43ed525d579d97dde23e052dca15bf96",
    "governance_wasm_hash": "bd936ef6bb878df87856a0b0c46034a242a88b7f1eeff5439daf6278febca6b7",
    "swap_wasm_hash": "8313ac22d2ef0a0c1290a85b47f235cfa24ca2c96d095b8dbed5502483b9cd18",
    "index_wasm_hash": "2adc74fe5667f26ea4c4006309d99b1dfa71787aa43a5c168cb08ec725677996",
    "ledger_wasm_hash": "25071c2c55ad4571293e00d8e277f442aec7aed88109743ac52df3125209ff45",
    "archive_wasm_hash": "f94cf1db965b7042197e5894fef54f5f413bb2ebc607ff0fb59c9d4dfd3babea"
}

While the latest available version (blessed by the NNS) is 8 steps ahead:

{
    "root_wasm_hash": "9c129437f868e54a2dc3c72438cd7977925d089f9982c35d0370eaa71d5e3a6d",
    "governance_wasm_hash": "487940ab827c929471dbd128fcb1c7d5dfe2f3f56d9e489e0f3b68122b56a94f",
    "swap_wasm_hash": "45408ed654561dfb17c84b86948dda9498aa0ba8ee669ae774e5faca830c4c24",
    "index_wasm_hash": "e155db9d06b6147ece4f9defe599844f132a7db21693265671aa6ac60912935f",
    "ledger_wasm_hash": "3b03d1bb1145edbcd11101ab2788517bc0f427c3bd7b342b9e3e7f42e29d5822",
    "archive_wasm_hash": "2b0970a84976bc2eb9591b68d44501566937994fa5594972f8aac9c8b058672f"
}
1 Like

@aterga thank you so much for your feedback on this issue. The recommendation is clear. I’d also like to see the ALICE SNS opt in to the automatic SNS updates as you announced here. Should the Manage Nervous System Parameters proposal in step 1 above also include this opt in or does that need to occur after the Advance SNS Target Version is caught up?

@peterparker and @rem.codes do either of you plan to implement the ManageNervousSystemParameters proposal type into proposals.network or the dev.ic-toolikit.app? What is your timeline? It would be nice if this proposal type could be submitted through your app.

@bobdotfun tagging you on this string of messages because it includes a description of the issues and proposed solutions for the ALICE SNS upgrades.

I don’t plan to add any other types of proposals to proposals.network currently, as I don’t have the need myself. PRs are always welcome, of course. :wink:

1 Like

Maybe the Alice SNS community would prefer to make separate decisions for the two conceptually orthogonal changes (min. neuron stake and ledger fee vs. automatic target version advancement). So splitting these changes into two independent proposals seems like a cleaner way approach.

But making both changes in just one proposal should be fine technically.

2 Likes

@wpb Are you aware that each row of the SNS Canisters table can be expanded to show the module hash? Or perhaps this is what you were describing as clunky. We don’t include the module hash in the non-expanded rows because it’s long and takes up too much space.

1 Like

No, I wasn’t aware of that feature. That’s awesome and helps a lot. Thank you for pointing it out.

1 Like