What happened in the “failed” SNS-1 proposal to upgrade the SNS-1 ledger canister?

TLDR;

  • The community agreed proposal to update the SNS-1 ledger appeared to fail when executed last week.
  • The ledger canister was in fact successfully upgraded. The ledger now exposes the candid API [Visible in the Internet Computer Dashboard].
  • It was marked as “failed” for a technical reason, explained in detail below.

Hi everyone,

Last week there was an SNS-1 proposal to upgrade the SNS-1 ledger canister to the latest version. This forum post explains why the proposal’s status was shown as “failed” after its execution even though the ledger canister was upgraded. We try to explain this on a high level, omitting some technical details.

How do upgrades work in SNS-1?
There is an NNS canister, called SNS-W, that stores which SNS canisters WASMs were approved by the NNS community. The idea is that SNSs can only be upgraded to such “pre-approved” WASM versions.

The SNS-1 governance canister organizes all upgrades for the SNS-1 canisters. It stores the currently deployed versions of the SNS-1 canisters. When an upgrade proposal is sent to SNS-1 governance, for example a proposal to upgrade the SNS-1 ledger last week, this has the following effects:

  1. SNS-1 governance compares the canister versions it knows to what is in SNS-W. If an upgrade is needed, there is one canister for which SNS-W has a newer version. In our case this was for the ledger canister.
  2. SNS-1 governance asks SNS root to upgrade the required canister, in our case the SNS-1 ledger canister, to the (new) version it just learned from SNS-W.
  3. SNS-1 governance waits a bit and then comes back to see if the actual canister versions match what it expects. In our case, this meant that SNS-1 checked for each SNS-1 canister whether it is still on the version from before the upgrade, except that for the SNS-1 ledger it checked that it is on the new upgraded version. If the versions match, the proposal is marked as “executed”, otherwise it is marked as “failed”.

What happened in the SNS-1 ledger upgrade case?
In the case of the SNS-1 ledger upgrade last week, the proposal was marked as “failed” even though the SNS-1 ledger canister was upgraded to the right version. While the SNS-1 ledger canister was running on the (new) expected version, another canister, namely the SNS-1 archive canister, did not have the version that SNS-1 governance expected when it did the checks in Step 3. Therefore SNS-1 governance set the proposal to “failed”.

Why did this happen?
As mentioned above, SNS-1 governance should always know which WASM versions all SNS-1 canisters currently run on. The underlying idea is that when it was initially deployed, SNS-1 governance was initialized with this information and since it orchestrates all upgrades of canisters, it can always update this information accurately.
However, there is one exception to this behavior: the archive canisters are spawned by the ledger canister to store the ledger transaction history forever. For this purpose each ledger canister has in its state the WASM of the archive and directly deploys it once it reaches a certain number of transactions. The SNS-1 ledger deployed the SNS-1 archive canister right after launch as it hit a certain number of transactions. However, SNS-1 ledger had a different archive WASM version than the version the SNS-1 governance canister expected. This went unnoticed until the upgrade last week, when the governance compared in Step 3 all the canisters deployed to what was expected, including the archive.

What do we propose to do?
In the short term, we propose to define a series of SNS-1 upgrade proposals to bring SNS-1 back on the upgrade path. This would make sure that SNS-W, SNS-1 governance, and SNS-1 ledger would all agree again on what WASM the archive and all other canisters should have.

The underlying problem is that in the current design there is no single source of truth for the archive canister’s version. Going forward, we plan to improve this and avoid similar cases for SNS-1 and other SNSs in the future.

I hope this helped to shed some light on what happened.
I now wish you all a happy end of this year and I am looking forward to seeing where SNS-1 takes us next year!

7 Likes