Proposal to add capabilities for emergency upgrades of governance canister via node owner/provider proposals

Some answers to your questions:

Can these emergency proposals only be submitted when the governance canister is down? Will this be enforced in code?

We pondered this but found no viable way to enforce this that would retain an operational advantage.

I’m also curious why only node providers that participate in the NNS subnet can submit these emergency proposals. Is it totally random which node providers happen to run nodes assigned to the NNS subnet? Or is there some special requirement / privilege?

Because this way the security assumptions are the exact same as the underlying system. I.e. we’re not giving more power than it already exists, just making it more explicit and auditable.

Also, what happens when the root canister goes down?

Good question. The root canister itself is controlled by another canister, the lifeline canister, which can upgrade it (which in turn is controlled by the root canister, forming a cycle). So if the root canister is broken, it can be upgraded by the lifeline, and vice versa: if the lifeline canister is broken it can be upgraded by the root canister. The only single point of failure atm is the governance canister.

4 Likes

Would it be possible to maintain a backup governance canister that runs alongside the primary one? Perhaps the backup canister maintains a known good state so that if the primary fails the node providers can vote to switch control to the backup canister. Then we can use the normal voting process to bring the system back to where it needs to be.

The problem here is that the backup governance canister would have to 1) have the exact same state as the main canister so that no transactions would be lost, which would imply additional messages and delays in anything that the governance canister does 2) if the governance canister has a bug, it’s likely that the backup would also have the same bug, thus not solving the problem.

If we have to put faith in the node providers during this sort of emergency I’m okay with that; but, I’d like their options to be very limited. Giving them the ability to propose and implement a change to the governance canister seems extremely risky.

In a sense we already do, this mechanism is just about making that more explicit and operationally viable and, more importantly, fully auditable.

2 Likes

I understand. I trust that Dfinity has done their due diligence exploring failover solutions before proposing further dependence on a small number of node operators.

I accept the reality of what you’re telling me. That being said, this proposal doesn’t really seem like it is up for community discussion. Even if we reject it today, it doesn’t change the fact that control of the NNS (essentially the entire IC) ultimately resides in the hands of a few NNS node providers. For that reason I see no point in voting against this proposal.

That being said, I would like to echo everyone’s request that Dfinity be more transparent about their plans for node deployment and data center on-boarding. Especially with regard to the NNS subnet. The only way I see us being able to secure the network is to ensure that the number of NNS node providers is significantly high enough to avoid collusion.

There are methods and technologies being developed to prevent third-party tampering of hardware/software. I trust Dfinity is looking into these solutions as well.

2 Likes

This is not my department but I do know that the general intention is to keep increasing this number more and more as time goes by.

One thing is for sure though, even at the current levels, many more parties would have to collude to maliciously change the IC than would for say bitcoin, if you consider mining pools.

6 Likes

Oh, in addition also consider that node providers are KYC’d entities, and thus prosecutable.

7 Likes

Today @dralves and @whizwang hosted a community conversation on this proposal. Thank you to all those who attended.

I will post the Youtube video as soon as it is live, but wanted to communicate the next steps:

  • 1-pager posted on the forum for review: September 10, 2021
  • Community Conversation with David Alves: September 16 2021
  • NNS Motion Proposal submission to approve design + project (Foundation will wait until the very end to vote to consider the early voters): Monday, October 11, 2021. 15:00 UTC
  • NNS Motion Proposal submission expires Wednesday, October 13, 2021. 15:00 UTC
  • If NNS Motion Proposal passes, implementation + deploy would take a week or so. Rough range October 18 - 25, 2021.
1 Like

Not strictly relates to the topic but are there more canisters in NNS?

I mean governance, ledger and registry canisters are well promoted due to their key role, but root and lifeline canisters are just mentioned on medium.

1 Like

@janosroden the NNS subnetwork currently hosts the following canisters:

  • Root canister (the controller of all the other canisters, which upgrades them)
  • Lifeline canister (the controller of the root canister, which upgrades it)
  • Registry canister (The canister that serves configuration to the internet computer)
  • Governance canister (The canister that implements governance)
  • Ledger canister (The canister that receives messages from users to send transactions)
  • Ledger archive canister (The canister that archives old ledger transaction blocks)
  • Cycles minting canister (The canister that mints cycles and sends them to canister on creation/top up)
  • Internet identity canister (The canister serving the Internet Identity dapp)
  • NNS dapp canister (The canister serving the NNS dapp)
9 Likes

Video of the community conversation with @dralves :

5 Likes

Aside @janosroden you can find interfaces and details for most of these canisters here: https://k7gat-daaaa-aaaae-qaahq-cai.ic0.app/dev/dfinity

3 Likes

Update:

I will submit an NNS motion proposal Monday October 11 (with expiration of Wednesday October 13).

  • the foundation will not vote until last hour or two so that it can absorb the feedback from the early voters as part of its own voting calculus.
  • if this goes through, then NNS team will submit a proposal to update the root canister
  • if it does not go through, back to the drawing board.
6 Likes

Update:

The NNS Motion proposal is live!

https://dashboard.internetcomputer.org/proposal/24807

2 Likes

Proposal passed! Thank you everyone.

@dralves will update the thread regarding deploying.

3 Likes

Was this ever deployed?

Good question. Yes it was.

You are right in that I dropped the ball in updating thread.

2 Likes