NNS Updates 2025-03-25, NNS Governance Security Hotfix

TL;DR

A security hotfix (135955) on the NNS Governance was applied a few hours ago. The source code is now published at commit cbebf5d2 and can be verified, which fixes a security issue. The procedure is done in accordance with the security patch policy defined in a previous motion proposal.

Context

The neuron management proposals can be used by a group of people to jointly manage a neuron. The managed neuron can execute commands according to proposals sent by its (up to 15) followees. Mostly because those proposals can only impact the target neurons (unlike other proposals that can impact the entire network), they are treated differently than other proposals in some ways: (1) their fee is 0.01 ICP, instead of 25 ICP and the proposer doesn’t get it back after adoption (2) the maximum number of such open proposals is 100K, rather than 200.

Problem

It is possible for such manage neuron proposals to occupy a large amount of space, by calling the neuron command MakeProposal which in turn contains another proposal that can be large. In a normal case, a 25 ICP fee might still be taken if the neuron management proposal is adopted, the managed neuron is instructed to make the proposal, and the managed neuron could lose the 25 ICP if the proposal is rejected. However, the proposer can make the inner proposal invalid or simply make sure that the managed neuron doesn’t have enough fee. In these cases, the proposals still occupy a lot of space but the proposer avoids having to pay the higher fees. If such an attack is carried out, only ~15 ICPs (~1500 proposals) are needed to occupy 3GiB of space (2 MiB per proposal), which can make the NNS Governance hit the 3.5GiB soft limit, and certain operations like creating new neurons and making certain proposals will be rejected by NNS Governance as it enters a degraded mode.

Fix 1: Command Validation

Since this is a security hotfix, we aim to make it as simple as possible. Before creating any neuron management proposal (which occupies the wasm memory space), the NNS Governance will apply more command validation (our analysis showed that those 4 are the only manage neuron commands with unbounded data).:

  1. check that the neuron management command isn’t MakeProposal/DisburseMaturity, effectively disabling them within neuron management proposals
  2. for Follow/Disburse, validate that the vec fields within them have reasonable size

Impact of Disabling Commands

  1. The DisburseMaturity functionality is not supported yet - if a client sends a DisburseMaturity command directly through the manage_neuron canister method (instead of a neuron management proposal), it will simply fail. However, the canister only performs validation when the neuron management command is about to be performed, which still makes it possible for such neuron management proposals to occupy space.

  2. In the last 2 years, only 3 neuron management proposals (134354, 134375) and 135853 were made where the command within them is another proposal.

  3. Those 2 proposals are 2 attempts at the same goal, so for all intents and purposes, the use case we are disabling only happened once in the last 2 years. Therefore it seems justified to take it away in a security hotfix.

  4. Those proposals are clearly not intended as attacks as mentioned above, since they are not large and there were only 2.

  5. We believe that the use case of sending a RegisterKnownNeuron proposal as part of a neuron management proposal is legitimate - while the proposer of a proposal is usually not very important, it does make more sense for the known neuron itself to send out the RegisterKnownNeuron proposal. We will discuss it below in more detail.

Fix 2: Lower # of Open Neuron Management Proposals

We propose to limit the total number of open neuron management proposals to 10K instead of 100K. Our analysis showed that in the last 2 years, there were roughly 7K neuron management proposals. Therefore we believe that this change should not cause inconveniences for the users.

Storage Limit After Hotfix

The 2 changes proposed above lowered the total storage that can be used by all the neuron management proposals, in 2 ways:

  1. The number of open proposals is changed from 100K to 10K
  2. The total size of a neuron management proposal changed from 2MiB (message size limit) to 35KiB

After the hotfix, the total storage that can be used by neuron management proposals become < 10K * 35KiB < 350MiB (from 100K * 2MiB ~ 200GiB). Note that the 35KiB is an over-estimation, where the dominant part is the 30KB proposal summary.

Next Steps

Validation on ManageNeuron Commands

In general, we need to consider validating neuron management proposals more carefully. We will propose more changes around validating ManageNeuron commands (currently, the validation logic is mixed with the execution logic) in future proposals.

Request for Feedback: Allow Certain Proposals in the future

Since not all proposals can be large (e.g. the RegisterKnownNeuron proposals have a clear bound), it’s possible to re-enable certain proposals to be made through neuron management proposals in the future, without sacrificing security:

  1. Allow certain types of proposals to be made within neuron management proposals
  2. Enforce a size limit of the proposals within the neuron management proposals

However, we believe that the current state of disallowing MakeProposal commands is also the best option going forward. For use cases like RegisterKnownNeuron, such proposals can still be made directly by the proposer (through the manage_neuron canister method), and the to-be known neuron can be instructed to vote on it (either through a neuron management proposal with RegisterVote, or through followee configuration)

7 Likes

You missed one :slightly_smiling_face: 135853

That makes 3. Now - who is this lorimer character, and should we be concerned… :wink:

This would be nice, please do. I think it’s a good way for a team to formally signal their agreement to move forward with something they’d collectively like to propose to the wider community.


Thanks for sharing these details. I hadn’t considered the potential for misuse as described here. I’m glad DFINITY is on the ball with these matters.

3 Likes

proposal - 135955 – Cyberowl | CodeGov

Checks

Vote: ADOPT
Hash Match: MATCH
Feedback: NONE
Proposer Check: MATCH
Target Canister: MATCH
Reason:
Build completed successfully, hashes are verified.

3 Likes

Proposal #135955 for Governance — Zack | CodeGov

Vote: Adopted

Reason: The build is reproducible and the wasm hash is a match.
The canister id rrkah-fqaaa-aaaaa-aaaaq-cai is correct and the install mode is upgrade.

Given the fact that this was a Hotfix, and after it was adopted with commit cbebf5d explained in details after execution, the check to see that manage neuron proposals are not too large, as they can take up lots of space, makes sense.

About CodeGov

CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron’s Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

3 Likes

Proposal 135955 - Zane | CodeGov

Vote: ADOPT
Reason: Build completes successfully, both hashes and reviewed commits match their descriptions.

cbebf5d2 Reduced MAX_NUMBER_OF_OPEN_MANAGE_NEURON_PROPOSALS from 100k to 10k. Added more validation in validate_manage_neuron_proposal to prevent abuse due to the cheap cost of such proposals. MakeProposal commands have been disabled, DisburseMaturity is not fully implemented yet and therefore doesn’t have validation for subaccount size, so it has also been disabled. Disburse now validates the ICP account address can be successfully converted to an account identifier, which means it is a valid address and doesn’t contain huge amount of data. Finally number of followees passed in Follow commands is checked to ensure it isn’t greater than MAX_FOLLOWEES_PER_TOPIC , i.e 15. If either of these checks fail an error is returned.

About CodeGov

CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron’s Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

2 Likes

Thanks for the feedback! I added 135853 to the original post for completeness (the post was written before the proposal was created).

This would be nice, please do. I think it’s a good way for a team to formally signal their agreement to move forward with something they’d collectively like to propose to the wider community.

Could you comment on whether you think it would be sufficient to have the to-be known neuron vote on it instead (using RegisterVote command)?

2 Likes

Can you share if this was an actual black hat attack on the network? If you don’t reply I will take that as a yes.

2 Likes

I guess so. It’d be a shame to lose the functionality for good, but it’s not of any real importance. It’s certainly not worth splitting hairs over, or sacrificing on security. I was actually surprised to see it was possible. The D-QUORUM co-founding team were all watching in anticipation to see if it would succeed. Awesome that it did.

I guess this will make D-QUORUM and CO.DELTA the only collaboratively-proposed known neurons. That’s kinda cool.

I’d keep an eye on the proposer. Definitely up to no good :disguised_face:

Proposal 135955 – LaCosta | CodeGov

Vote: ADOPT

Governance Canister

image
Reason:
Build successful and hashes match, commits look great and match the description. Found no issues.
This proposal is an Hotfix explained on the first post of the thread that disables the MakeProposal command on Manage Neurons Proposals to avoid this proposals occupying a lot of space which could put the NNS in a degraded mode.

[cbebf5d2d7]: Reduces the maximum number of open manage_neurons_proposals from 100K to 10K. Disables commands MakeProposal and DisburseMaturity. Adds a check to the Disburse command making sure that the ICP account address can be converted to a valid AccountIdentifier and therefore doesn’t contain large amounts of data. A check for the Follow command making sure that the number of followees isn’t greater than MAX_FOLLOWEES_PER_TOPIC = 15. Writes a test test_disallow_large_manage_neuron_proposals to validate the logic.

About CodeGov

CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron’s Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

2 Likes

Adopt proposal 135955

Proposal

  • Canister id rrkah-fqaaa-aaaaa-aaaaq-cai is indeed the governance canister.
  • The upgrade args correspond to the empty args.
  • The install mode is indeed upgrade.
  • The wasm hash is reproducible.

image

Code Review

  • Decrease MAX_NUMBER_OF_OPEN_MANAGE_NEURON_PROPOSALS, which corresponds to the number of open proposals for manage neuron, from 100k to 10k.
  • The total size of a neuron management proposal changed from 2mb to 35kb.