# Upgrade the Governance Canister to Commit 7e25f22
__Proposer__: jason.zhu at dfinity.org
__Source code__: [7e25f225129bbe6e44191b580b387362864145ee][new-commit]
[new-commit]: https://github.com/dfinity/ic/tree/7e25f225129bbe6e44191b580b387362864145ee
## New Commits
```
$ git log --format="%C(auto) %h %s" fa149557c44967f621c03224460eb6639f5c03c6..7e25f225129bbe6e44191b580b387362864145ee -- ./rs/nns/governance ./rs/sns/init
7e25f22512 feat(nns): Adjust batch limit for moving neurons to stable memory (#2778)
fbbb395308 perf(nns): Add benchmarks for periodic tasks (#2729)
2f63d24ecb fix(nns): Avoid cloning heap_neurons to avoid performance penalty (#2726)
c9065698a9 feat(nns): Move inactive neurons in the heap to stable storage through a timer (#2643)
1d73be3778 refactor(nns): Make cast_vote_and_cascade_follow async (#2670)
49000055d3 fix: Try to address spurious `unresolved import` errors (#2668)
363e697195 refactor(nns): Move cast_vote_and_cascade_follow into Governance struct (#2600)
139b0f21c0 refactor(nns): Refactor voting as precursor for more scalability (#2528)
a920fc7c4f refactor(nns): Spin out sum_weighted_voting_power. (#2619)
```
## Current Version
__Current git hash__: fa149557c44967f621c03224460eb6639f5c03c6
__Current wasm hash__: fe417b2c69c36ecc2b86d8e1ec33e170a2b69fc8dc612124ec89bffba53b2438
## Verification
See the general instructions on [how to verify] proposals like this. A "quick
start" guide is provided here.
[how to verify]: https://github.com/dfinity/ic/tree/7e25f225129bbe6e44191b580b387362864145ee/rs/nervous_system/docs/proposal_verification.md
### WASM Verification
See ["Building the code"][prereqs] for prerequisites.
[prereqs]: https://github.com/dfinity/ic/tree/7e25f225129bbe6e44191b580b387362864145ee/README.adoc#building-the-code
```
# 1. Get a copy of the code.
git clone git@github.com:dfinity/ic.git
cd ic
# Or, if you already have a copy of the ic repo,
git fetch
git checkout 7e25f225129bbe6e44191b580b387362864145ee
# 2. Build canisters.
./ci/container/build-ic.sh -c
# 3. Fingerprint the result.
sha256sum ./artifacts/canisters/governance-canister.wasm.gz
```
This should match `wasm_module_hash` field of this proposal.
I have successfully built governance canister and validated the hash, confirming that all commit messages accurately describe the corresponding modifications in the code.
Hash Match: MATCH
Feedback: NONE
Proposer Check: MATCH
Commits
7e25f22512
Matches description of adjusting batch limit lower for moving neurons to stable memory.
fbbb395308
Benchmarks for new scenarios, such as list_ready_to_spawn_neuron_ids and list_neurons_ready_to_unstake_maturity, have been added for both heap and stable storage. These tests evaluate the performance of identifying neurons ready to spawn or unstake maturity.
2f63d24ecb
Instead of cloning neurons during iteration, the changes use Cow (copy-on-write) to minimize unnecessary data duplication by iterating with references where possible.
c9065698a9
A new scheduling system was introduced to manage seeding and neuron storage adjustments, allowing neurons to migrate between heap and stable memory based on activity and storage policies.
The NeuronStore now supports batch adjustments of neuron storage, using defined batch sizes and instruction limits to optimize storage management while ensuring system stability.
1d73be3778 make_proposal, register_vote, and cast_vote_and_cascade_follow are refactored to support asynchronous execution async
49000055d3
Matches description to resolve intermittent issues with cargo check complaining about the conditional exclusion of ic_nns_governance::storage::reset_stable_memory, the fix explicitly enables the test feature for the ic-nns-governance crate during tests.
363e697195
Integrates the cast_vote_and_cascade_follow function directly into the Governance struct.
139b0f21c0
Refactor the voting logic by moving the cast_vote_and_cascade_follow function into a new voting module and encapsulating its behavior within a state machine, ProposalVotingStateMachine.
a920fc7c4f
Refactor consolidates the calculation of voting rewards into a reusable function sum_weighted_voting_power within the proposals module.
7e25f22512 feat(nns):
Done by setting lower BATCH_SIZE_FOR_MOVING_NEURONS to 200 from 1000 and MAX_NUM_INSTRUCTIONS_PER_BATCH to 1B from 5, in the batch_adjust_neurons_storage function. fbbb395308 perf(nns):
Added benchmarks list_ready_to_spawn_neuron_ids and list_neurons_ready_to_unstake_maturity for both heap and stable neuron since tests run show improvments for instructions. 2f63d24ecb fix(nns):
The previously applied hotfix, allready reviewed using Cow to avoid unnecessarily cloning heap_neurons. c9065698a9 feat(nns):
Uses a timer (every hour or 5 seconds for batches) to move currently only the inactive neurons from the heap memory to stable storage. 1d73be3778 refactor(nns):
Sets make_proposal, register_vote, and the cast_vote_and_cascade_follow functions to async enabling to perform asynchronous operations. 49000055d3 fix:
Adds the ic-nns-governance crate as dependency when the test feature is enabled, to the toml file. 363e697195 refactor(nns):
The cast_vote_and_cascade_follow function is moved to the Governance struct, previously was in the VotingStateMachines struct, before setting it async in the future. 139b0f21c0 refactor(nns):
Another step towards allowing to scale voting with following that creates a state machine for voting and it uses up to one machine per proposal. a920fc7c4f refactor(nns):
The sum_weighted_voting_power function is used for calculating the total and per neuron weighted voting power across a set of proposals.
Reason:
Build successful and hashes match, commits look great and match the description. Found no issues.
Review:
[7e25f22512]: Reduces the batch limit for moving neurons to stable memory from 1000 to 200 neurons per batch and the maximum number of instructions allowed per batch from 5_000_000_000 to 1_000_000_000 in order to cope with finalization drops on the NNS subnet when the system had a high instruction load.
[fbbb395308]: Adds 4 new benchamrks list_ready_to_spawn_neuron_ids_heap, list_ready_to_spawn_neuron_ids_stable, list_neurons_ready_to_unstake_maturity_heap and `list_neurons_ready_to_unstake_maturity_stable. This was done due to a recent regression on listing neurons ready to unstake or spawn in order to prevent further ones. While running against changes from the PR #2726, it showed that the heap optimizations led to significant performance gains.
[2f63d24ecb]: Avoids unnecessary cloning of heap_neurons to improve performance. The key change in the commit is the switch from using iterators over cloned Neuron objects to iterators over Cow<Neuron> with Cow meaning (Clone on Write), more details on this type here. This is implemented in with_active_neurons_iter and with_active_neurons_iter_sections iterators where basically it allows to clone the data lazily only when mutation or ownership is required.
[c9065698a9]: Introduces a new mechanism to move inactive neurons from the heap memory to stable storage using a scheduled timer. The function target_storage_location is responsible for determining the location in which to store the neuron (currently only inactive neurons).
[1d73be3778]: Makes the function cast_vote_and_cascade_follow async and the functions make_proposal and register_vote that depend on it.
[49000055d3]: cargo check randomly complains of ic_nns_governance::storage::reset_stable_memory during testing of the ic-nns-governance crate. The fix was to add the following line to Cargo.toml: ic-nns-governance = { path = ".", features = ["test"] } ensuring that the crate is always built when the test feature is enabled.
[363e697195]: Moves the function cast_vote_and_cascade_follow into the Governance struct.
[139b0f21c0]: Refactors the voting process in the NNS governance module by creating a state machine to handle voting in order to improve scalability. Previously the function cast_vote_and_cascade_follow handled all the calculations but now it delegates this calculations to the state machine. Up to one ProposalVotingStateMachine is used for each proposal.
[a920fc7c4f]: Refactors the code and moves the logic for calculating the total and per neuron weighted voting power for a set of proposals into a new function sum_weighted_voting_power within the proposals module.
Vote: ADOPT
Reason: Build is successful and both code changes and hashes match.
7e25f22512 Reduced BATCH_SIZE_FOR_MOVING_NEURONS value from 1k to 200 and MAX_NUM_INSTRUCTIONS_PER_BATCH from 5 million to 1 million.
fbbb395308 Moved some logic to generate a subaccount from id into its own method named subaccount_from_id. Added 2 helper methods to be used in benchmarks, one is used to create mock neurons that can be spawned and the other one for neurons unstakeable maturity. Added benchmarks to track resource usage of neuron spawning and unstaking maturity for both heap and stable neurons.
2f63d24ecb Avoid unnecessary clones of heap neurons when iterating from the NeuronStore, to do this the callbacks passed to with_active_neurons_iter and with_active_neurons_iter_sections now takes in a Neuron wrapped into a clone on write smart pointer, which allows to return either a reference or an owned value with stable store neurons mapped as Cow:Owned and heap neurons as Borrowed
c9065698a9 Avoid repetition of the logic used to determine where a neuron should be fetched from by moving it to a dedicated method. Implemented a timer to move neurons from heap to stable storage. The timer calls a new method named batch_adjust_neurons_storage, this attempts to move up to BATCH_SIZE_FOR_MOVING_NEURONS neurons from heap to stable storage starting from a specified neurons id, when it is done it returns the last neuron id of the moved batch. The method also keeps track of instructions used during execution and if more than MAX_NUM_INSTRUCTIONS_PER_BATCH are used then it returns early. When there are no more neurons left to move None variant is returned. The timer runs every 5 seconds while there are still batches left to process, in case there aren’t is scheduled to run every hour to check if new neurons have been added to the heap and if so it starts processing them.
1d73be3778 Modified cast_vote_and_cascade_follow method to be async and updated some code to reflect this change.
49000055d3 Explicitly define ic-nns-governance crate as a dev dependency when test feature is enabled.
363e697195 Moved cast_vote_and_cascade_follow from VotingStateMachines to Governance struct, its interface has also slightly changed as it is no longer needed to pass the neuron store and ballots since they are already available in the governance struct, ProposalId and NeuronId are also passed by value instead of as immutable references.
139b0f21c0 Implement state machines to handle proposal voting and followee updates, each proposal has its own state machine and their state is preserved in global state inside VotingStateMachines struct which contains a map of active state machines indexed by proposal id.
a920fc7c4f Moved logic to calculate the weighted total and per neuron voting power out of distribute_rewards into its own dedicated method.
