Thanks @dsharifi, this explanation seems largely consistent with my understanding. For anyone looking for more info, @Manu (the handsome chap that he is) did a great video years ago, providing an overview of how the conensus mechanisms slot together. Other details are scattered around docs and source code comments.
@dsharifi, in your overview above you’ve explained why the notorisarion delay (ND) exits, but not why an appropriate value should now be considered independent of subnet size.
If this is the case, could you describe the innovation that has made this so? All large subnets currently have an ND of 1000ms precisely because the statement that you’re making would previously have been regarded as inaccurate (or misleading).
My understanding is that an appropriate setting for ND is dependent on subnet size, but I suspect that recent performance improvement makes the difference marginal to the point of not being significant (such that configuration conistency is preferred over hyper-optimising).
This is my assumption, but the reality hasn’t been made clear, or justified in a way that can be sensibly reasoned about.
I’d like to reiterate my other question too. I haven’t seen an answer for this yet either
Yes, you are right that a recent change has allowed us to lower the ND. The major reason is due to the new P2P layer that was rolled out to all subnets earlier this year. The new P2P layer builds on top of Quic and is highly concurrent, which allows nodes to deliver artifacts with a much smaller latency than before. This new P2P layer also scales better with larger subnets, which is why we want to unify the ND across all subnets to 300ms.
We went with the Internet Identity subnet as it only has 5 canisters installed, leading to lower overheads for block processing. However, any of the other subnets could also have been valid candidates. Keep in mind that we are planning to propose a gradual, and we do intend to propose to change the ND for the fiduciary, SNS and NNS subnet as well.
Voted to adopt proposal 134042, as the reasoning is sound and the proposal description matches the payload.
This proposal is intended to replace one node with another, in order to gain insights into the stability of nodes of the node operator of the new node, given that this node operator does not yet have nodes in any subnet. As seen in the proposal (which I verified using the DRE tool), the proposed change leaves the target topology within the requirements and improves decentralisation with respect to country.
TDLR: As a matter of principle, I try to maintain a policy of rejecting proposals that objectively misrepresent the technical behaviour of the proposal. If proposal summary accuracy is not a strict requirement, this leaves wiggle room to mislead (now, and in the future). I have therefore rejected this proposal.
… a node operator 7mdax currently does not have nodes in any subnet. To get insights into the stability of the nodes of this node operator, we propose to add one of the operator’s nodes to subnet uzr34
Claims made in the proposal summary can be validated via the IC API. Reviewing the returned JSON shows that the node operator 7mdax operates a single node, which is indeed currently unassigned.
An UP node in South Africa is removed and replaced with this new Latvian 7mdax node. As can be seen in the map below, this reduces decentralisation in terms of geographic and continental clustering. However these metrics are not part of the formal IC Target Topology. Country diversity is a metric that is improved by this proposal, and this is indeed a metric that forms part of the IC Target Topology. The general claim made that decentralisation ‘get’s better’ is therefore fair.
However, at least one part of the proposal is inaccurate. →
the number of different NP actors increases from 30 to 31
As displayed in the subnet characteristic counts in the Decentralisation Stats below, the number of node providers and node operators is 31, and remains 31 after this proposal has executed.
I believe proposal summary accuracy is extremely important, along with consistent voting principals that are easy for followers to understand (leaving little room for undefined voting behaviour). I have therefore rejected this proposal. It’s not my intention to block the intentions of this proposal, and I’d be happy to accept a replacement proposal with a summary that is technically accurate. I also apologise if I’ve misunderstood any of these details.
Decentralisation Stats
Subnet node distance stats (distance between any 2 nodes in the subnet) →
Smallest Distance
Average Distance
Largest Distance
EXISTING
0 km
7898.216 km
19448.574 km
PROPOSED
0 km (NaN%)
7666.342 km (-2.9%)
19448.574 km
This proposal slightly reduces decentralisation, considered purely in terms of geographic distance (and therefore there’s a slight theoretical reduction in localised disaster resilience).
Subnet characteristic counts →
Continents
Countries
Data Centers
Owners
Node Providers
Node Operator
EXISTING
5
24
31
31
31
31
PROPOSED
5
25 (+4%)
31
31
31
31
This proposal slightly improves decentralisation in terms of jurisdiction diversity.
Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) →
Continent
Country
Data Center
Owner
Node Provider
Node Operator
EXISTING
10
3
1
1
1
1
PROPOSED
11 (+10%)
3
1
1
1
1
Decentralisation in terms of distribution between continents is reduced (however this is not a formal part of the IC Target Topology).
The above subnet information is illustrated below, followed by a node reference table:
Map Description
Red marker represents a removed node (transparent center for overlap visibility)
Green marker represents an added node
Blue marker represents an unchanged node
Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)
Known Neurons to follow if you're too busy to keep on top of things like this
If you found this analysis helpful and would like to follow the vote of the LORIMER known neuron in the future, consider configuring LORIMER as a followee for the Subnet Management topic.
Another good neuron to follow is Synapse (follows the LORIMER and CodeGov known neurons for Subnet Management, and is a generally well informed known neuron to follow on numerous other topics)
I think this part of the DRE tool output actually splits the process into two parts - removing one node and then adding another - rather than just treating the whole swap as one step, so the first step reduces NP actors from 31 to 30 and the second step (this one) increases them from 30 to 31. I don’t think it’s the best way to present the information but that would be my interpretation of it.
xav3a-kdo3a-2rgbg-o6vnk-clat5-bcc7w-vmnej-z55rx-mfx26-7xugo-tqe [health: healthy, >impact on decentralization: (gets better) the average log2 of Nakamoto Coefficients across all >features increases from 3.2044 to 3.2845]
Nodes added:
poyg5-cbmcm-a372x-j72lt-zm4rz-4sf5v-ajtle-hyqkt-rdgaz-eqzmi-5qe [health: healthy, impact >on decentralization: (gets better) the number of different NP actors increases from 30 to 31]
I think the fact that the ‘nodes removed’ part doesn’t also state that the NP actors decreases from 31 to 30 makes the proposal summary difficult to follow and interpret correctly.
That being said, there’s a lot in the proposal summary, and I really appreciate that it provides voters with this sort of info (which still needs validating of course). If I hadn’t been so eager on the reject, I think I’d probably adopt this now based on your analysis.
@sat do you think it would take much to tweak the tool to describe the overall outcome, rather than the outcome at each step? (not to sound ungrateful for the awesome work that has already gone into improving the proposal summaries)
The tool already describes the final outcome - there is no change in the decentralization. For some reason I thought it would be useful to know the effect on decentralization for every step in the calculation. But I don’t see that anymore so I’ll probably just drop the intermediate steps and just share the node health (the way the tooling sees it) at the time of proposal submission.
Thanks for the pedantic and useful reviews and feedback @timk11 and @Lorimer - great work as always!
The proposal 134042 sadly failed to execute.
The new code for ensuring at least one node is used per operator had a bug in that it considered all healthy nodes, which ignores the nodes from open proposals.
The bug should be fixed in
Voted to adopt proposal 134042. The proposal replaces one node from subnet uzr34:
Removed Node: xav3a, Added Node: poyg5.
The proposal was verified using the DRE tool to verify the metrics stated and there was no impact on decentralization. The motivation behind this proposal is to get insights into the stability of the nodes operated by the node operator 7mdax. Using the ic-admin tool as follows:
I was able to verify the node_provider_principal_id, 3oqw6, dc_id rg1 which matches with the one from the proposal and currently has the only one node under it’s control.
The proposal makes sense in that it replaces one healthy node from Africa xav3a , with Riga1, LT node poyg5 while not afecting decentralization in order “To get insights into the stability of the nodes of this node operator”.
As a side note at time of writing this post the proposal already failed as explained by Sasha. I voted one day earlier already to adopt.
Voted to adopt proposal 134174, as the reasoning is sound and the description matches the payload. This proposal is intended to replace 1 dead node, mswad and 1 healthy node. Node mswad appears as “Status: Active” on the IC dashboard, but is shown in the Node Provider Rewards tool as having had intermittent failures over the past few weeks.
The proposal is a bit tricky one in sense in that it replaces one active node from Warszawa 2, PL mswad with the Active status and a small percentage of failed blocks for the last week as per NPR dashboard at the time of review so it is safe to say it’s degraded since majority of failed blocks were in October? (not so sure what the timeline is for this) and the xav3a node from Africa with qlk52 from Riga1,LV and respectively g4avo with `Awaiting’ status from the capital of Romania, with an overall improvement.
replacing node xav3a-kdo3a-2rgbg-o6vnk-clat5-bcc7w-vmnej-z55rx-mfx26-7xugo-tqe to optimize network topology
Decentralisation Stats
Subnet node distance stats (distance between any 2 nodes in the subnet) →
Smallest Distance
Average Distance
Largest Distance
EXISTING
0 km
7898.216 km
19448.574 km
PROPOSED
0 km (NaN%)
7673.28 km (-2.8%)
19448.574 km
This proposal slightly reduces decentralisation, considered purely in terms of geographic distance (and therefore there’s a slight theoretical reduction in localised disaster resilience).
Subnet characteristic counts →
Continents
Countries
Data Centers
Owners
Node Providers
Node Operator
EXISTING
5
24
31
31
31
31
PROPOSED
5
25 (+4%)
31
31
31
31
This proposal slightly improves decentralisation in terms of jurisdiction diversity.
Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) →
The above subnet information is illustrated below, followed by a node reference table:
Map Description
Red marker represents a removed node (transparent center for overlap visibility)
Green marker represents an added node
Blue marker represents an unchanged node
Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)
Known Neurons to follow if you're too busy to keep on top of things like this
If you found this analysis helpful and would like to follow the vote of the LORIMER known neuron in the future, consider configuring LORIMER as a followee for the Subnet Management topic.
Another good neuron to follow is Synapse (follows the LORIMER and CodeGov known neurons for Subnet Management, and is a generally well informed known neuron to follow on numerous other topics)
Although not a concerning Average Failure Rate, there is a day specifically 23rd of October where the Failure Rate hit 71%. I’m not opposed to replacing the node with a more stable one.
xav3a, Dashboard Status Active, replacement improves decentralization on the Area metric (reduces number of nodes in Gauteng from 2 to 1) and the Country metric (reduces number of nodes in ZA from 3 to 2).
Added Nodes: qlk52 and g4avo.
The proposal was verified using the DRE tool to verify the metrics stated and that they were in line with the target topology requirements.
The motivation makes sense as explained by Rüdiger here and the proposer neuron 17511507705568200227 is the same as for other Boundary Nodes related topics.
Since this is a Subnet Management topic related to this uzr34 subnet, I thought it would be more appropriate to keep our Votes post in this thread instead of spamming the official Boundary Node thread.
TLDR: I’m planning to reject. This proposal is asking the community to make unnecessary trust assumptions, allowing anyone with control over the 2igsz-4cjfz-unvfj-s4d3u-ftcdb-6ibug-em6tf-nzm2h-6igks-spdus-rqe principal to bypass the NNS and directly deploy anything to the II subnet.
Allow a principal controlled by the boundary node team to deploy canisters on the uzr34 subnet, which is required to install the rate-limiting and salt sharing canister.
A malicious actor who gains (or has) control over that principal could use this opportunity to action a DOS attack by consuming subnet resources (an expense would be incurred by the attacker - but not an impractically large one). There’s no way to verify the code that will be deployed to this critical subnet.
I trust DFINITY and therefore the 2igsz-4cjfz-unvfj-s4d3u-ftcdb-6ibug-em6tf-nzm2h-6igks-spdus-rqe that was announced. However, I do no believe that this trust is something that should be asked of the community (otherwise what’s the NNS for in the first place).
Presumably the missing verifiability functionality has not been prioritised because voters have been willing to offer their trust for matters such as this so far. By rejecting this proposal I hope to keep visibility on the lack of this functionality.
This proposal authorises a Dfinity principal to deploy canisters on this subnet, with the intention of deploying a rate-limiting canister as part of the incident handling mechanism discussed here and then revoking this authorisation through a separate proposal.
My decision on this was a careful weigh-up given the arguments that have already been made. While a decentralised approach to deploying the new canister would be desirable, the tooling to enable this is not yet in place. This proposal asks us to trust the boundary node team to deploy the canister(s) as planned and then to follow this up by supporting a proposal to revoke their access. This is in keeping with the general approach approved by the community through proposal 134031, although noting that the proposal did not mention these specifics. To my mind, and specific to this case, the risks of trusting the team are less than the risks that the overall plan is intended to mitigate.
I also note that the boundary node team has requested the features needed to enable a decentralised canister deployment process and I would urge that this be prioritised. Questions: ● How big a job is this? ● Are there practical issues that might make this process difficult?
can be validated via the 
