This topic is intended to capture Subnet Management activities over time for the o3ow2 subnet, providing a place to ask questions and make observations about the management of this subnet.
At the time of creating this topic the current subnet configuration is as follows:
TLDR: Offline node in Switzerland replaced with one in the Netherlands. This looks good, however a few points Iād like some clarity on before voting (same as quoted below).
Decentralisation Stats
Subnet node distance stats (distance between any 2 nodes in the subnet) ā
Smallest Distance
Average Distance
Largest Distance
EXISTING
476.293 km
8783.597 km
19445.845 km
PROPOSED
173.657 km (-63.5%)
8790.029 km (+0.1%)
19445.845 km
This proposal locally decreases decentralisation while on average improves decentralisation, considered purely in terms of geographic distance.
Subnet characteristic counts ā
Continents
Countries
Data Centers
Owners
Node Providers
EXISTING
5
13
13
13
13
PROPOSED
5
13
13
13
13
Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) ā
The above subnet information is illustrated below, followed by a node reference table:
Map Description
Red marker represents a removed node (transparent center for overlap visibility)
Green marker represents an added node
Blue marker represents an unchanged node
Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)
Known Neurons to follow if you're too busy to keep on top of things like this
If you found this analysis helpful and would like to follow the vote of the LORIMER known neuron in the future, consider configuring LORIMER as a followee for the Subnet Management topic.
Other good neurons to follow:
CodeGov (will soon be committed to actively reviewing and voting on Subnet Management proposals based on those reviews)
WaterNeuron (the WaterNeuron DAO frequently discuss proposals like this in order to vote responsibly based on DAO consensus)
Itās expected that unassigned nodes run on a different version. Nodes will automatically upgrade to the subnetās version when they join the new subnet.
The upgrade path guarantees are +1/-1 (not strictly), and only for replica. In this case the node is not in a subnet so there is little state to upgrade / downgrade, so generally there is little incompatibility that could be expected.
Upgrading unassigned nodes is not something weāve seen failing in the past.
Thanks for clarifying @sat, this makes sense. Iāve held off my vote so far. This all sounds good, and Iām now more informed for future proposals . Iāll vote to adopt this shortly.
I gather that this is a potential danger, but itās just very rare, and I suppose only worth being concerned about if the unassigned nodes are very far behind the subnet.
Just adding this comment for future reference in case anyone stumbles across this thread and is looking for more info.
DFINITY will submit an NNS proposal today to reduce the notarization delay on the subnet, o3ow2, similar to what has happened on other subnets in recent weeks (you can find all details in this forum thread).
TLDR: Iāve voted to reject, given that thereās no means of verifying this one. Iām happy with the discussed way forward though (to include a reference to evidence in future proposals).
1 removed node in Romania replaced with a node in China. No significant impact on decentralisation. The reason for the swap is stated in the proposal, but I see no clear way to verify this.
@sat, @SvenF and/or @dmanu, how should one go about verifying the need for a proposal like this. It would be great if the proposal could link to discussion with the node provider, or something like that (to validate their intentions for āredeplyment with IPv4ā).
Decentralisation Stats
Subnet node distance stats (distance between any 2 nodes in the subnet) ā
Smallest Distance
Average Distance
Largest Distance
EXISTING
317.676 km
8783.593 km
19448.574 km
PROPOSED
317.676 km
9117.484 km (+3.8%)
19448.574 km
This proposal slightly increases decentralisation, considered purely in terms of geographic distance (and therefore thereās a slight theoretical increase in localised disaster resilience).
Subnet characteristic counts ā
Continents
Countries
Data Centers
Owners
Node Providers
EXISTING
5
13
13
13
13
PROPOSED
5
13
13
13
13
Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) ā
Continent
Country
Data Center
Owner
Node Provider
EXISTING
4
1
1
1
1
PROPOSED
5 (+25%)
1
1
1
1
Slightly worse situation in terms of clustering countries in a single continent. This isnāt an official IC Target Topology metric though.
The above subnet information is illustrated below, followed by a node reference table:
Map Description
Red marker represents a removed node (transparent center for overlap visibility)
Green marker represents an added node
Blue marker represents an unchanged node
Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)
Known Neurons to follow if you're too busy to keep on top of things like this
If you found this analysis helpful and would like to follow the vote of the LORIMER known neuron in the future, consider configuring LORIMER as a followee for the Subnet Management topic.
Other good neurons to follow:
Synapse (follows the LORIMER and CodeGov known neurons for Subnet Management, and is a generally well informed known neuron to follow on numerous other topics)
CodeGov (actively reviews and votes on Subnet Management proposals, and is well informed on numerous other technical topics)
WaterNeuron (the WaterNeuron DAO frequently discuss proposals like this in order to vote responsibly based on DAO consensus)
While the proposal is correct and it replaces a healthy node from Bucharest with the motivation āin order to be redeployed, in this case with ipv4ā based on āas per user requestā.
Couple of questions jump to mind:
What is this user request ? Is it the NP or users from the specific subnet ?
Where if anywhere can we see the request (obviously Dfinity wouldnāt just submit unfounded proposals but a forum post as a heads up would be nice)
Looking at the docs in Deployment and Updating nodes ipv4 we can see there are two ways of doing this, one of them does not require redeployment. So the question is will this node really be redeployed and if so will it loose itās place since it was replaced by what I am guessing is not temporary ?
Hi @ZackDS thanks for the questions, I guess replacing a node for maintenance is a ānew caseā that we have not discussed yet in terms of process, so good that you bring this up.
To give some background, all node machines currently support IPv6. All Gen1 NPs are requested to have at least 2 nodes deployed with IPv4, the reason being that is supports some features like http outcalls and bitcoin. But a node provider can only reinstall a node when it is no active in a subnet. If a node is active in a subnet, the node first has to be removed from that subnet and replaced by another node machine. So far, the DRE team is submitting these proposals (although in the future we want to instruct NPs to do this themselves).
Yes, the Ops team from Dfinity requested the NP to redeploy IPv4 on these two nodes, and the NP subsequently asked the DRE team to remove one of the nodes that was in a subnet to be removed.
So far, the requests are done directly to the Dfinity OPS team (e.g. through the Element channel we have for NPs). What we can do for example is ask each NP to add a short forum post in the subnet management thread each time they request to replace the node. Let me know if you think this would work.
All Gen1 nodes are deployed with HSM key, the feature to deploy a node without HSM using dfx was only added recently. So this basically means there is no other way than first to remove the node from a subnet.
@ZackDS@timk11 let me know what you think would be the best process for validating these redeployment requests. I am open to either reject this proposal and to resubmit it with e.g. a forum post by the NP, or we adopt this proposal and follow the new process the next time a replacement requests comes along.
Thank you for the very detailed information. After this update I will vote to adopt the proposal, since it is all clear it makes no sense to reject at this point.
For future I think we can all agree a short forum post from the NP would suffice.
This proposal replaces node wjx4k as per a request from the node provider. Decentralisation parameters are unchanged and remain within the requirements of the target topology. I agree that a forum post from the node provider acknowledging the request would be the best way to verify this. I did have a look in the IC Node Providers Element channel but I couldnāt see where this request was made. However, Iām happy to go with your explanation that the request was made directly to the team.
Voted to adopt proposal 133459. The proposal aims to replace the node wjx4k on subnet o3ow2 with node za7cm. The node provider of node wjx4k has requested for this to be removed for redeplyment with IPv4 and this has been confirmed by @SvenF.
In terms of moving forward with this type of proposals I do think it would be easier if the NP makes a forum post explaining the situation.
. Iāll vote to adopt this shortly.
