Unhalt subnet lhg73-sax6z-2zank-6oer2-575lz-zgbxx-ptudx-5korm-fy7we-kh4hl-pqe, after recovery and update ssh readonly access
This proposal has already executed, at 2024-12-27, 19:12:34 UTC (minutes after it was submitted several hours ago). As can be seen in the screenshot below, after stalling, halting, CUP, and unhalting, the subnet is back on track and processing blocks (with the condition improving since the unhalt proposal executed).
Hey @sat@SvenF. I’ve noticed over the last 2 days that ICP Swap has become very sluggish. I’ve been using it daily for the last month and have never noticed any major issues. It is always responsive and reasonable quick to complete my trading activities. However, over the last 2 days it has become unbearable. I know there was an increase in activity due to both Panda and Alice (and probably others), but I wouldn’t expect ICP Swap to get bogged down because of that increase in volume. At other times of increased activity is seems to have responded just fine. I’m actually wondering if the performance could be related to how the nodes are distributed and the fact that the state is higher than 400 GiB. The application subnet that hosts ICP Swap is lhg73.
As can be seen in the screen capture, there is only one node in the US for this subnet. I’m in the Houston, Texas area (pink arrow) and the one node is in the Allentown, Pennsylvania data center. This strikes me as odd that there would be only one data center in the US for the subnet that hosts ICP Swap given that a lot of people from the US are likely heavy users of the app.
It seems like the rest of the world has reasonable node density, but North America seems sparse. In the European region, there are 5 nodes while in North America there is 1 node. I understand the desire to have nodes in different countries, but it doesn’t quite seem appropriate if that causes a performance issue in parts of the world where there is a high density of ICP users.
Anyway, I’m curious if you think the lack of nodes in this part of the world, or even the 525 GiB state, could be contributing to the performance issues that I’m observing with ICP Swap?
IIUC, ICP Swap is running on multiple subnets, and it needs all of them to be in great shape to perform well.
I don’t have complete information, but from what I know they use k44fs for their pool and pjljw for their assets.
They also have a separate canister for every liquidity pool. So their architecture certainly can be streamlined and optimized. I believe they are already aware that we see this as a problem, but feel free to reach out to them again and ask if there are any improvements they can make.
I’m sharing here lhg73 subnet metrics from 4 weeks ago:
So performance of that subnet doesn’t seem to have changed much. It would still be worth chatting with them to see if they could improve performance on their side.
This proposal replaces 1 node in subnet lhg73, appearing in the decentralization tool as “DOWN”. As shown in the proposal, decentralisation parameters are unchanged and remain within the requirements of the target topology.
About CodeGov
CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neurons’ Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralisation of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.
Learn more about CodeGov and its mission at codegov.org.
TLDR: One offline node replaced with an unassigned node. IC Target Topology metrics remain unchanged, but the average distance between nodes increases slightly.
Country Discrepancies (1)
Minor discrepancy in terms of distance (within a margin of error)
Subnet node distance stats (distance between any 2 nodes in the subnet) →
Smallest Distance
Average Distance
Largest Distance
EXISTING
375.992 km
8504.338 km
19325.937 km
PROPOSED
375.992 km
8638.482 km (+1.6%)
19325.937 km
This proposal slightly increases decentralisation, considered purely in terms of geographic distance (and therefore there’s a slight theoretical increase in localised disaster resilience).
Subnet characteristic counts →
Continents
Countries
Data Centers
Owners
Node Providers
Node Operator
EXISTING
6
13
13
13
13
13
PROPOSED
6
13
13
13
13
13
Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) →
The above subnet information is illustrated below, followed by a node reference table:
Map Description
Red marker represents a removed node (transparent center for overlap visibility)
Green marker represents an added node
Blue marker represents an unchanged node
Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)
Black dotted line connects to a small black marker that shows where the IP address indicates the node is located (according to ipinfo.io). This is only displayed if it conflicts with where IC records indicate the node is located. See Country Discrepancies section above for more info.
You may wish to follow the CO.DELTA known neuron (coming soon) if you found this analysis helpful.
CO.DELTA △
We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:
Look: We observe the details and context of NNS proposals
Test: We test and verify the claims made by those proposals
Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.
Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.
Note that this analysis involved data provided by the IC-API, which is not open source. I’m in the process of switching over to more verifiable sources of this sort of information for future proposal reviews. See here for related discussion.
Current Nakamoto Coefficients and Topology, avg = 5.00
Attribute
Nakamoto Coefficient
Identical attribute values
Max allowed identical values
Unique Counts
Country
5
1
2
13
City
5
1
1
13
Data Center
5
1
1
13
Data Center Owner
5
1
1
13
Node Provider ID
5
1
1
13
Proposed Nakamoto Coefficients and Topology, avg = 5.00
Attribute
Nakamoto Coefficient
Identical attribute values
Max allowed identical values
Unique Counts
Country
5
1
2
13
City
5
1
1
13
Data Center
5
1
1
13
Data Center Owner
5
1
1
13
Node Provider ID
5
1
1
13
You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.
CO.DELTA
We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:
Look: We observe the details and context of NNS proposals.
Test: We test and verify the claims made by those proposals.
Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.
Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.
TLDR:
The proposal replaces offline nodes in Allentown (North America). The decentralization stats remain the same.
Warning
The last few proposals that address the outage of DC is offloading all nodes to one Node provider. @DRE-Team Is there a way we can address this issue in the future to offload nodes into multiple Node providers. I will vote to Adopt to recover this issue in hopes that this feature will be addressed.
You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.
CO.DELTA
We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:
Look: We observe the details and context of NNS proposals
Test: We test and verify the claims made by those proposals
Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.
Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.
Vote: Adopted Reason:
The proposal replaces dead Offline status node g4eis from the AW1 DC in Pennsylvania, with unassigned healthy Awaiting status node l3uqp from Dallas without any change to decentralization.
About CodeGov
CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron’s Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.
Learn more about CodeGov and its mission at codegov.org.
The proposal replaces a dead nodes on subnet lhg73:
dead node g4eisDashboard Status: Offline with node l3uqpDashboard Status: Awaiting
There is no impact in the overall decentralization across all features.
About CodeGov
CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron’s Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.
Learn more about CodeGov and its mission at codegov.org.
This proposal has already executed, at 2024-12-27, 19:12:34 UTC (minutes after it was submitted several hours ago). As can be seen in the screenshot below, after stalling, halting, CUP, and unhalting, the subnet is back on track and processing blocks (with the condition improving since the unhalt proposal executed).
Warning