Subnet Management - lhg73 (Application)

Thanks @Daniel_Web3 we’re monitoring that and notifying the node providers as well, and have weekly node replacements submitted every Friday unless there are critical issues with many nodes at the same time. In this situation, the outages seems to be coincidental and not related to the issue last week.

1 Like

Came across this just now which is useful/interesting context

Voted to adopt proposal 134278. The proposal seeks to remove a cordoned node from the subnet and specifies that the associated data centre is being offboarded “after 48 months”. Decentralisation parameters are unchanged. The necessary context is provided by this forum post and associated discussion. For future proposals of this type I recommend that the background context be included in the proposal text for ease of verification.

I’ve voted to reject proposal 134278. It makes claims that I see no clear way of verifying, and supporting statements regarding the ownership of the data centre are inconsistent with records held in the IC registry. The proposal was announced here.

Voted to adopt proposal 134278. This proposal is part of a sequence of steps to remove cordoned nodes from subnets as the associated data centeres are being offboarded after 48 months of their respective DC contracts that are still private and were signed up before the Genesis. There is a great and detailed explanation of this changes in this forum post and the forum thread it is in. In the wiki there is a series of Steps for Gen-1 Node onboarding after 48 months that need to be followed in order for the nodes to continue earning rewards which starts by making a forum post in the following thread. As we can verify no one as come forward with nodes from the DCs in this proposals so I don’t see any issues with the removal of this nodes.

A new proposal with id 134550 has been submitted, please take a look.

Click here to open proposal details

Replace nodes in subnet lhg73

Motivation:
The following nodes in subnet lhg73 have been cordoned and need to be removed from the subnet:

Decentralization Nakamoto coefficient changes for subnet lhg73-sax6z-2zank-6oer2-575lz-zgbxx-ptudx-5korm-fy7we-kh4hl-pqe:

    node_provider: 5.00 -> 5.00    (+0%)
      data_center: 5.00 -> 5.00    (+0%)
data_center_owner: 5.00 -> 5.00    (+0%)
             area: 5.00 -> 5.00    (+0%)
          country: 5.00 -> 5.00    (+0%)

Mean Nakamoto comparison: 5.00 → 5.00 (+0%)

Overall replacement impact: equal decentralization across all features

Details

Nodes removed:

  • mihvd-umv3j-cjsl2-bfsdu-td7aw-2y6if-aw4fn-cghkm-v2oxd-kj75q-cae [health: healthy]
  • vlbpb-szwgu-jqul3-kpfje-ozgkp-jdwz4-f2hhq-6w6tf-e2lrl-4bmad-xae [health: healthy]

Nodes added:

  • u6j47-wepye-qtjf3-vkvnv-q6nfc-a3wky-grxp6-wbjuy-xejgs-qalg4-4ae [health: healthy]
  • 23kbh-5xqrz-2avco-tddeh-p6724-shtvz-rg3nq-d7rhu-ztedb-vki46-dqe [health: healthy]
    node_provider                                                              data_center            data_center_owner              area                                  country        
    -------------                                                              -----------            -----------------              ----                                  -------        
    3oqw6-vmpk2-mlwlx-52z5x-e3p7u-fjlcw-yxc34-lf2zq-6ub2f-v63hk-lae       1    an1          1 -> 0    Africa Data Centres       1    Bogota                           1    AU            1
    6nbcy-kprg6-ax3db-kh3cz-7jllk-oceyh-jznhs-riguq-fvk6z-6tsds-rqe       1    aw1               1    Data Inn                  1    Flanders                    1 -> 0    BE       1 -> 0
    7a4u2-gevsy-5c5fs-hsgri-n2kdz-dxxwf-btcfp-jykro-l4y7c-7xky2-aqe       1    bg1               1    Datacenter United    1 -> 0    Gauteng                          1    CH            1
    7ryes-jnj73-bsyu4-lo6h7-lbxk5-x4ien-lylws-5qwzl-hxd5f-xjh3w-mqe       1    ge1               1    Digital Realty            1    Geneva                           1    CO            1
    bvcsg-3od6r-jnydw-eysln-aql7w-td5zn-ay5m6-sibd2-jzojt-anwag-mqe       1    hk1               1    EdgeUno                   1    HongKong                         1    EE            1
    fwnmn-zn7yt-5jaia-fkxlr-dzwyu-keguq-npfxq-mc72w-exeae-n5thj-oae       1    jb2               1    Equinix              0 -> 1    Ljubljana                        1    FR       1 -> 0
    nmdd6-rouxw-55leh-wcbkn-kejit-njvje-p4s6e-v64d3-nlbjb-vipul-mae       1    lj1               1    HighDC                    1    Navi Mumbai                      1    HK            1
    otzuu-dldzs-avvu2-qwowd-hdj73-aocy7-lacgi-carzj-m6f2r-ffluy-fae       1    mr1          1 -> 0    NEXTDC                    1    Pennsylvania                     1    IN            1
    r3yjn-kthmg-pfgmb-2fngg-5c7d7-t6kqg-wi37r-j7gy6-iee64-kjdja-jae       1    nm1               1    Posita.si                 1    Provence-Alpes-Cote d'Azur  1 -> 0    JP       0 -> 1
    rbn2y-6vfsb-gv35j-4cyvy-pzbdu-e5aum-jzjg6-5b4n5-vuguf-ycubq-zae  1 -> 0    sc1               1    Rivram                    1    Queensland                       1    LT            1
    sixix-2nyqd-t2k2v-vlsyz-dssko-ls4hl-hyij4-y7mdp-ja6cj-nsmpf-yae  0 -> 1    sg2               1    Telia DC                  1    Singapore                        1    SE       0 -> 1
    ulyfm-vkxtj-o42dg-e4nam-l4tzf-37wci-ggntw-4ma7y-d267g-ywxi6-iae       1    sh1          0 -> 1    Telin                     1    Stockholm                   0 -> 1    SG            1
    vegae-c4chr-aetfj-7gzuh-c23sx-u2paz-vmvbn-bcage-pu7lu-mptnn-eqe       1    ta2               1    Tierpoint                 1    Tallinn                          1    SI            1
    wdjjk-blh44-lxm74-ojj43-rvgf4-j5rie-nm6xs-xvnuv-j3ptn-25t4v-6ae       1    ty1          0 -> 1    Unicom                    1    Tokyo                       0 -> 1    US            1
                                                                               vl2               1                                   Vilnius                          1    ZA            1
1 Like

Proposal 134550

TLDR: I’ll adopt. The proposal links directly to what appears to be discussion with the NP about the proposal. Decentralisation stats look good.

Decentralisation Stats

Subnet node distance stats (distance between any 2 nodes in the subnet) →

Smallest Distance Average Distance Largest Distance
EXISTING 317.676 km 8096.384 km 18505.029 km
PROPOSED 549.609 km (+73%) 8476.115 km (+4.7%) 18505.029 km

This proposal slightly increases decentralisation, considered purely in terms of geographic distance (and therefore there’s a slight theoretical increase in localised disaster resilience). :+1:

Subnet characteristic counts →

Continents Countries Data Centers Owners Node Providers Node Operator
EXISTING 5 13 13 13 13 13
PROPOSED 5 13 13 13 13 13

Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) →

Continent Country Data Center Owner Node Provider Node Operator
EXISTING 6 1 1 1 1 1
PROPOSED 5 (-16.67%) 1 1 1 1 1

See here for acceptable limits → Motion 132136

The above subnet information is illustrated below, followed by a node reference table:

Map Description
  • Red marker represents a removed node (transparent center for overlap visibility)
  • Green marker represents an added node
  • Blue marker represents an unchanged node
  • Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
  • Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)

Node Changes
Action Node Status Continent Country Data Center Owner Node Provider Node Operator
Remove mihvd UP :bar_chart: Europe Belgium Antwerp (an1) Datacenter United Allusion pgunx
Remove vlbpb UP :bar_chart: Europe Germany Marseille (mr1) Digital Realty DFINITY Stiftung ampm3
Add 23kbh UNASSIGNED :bar_chart: Asia Japan Tokyo (ty1) Equinix Starbase cqjev
Add u6j47 UNASSIGNED :bar_chart: Europe Sweden Stockholm 1 (sh1) Digital Realty DFINITY Stiftung lgp6d
Other Nodes
Node Status Continent Country Data Center Owner Node Provider Node Operator
56ovz UP :bar_chart: Oceania Australia Queensland 1 (sc1) NEXTDC ANYPOINT PTY LTD srrm2
c3xxv UP :bar_chart: Europe Switzerland Geneva (ge1) HighDC Archery Blockchain SCSp yngfj
r4642 UP :bar_chart: Asia China HongKong 1 (hk1) Unicom Pindar Technology Limited vzsx4
ihttm UP :bar_chart: Americas Costa Rica Bogota 1 (bg1) EdgeUno Geeta Kalwani 74vhn
bptaj UP :bar_chart: Europe Estonia Tallinn 2 (ta2) Telia DC Vladyslav Popov y3nsb
pdo46 UP :bar_chart: Asia India Navi Mumbai 1 (nm1) Rivram Rivram Inc mpmyf
um4so UP :bar_chart: Europe Lithuania Vilnius 2 (vl2) Data Inn George Bassadone inluf
cpywp UP :bar_chart: Asia Singapore Singapore 2 (sg2) Telin OneSixtyTwo Digital Capital qffmn
uzy2p UP :bar_chart: Europe Slovenia Ljubljana (lj1) Posita.si Fractal Labs AG gl27f
g4eis UP :bar_chart: Americas United States of America (the) Allentown (aw1) Tierpoint Bigger Capital codio
5v4on UP :bar_chart: Africa South Africa Gauteng 2 (jb2) Africa Data Centres Honeycomb Capital (Pty) Ltd 3bohy

*This comment references the latest comment in the Subnet Management - General Discussion thread only to generate an automated cross-link from the general thread (to improve topic navigation).


You may wish to follow D-QUORUM if you found this analysis helpful.

Known Neurons to follow if you're too busy to keep on top of things like this

If you found this analysis helpful and would like to follow the vote of the LORIMER known neuron in the future, consider configuring LORIMER as a followee for the Subnet Management topic.

Additional good neurons to follow:

  • D-QUORUM (a highly decentralized neuron that follows neurons that have been elected by the NNS)
  • Synapse (currently follows the LORIMER and CodeGov known neurons for Subnet Management, and is a generally well informed known neuron to follow on numerous other topics)
  • CodeGov (actively reviews and votes on Subnet Management proposals, and is well informed on numerous other technical topics)
  • WaterNeuron (the WaterNeuron DAO frequently discuss proposals like this in order to vote responsibly based on DAO consensus)

Note that this analysis involved data provided by the IC-API, which is not open source. I’m in the process of switching over to more verifiable sources of this sort of information for future proposal reviews. See here for related discussion.

Voted to adopt proposal #134550.

The proposal replaces two cordoned healthy Active status node mihvd from the AN1 Data Center in Belgium, and cordoned healthy Active status node vlbpb from the MR1 Data Center in France, with unassigned healthy Awaiting status node u6j47 from Sweden and with unassigned healthy Awaiting status node 23kbh from Tokyo, without any change to the decentralization of the subnet.
The motivation makes sense and the provided Forum link included in the summary provides further info, also it can be checked here.

Voted to adopt proposal 134550.

This proposal replaces 2 nodes, due to offboarding AN1 and MR1 data centres. Decentralisation parameters are unchanged and remain within the requirements of the target topology. Data centre details are consistent with the links provided in the proposal.

Proposal 134550

Vote: ADOPT

Replaces cordoned nodes mihvd and vlbpb with nodes 2grd6 and 23kbh on subnet lhg73.
The reason for this proposal is to offboard MR1 and AN1 DCs consistent with forum posts made on the forum thread used for posts regarding the renovation/sell of Gen-1 node machines by NPs.
Both the NP and DC stated in the forum post match the ones from the node being removed in the proposal.

This subnet has stalled.

Proposal 134605, 134606 and 134607

Vote: ADOPT

The subnet lhg73 stalled. DFINITY made 3 proposals in order to make a simple subnet recovery but the root cause is yet to be found as explained in the following thread. DFINITY has already cast their vote in order to recover the subnet as soon as possible and is currently running with no issues, so I have voted to adopt all three proposals to recover this subnet.

3 Likes

Proposal 134605

TLDR: I’ve voted to adopt.

Halt subnet lhg73-sax6z-2zank-6oer2-575lz-zgbxx-ptudx-5korm-fy7we-kh4hl-pqe, for recovery and update ssh readonly access

:point_up: This proposal has already executed, at 2024-12-26, 19:00:56 UTC (minutes after it was submitted several hours ago). As can be seen in the screenshot below, the subnet had stalled and was no longer processing blocks for a couple of hours prior to this proposal to halt the subnet.

.

It’s worth noting that this subnet stalled not so long ago under similar circumstances.

1 Like

Proposal 134606

TLDR : I’ve voted to reject

Update recovery CatchUp Package of subnet: lhg73 to height: 130666000

I voted to reject this CUP proposal, the reasoning for which I’ve previously discussed with DFINITY

See the prior proposal above for context

.

1 Like

Proposal 134607

TLDR: I’ve voted to adopt.

Unhalt subnet lhg73-sax6z-2zank-6oer2-575lz-zgbxx-ptudx-5korm-fy7we-kh4hl-pqe, for recovery and update ssh readonly access

:point_up: This proposal has already executed, at 2024-12-26, 20:43:06 UTC (minutes after it was submitted several hours ago). As can be seen in the screenshot below, after stalling, halting, CUP, and unhalting, the subnet is back on track and processing blocks (with the condition improving since the unhalt proposal executed).

1 Like

Voted to adopt proposals 134605, 134606 and 134607.

This subnet appears to have stalled again

1 Like

Proposal 134623

TLDR: I’ve voted to adopt.

Halt subnet lhg73-sax6z-2zank-6oer2-575lz-zgbxx-ptudx-5korm-fy7we-kh4hl-pqe, for recovery and update ssh readonly access

:point_up: This proposal has already executed, at 2024-12-27, 18:09:58 UTC (minutes after it was submitted hours ago). As can be seen in the screenshot below, the subnet had stalled and was no longer processing blocks for a handful of minutes prior to this proposal to halt the subnet.

This is the third recent stall, with this last one arriving since the comment above.

Proposal 134629

TLDR : I’ve voted to reject

Update recovery CatchUp Package of subnet: lhg73 to height: 130812000

I voted to reject this CUP proposal, the reasoning for which I’ve previously discussed with DFINITY

See the prior proposal above for context