Subnet Management - mkbc3 (Application)

This topic is intended to capture Subnet Management activities over time for the mkbc3 subnet, providing a place to ask questions and make observations about the management of this subnet.

This subnet was created to help support the growth of Caffeine AI

At the time of creating this topic the current subnet configuration is as follows:

Expand
{
  "version": 51936,
  "records": [
    {
      "key": "subnet_record_mkbc3-fzim5-s5pye-pbnzo-uj5yv-raphe-ceecn-ejd6g-5poxm-dzuot-iae",
      "version": 51936,
      "value": {
        "membership": [
          "dfcny-caobt-j247e-73ira-s3ubk-3wmbu-ghlkn-5rbej-gn2t4-xenzr-xqe",
          "mvq4c-mbmpm-2u2mv-h74op-kekjk-6rudx-jl7qb-rpxuu-zicsk-o25td-4qe",
          "6kuom-ucaom-u7xm5-s556x-3rnyk-hdjzd-wkxnx-wsejf-vpnhz-lniqi-nae",
          "m345s-atzkk-etodt-4bssh-cxcit-aw5ou-kozs4-5bipl-i46aj-uwnnh-bqe",
          "7pxqq-e4kmr-saxrs-jziwc-sfdft-bl2wh-k7vhl-6a35o-2stq5-v6uso-3qe",
          "fnopq-cvgfn-6d32v-y7n5g-kkq6x-kpmce-7tqx3-esaht-b2ovh-qhznv-5qe",
          "zgtrt-4vlgr-pbytl-t2yqq-qf4nk-wyoos-vrfpu-hxqcw-tcnfu-73kjb-pae",
          "zdean-nwelu-yqpdn-juy4c-avlne-xpl6z-5hj4p-gh7s3-nkfyg-piu5w-jqe",
          "vodet-jwe7w-uo3lw-mzljp-orvu4-d3e2q-ymdga-kqlck-wgs5j-hmcyz-sae",
          "asad5-qg3gv-p5hrf-7liwa-zkoia-t2imf-sztmb-okb3k-gc6ei-xzp77-5qe",
          "wazkf-2xehg-y26ek-2uq64-pkrbh-6frmq-4q7lf-5gaya-44q4p-v5r5z-cqe",
          "4gvj7-o7ylc-hpbec-daxvh-wlmpi-3w3sg-nzhdw-e7r65-edxmw-x5gzq-tqe",
          "4i64c-7p2wc-yotet-r64av-qwt2b-hp3hm-v3wad-bp5md-aeotm-gsofp-jqe"
        ],
        "nodes": {},
        "max_ingress_bytes_per_message": 2097152,
        "max_ingress_messages_per_block": 1000,
        "max_block_payload_size": 4194304,
        "unit_delay_millis": 1000,
        "initial_notary_delay_millis": 300,
        "replica_version_id": "e915efecc8af90993ccfc499721ebe826aadba60",
        "dkg_interval_length": 499,
        "dkg_dealings_per_block": 1,
        "start_as_nns": false,
        "subnet_type": "application",
        "features": {
          "canister_sandboxing": false,
          "http_requests": true,
          "sev_enabled": false
        },
        "max_number_of_canisters": 0,
        "ssh_readonly_access": [],
        "ssh_backup_access": [],
        "chain_key_config": null
      }
    }
  ]
}

A new proposal with ID 137800 has been submitted, please take a look.

Click here to open proposal details

Replace a node in subnet mkbc3

Motivation:
Replacing a dead node

Decentralization Nakamoto coefficient changes for subnet mkbc3-fzim5-s5pye-pbnzo-uj5yv-raphe-ceecn-ejd6g-5poxm-dzuot-iae:

    node_provider: 5.00 -> 5.00    (+0%)
      data_center: 5.00 -> 5.00    (+0%)
data_center_owner: 5.00 -> 5.00    (+0%)
             area: 5.00 -> 5.00    (+0%)
          country: 3.00 -> 3.00    (+0%)

Mean Nakamoto comparison: 4.67 → 4.67 (+0%)

Overall replacement impact: equal decentralization across all features

Details

Nodes removed:

  • asad5-qg3gv-p5hrf-7liwa-zkoia-t2imf-sztmb-okb3k-gc6ei-xzp77-5qe [health: degraded]

Nodes added:

  • bs7jr-2hrlr-krdlk-qmuzs-hkphr-3jzx5-f5smp-6xwyp-inleo-xg4qg-iqe [health: healthy]
    node_provider                                                              data_center            data_center_owner         area                             country        
    -------------                                                              -----------            -----------------         ----                             -------        
    4r6qy-tljxg-slziw-zoteo-pboxh-vlctz-hkv2d-7zior-u3pxm-mmuxb-cae       1    an1               1    Africa Data Centres  1    Brussels Capital            1    AU            1
    6nbcy-kprg6-ax3db-kh3cz-7jllk-oceyh-jznhs-riguq-fvk6z-6tsds-rqe       1    bn1               1    Continent8           1    Douglas                     1    BE            2
    6r5lw-l7db7-uwixn-iw5en-yy55y-ilbtq-e6gcv-g22r2-j3g6q-y37jk-jqe       1    br1               1    Datacenter United    1    Flanders                    1    CH            2
    acqus-l4yyc-h44lw-grfxw-h7jqf-mtvt3-huwmj-4s372-sc5db-5nsfr-2ae       1    dl1               1    Digital Realty       1    Gauteng                     1    CZ            1
    bvcsg-3od6r-jnydw-eysln-aql7w-td5zn-ay5m6-sibd2-jzojt-anwag-mqe       1    fr2          0 -> 1    Equinix              1    Geneva                      1    DE       0 -> 1
    c5svp-7pkmf-agz5x-536k7-r7rcw-4wn3a-eo7pt-ry7su-j42uq-bvnzf-iqe       1    ge1               1    Evocative            1    Hesse                  0 -> 1    IM            1
    eipr5-izbom-neyqh-s3ec2-52eww-cyfpg-qfomg-3dpwj-4pffh-34xcu-7qe       1    im2               1    Flexential           1    Maribor                     1    JP       1 -> 0
    glrjs-2dbzh-owbdd-fpp5e-eweoz-nsuto-e3jmk-tl42c-wem4f-qfpfa-qqe       1    jb2               1    Green.ch             1    Queensland                  1    SG            1
    nmdd6-rouxw-55leh-wcbkn-kejit-njvje-p4s6e-v64d3-nlbjb-vipul-mae       1    mb1               1    HighDC               1    Singapore                   1    SI            1
    rbn2y-6vfsb-gv35j-4cyvy-pzbdu-e5aum-jzjg6-5b4n5-vuguf-ycubq-zae       1    sc1               1    Master Internet      1    South Moravian Region       1    US            2
    sixix-2nyqd-t2k2v-vlsyz-dssko-ls4hl-hyij4-y7mdp-ja6cj-nsmpf-yae  1 -> 0    se1               1    NEXTDC               1    Texas                       1    ZA            1
    unqqg-no4b2-vbyad-ytik2-t3vly-3e57q-aje2t-sjb5l-bd4ke-chggn-uqe       1    sg1               1    Posita.si            1    Tokyo                  1 -> 0                   
    w4buy-lgwzr-pccs7-huzhh-qqnws-rns75-iaoox-jolrm-xs2ra-vdu3o-2qe       1    ty1          1 -> 0    Telin                1    Washington                  1                   
    wdnqm-clqti-im5yf-iapio-avjom-kyppl-xuiza-oaz6z-smmts-52wyg-5ae  0 -> 1    zh6               1                              Zurich                      1                   
1 Like

Proposal 137800 Review | Lorimer :infinity: :dog_face: - CO.DELTA △

VOTE: YES

TLDR: Replaces a node that’s been degraded for a long time with an up node.

The proposal leads to further clustering within Europe and a reduction in the average distance between nodes, but the resulting topology is within the IC Target Topology specification.

Decentralisation Stats

Subnet node distance stats (distance between any 2 nodes in the subnet) →

Smallest Distance Average Distance Largest Distance
EXISTING 42.554 km 7707.899 km 16501.202 km
PROPOSED 42.554 km 7002.552 km (-9.2%) 16501.202 km

This proposal slightly reduces decentralisation, considered purely in terms of geographic distance (and therefore there’s a slight theoretical reduction in localised disaster resilience). :-1:

Subnet characteristic counts →

Continents Countries Data Centers Owners Node Providers Node Operator
EXISTING 5 10 13 13 13 13
PROPOSED 5 10 13 13 13 13

Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) →

Continent Country Data Center Owner Node Provider Node Operator
EXISTING 7 2 1 1 1 1
PROPOSED 8 (+14.29%) 2 1 1 1 1

See here for acceptable limits → Motion 137147

The above subnet information is illustrated below, followed by a node reference table:

Map Description
  • Red marker represents a removed node (transparent center for overlap visibility)
  • Green marker represents an added node
  • Blue marker represents an unchanged node
  • Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
  • Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)

Node Changes
Action Node Status Continent Country Data Center Owner Node Provider Node Operator
Remove asad5 DEGRADED :bar_chart: Asia Japan Tokyo (ty1) Equinix Starbase cqjev
Add bs7jr UP :bar_chart: Europe Germany Frankfurt 2 (fr2) Equinix Virtual Hive Ltd ujq4k
Other Nodes
Node Status Continent Country Data Center Owner Node Provider Node Operator
m345s UP :bar_chart: Oceania Australia Queensland 1 (sc1) NEXTDC Karel Frank f3toa
zdean UP :bar_chart: Europe Belgium Antwerp (an1) Datacenter United DeNoDe z4wll
4i64c UP :bar_chart: Europe Belgium Brussels (br1) Digital Realty Allusion mjeqs
mvq4c UP :bar_chart: Europe Switzerland Geneva (ge1) HighDC Decentralized Entities Foundation xdara
6kuom UP :bar_chart: Europe Switzerland Zurich 6 (zh6) Green.ch Sygnum Bank ciprs
zgtrt UP :bar_chart: Europe Czechia South Moravian Region 1 (bn1) Master Internet Maksym Ishchenko c3oqp
fnopq UP :bar_chart: Europe Isle of Man Douglas 2 (im2) Continent8 Zarety LLC ylbc3
dfcny UP :bar_chart: Asia Singapore Singapore (sg1) Telin OneSixtyTwo Digital Capital wuhly
vodet UP :bar_chart: Europe Slovenia Maribor (mb1) Posita.si BlockFinance ozfkj
7pxqq UP :bar_chart: North America United States of America (the) Dallas (dl1) Flexential 87m Neuron, LLC mw64v
4gvj7 UP :bar_chart: North America United States of America (the) Washington 1 (se1) Evocative DFINITY Stiftung xph6u
wazkf UP :bar_chart: Africa South Africa Gauteng 2 (jb2) Africa Data Centres Honeycomb Capital (Pty) Ltd 3bohy


You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.

CO.DELTA △

We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:

  • Look: We observe the details and context of NNS proposals
  • Test: We test and verify the claims made by those proposals
  • Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.

Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.


Note that this analysis involved data provided by the IC-API, which is not open source. I’m in the process of switching over to more verifiable sources of this sort of information for future proposal reviews. See here for related discussion.

Proposal 137800 | Tim - CodeGov

Vote: Adopt

This proposal replaces node asad5 which appears in the dashboard as “Degraded”. As seen in the Node Provider Rewards tool, the node has had a high failure rate (around 13-16%) over the last several days for which data are recorded.

The replacement node appears in the dashboard as “Healthy”. As shown in the proposal, decentralisation parameters are unchanged and remain within the requirements of the target topology.

About CodeGov
CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, API Boundary API Boundary Node Management, Node Admin and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neurons' Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralisation of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

Proposal #137800 — ZoLee | CodeGov

Vote: Adopted

Reason:
Replaces Degraded node with IC_Replica_SlowerServerThanSubnetMedian status asad5 from TY1 DC in Tokyo with healthy unassigned node bs7jr from Frankfurt, without any change to decentralization.

About CodeGov
CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron's Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

Proposal 137800 Review | Malith H - CO.DELTA △

VOTE: YES :white_check_mark:

TLDR:
This proposal removes 1 node(s) from Tokyo and adds 1 replacement node(s) in Frankfurt 2.
The degrade node seems to be having major connectivity issue causing for blocks to miss. Decentralisation stats remain the same, vote to adopt

Subnet Details
Property Value
ID mkbc3…
Type application
Memory Usage 0.00 GB
Running Canisters 0
Description N/A
Provider Changes
Removed Added
Starbase Virtual Hive Ltd
Location Changes
Removed Added
Asia, Tokyo Europe, Frankfurt 2
Nodes Removed 1
Node ID Status Provider Data Center Location
asad5… DEGRADED Starbase ty1 Tokyo
Nodes Added 1
Node ID Status Provider Data Center Location
bs7jr… UNASSIGNED Virtual Hive Ltd fr2 Frankfurt 2

:white_check_mark: Passes:

:white_check_mark: Node asad5…: Health check passed.

:white_check_mark: Node asad5…: Remove from Subnet check passed.

:white_check_mark: Node bs7jr…: Replacement Status check passed.

:white_check_mark: Node bs7jr…: Not assigned to any subnet.

You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.

CO.DELTA

We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:

  • Look: We observe the details and context of NNS proposals
  • Test: We test and verify the claims made by those proposals
  • Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.

Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.

Proposal 137800 Review | aligatorr - CO.DELTA △

VOTE: YES

TLDR: Replaces unhealthy node.

  • Interconnected node providers detected: DeNoDe, Allusion :warning:
  • Proposed topology Nakamoto Coefficient stayed the same as on current topology.
Node Changes 1 removed, 1 added
Node ID Status Country City Node Provider Data Center Data Center Owner
asad5-qg3gv-p5hrf-7liwa-zkoia-t2imf-sztmb-okb3k-gc6ei-xzp77-5qebs7jr-2hrlr-krdlk-qmuzs-hkphr-3jzx5-f5smp-6xwyp-inleo-xg4qg-iqe DEGRADED → UP JPDE TokyoHesse StarbaseVirtual Hive Ltd ty1fr2 EquinixEquinix
Current Nakamoto Coefficients and Topology, avg = 4.50
Attribute Nakamoto Coefficient Identical attribute values Max allowed identical values Unique Counts
Country 3 2 10
Data Center 5 1 13
Data Center Owner 5 1 13
Node Provider ID 5 1 13
Proposed Nakamoto Coefficients and Topology, avg = 4.50
Attribute Nakamoto Coefficient Identical attribute values Max allowed identical values Unique Counts
Country 3 2 10
Data Center 5 1 13
Data Center Owner 5 1 13
Node Provider ID 5 1 13

You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.

CO.DELTA

We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:

  • Look: We observe the details and context of NNS proposals.
  • Test: We test and verify the claims made by those proposals.
  • Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.

Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.

1 Like

Proposal 137800 – LaCosta | CodeGov

Vote: ADOPT

Summary:
The replaced node is flagged as Degraded with a status of IC_Replica_SlowerServerThanSubnetMedian. Looking at the Node Provider Rewards we can see that this node is averaging a 16% failure rate. This is indeed not optimal performance and so the node should be replaced.

The proposal replaces one degraded node on subnet mkbc3:

  • Removed Nodes: nodes asad5, Dashboard Status: Degraded

  • Added Nodes: nodes bs7jr, Dashboard Status: Healthy and Not assigned to a subnet

The decentralization remains unchanged across all features

The subnet is within the current approved IC Target Topology so no issues here.

About CodeGov
CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these technical topics. We also have a group of Followees who vote independently on the Governance and the SNS & Neuron's Fund topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron, KongSwap, and Alice with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.