Subnet Management - General Discussion

Governance NNS proposal discussions
, ,
221

New ‘Change Subnet Membership’ proposals were submitted yesterday. Reviews to come… :point_down:

222

Executed yesterday

223

New ‘Change Subnet Membership’ proposals were submitted yesterday. Reviews to come… :point_down:

224

Since the above discussion I’ve been meaning to take a closer look at certain node providers. Here’s my first attempt.

Sybiling nodes… Does anyone else find this concerning? →

SYBILing nodes! :scream: Exploiting IC Network… Community Attention Required!

225

New ‘Change Subnet Membership’ proposals have been submitted. Reviews to come… :point_down:

226

The CO.DELTA team have been discussing the number of offline nodes across subnets. There are many. I plan to periodically update this post, providing timestamps and the state of affairs at that point in time. The aim is to increase visibility, auditing, and to stimulate discussion.

Degraded Nodes Summary at 30/04/2025 07:51:41 UTC

Down Nodes Summary at 30/04/2025 07:51:42 UTC

4 Likes
227

Isn’t there a remuneration reduction that has taken effect for node providers who have nodes that are offline in subnets? Perhaps @sat or @bjoernek can remind us how that works and if it is applicable. My concern with your plan @Lorimer is that it may not add value when there is already a penalty in place. It may only serve to further demotivate node providers.

228

As part of performance-based node rewards it was suggested

  • Unregistered nodes should receive no rewards.
  • Performance of unassigned nodes would be extrapolated from the performance of assigned nodes of a node provider. There is the expectation, that every node provider has at least one assigned node.

For more details, please see here.

2 Likes
229

It appears that Motion proposal 135054 was Adopted. This was NNS approval to implement the Performance Based Node Rewards. @bjoernek can you confirm that it has already been implemented?

Please correct me if I’m wrong, but this proposal 135054 means that when a node provider has a node that is down in a subnet, it is subject to a remuneration reduction formula and that formula is a penalty that applies to all of their undeployed nodes as well. In other words, nodes in subnets that are not performing well already subject the node provider to a significant financial loss. Is this an accurate summary?

https://dashboard.internetcomputer.org/proposal/135054

1 Like
230

Correct, the according motion proposal has been approved by the NNS. The implementation is work in progress and the feature is not yet live.

Yes, this is a correct summary. I would only add that penalties only apply for longer down times. A failure rate of up to 10% would not result in any reduction.

2 Likes
231

Thanks @wpb, I understand your concern. However I do think visibility, tracking, and prompts for discussion and reflection are valuable. Even with measures and mechanisms in place, it’s important to check-up on how well they’re working and whether course correction may be needed. As a decentralised network, I think bringing this to the forum regularly is useful.

I’ve added some further details to the structure of that post, based on some feedback from @quint, and also incorporating additional links that I include in my SM reviews. I’ve also provided a summary by node provider showing what percentage of their assigned nodes are degraded or offline.

I plan to create a new post each month, and periodically update that post throughout the month, relying on discord’s change tracking to provide time travel for the reader (over the course of that month). I’ll post one for April in a sec using a post format that I’ll intend not to change (or only change minimally if needed).

Thanks for clarifying @bjoernek. Hopefully this will incentivise NPs to take a more active role in subnet management (raising proposals to swap out their offline/degraded nodes). Of course, if they do this proactively, the measure is no longer a good indicator of their contribution to network resources.

If a node provider has 10 nodes, but only ever has 1 assigned, and the assigned node is always online, but the unassigned nodes are always offline or just degraded - should they really be rewarded as though all nodes are online?

2 Likes
232

April: Summary at 2025-04-30 22:12

See this post and following discussion for context.

Degraded or Down Nodes

Summary Grouped by Node Provider

Check current state of affairs

1 Like
233

As discussed, performance based node rewards will incentives nodes to stay online. If I understand correctly, currently the only potential penalty is that node might be rejected in SM proposal.

1 Like
234

May: Summary at 2025-05-15 18:19 UTC

See this post and following discussion for context.

Degraded or Down Nodes

Summary Grouped by Node Provider

Check current state of affairs

1 Like
235

With the planned addition of further subnets, the ratio of subnet assigned nodes will increase, meaning a significant portion of each node provider’s nodes will need to be online (also resulting in the collection of performance metrics of these nodes).

If we really encounter a situation where most of a particular node provider’s nodes are continuously offline, this issue can be raised with the provider — and in the worst case, could result in the offboarding of those nodes.

2 Likes
236

Subnet: e66qm-3cydn-nkf4i-ml4rb-4ro6o-srm5s-x5hwq-hnprz-3meqp-s7vks-5qe - ICP Dashboard

image
image1297×855 74 KB

All w4rem nodes were also temporarily offline a second ago when I took the May snapshot. However they appear to be back online now.

Looks like this might be related to GuestOS deployments. e66qm’s one executed about 15 minutes ago.

Proposal 136411 / NNS Dapp

Update: e66qm back up. Now all nodes in 6pbhf are down. The GuestOS deployment for that subnet was executed 20 minutes ago.

@sat I’ve only just started taking regular snapshots across all subnets. Is it normal to see this sort of downtime during GuestOS deployments?

1 Like
237

Cross-posting as this relates to, and addresses some of my questions above.

I also note that in my post above I conflate the proposal execution timestamp with the subnet deployment point in time. The proposal execution just updates the registry (as far as I understand). The subnet itself is later responsible for pulling that config from the registry and updating itself (hence the delay between proposal execution and the subnet going down briefly).