Update of Gen1 NP Remuneration

General
1

Summary

The NNS is designed to maximize node decentralization. As highlighted in the developer forum post “The State and Direction of Decentralization & Nodes on the Internet Computer”, there have been various gaps between design and implementation that are being addressed in the node decentralization roadmap.

Over the next few days, there will be NNS proposals that will close one of those gaps: node provider remuneration. These NNS proposals are to upgrade the NNS so current node providers’ remuneration is much more transparent, automated, and standardized as NNS was originally designed.

This is an important step towards removing human factors in the NNS towards the goal of making the Internet Computer the most decentralized blockchain in the world. It is important to be intellectually honest about the shortcomings between design and implementation (without any delusions) so we can fulfill the IC’s potential.

Background

As of January 28, 2022, there are

Node Provider Remuneration: The Design Intent

The NNS is designed so node providers are remunerated in ICP for the well-behaving nodes they have in the network. This amount of ICP they receive is pegged to XDR. The NNS and the open market set heterogeneous but transparent standards for what node providers are paid.

The long-term design goal is to create a healthy network that incentivizes node providers to have necessary hardware available and ready in data centers. The constraint is to make sure that node providers would not be paid so much that it creates unnecessary ICP minting (inflation).

Closing the gap between design and implementation: The NNS Proposals

The current state of the world is a combination of the following:

  1. NNS proposals are submitted to pay node providers (the NNS mints and pays node providers). Community votes on these. This is good and as intended.
  2. NNS proposals are submitted on the node providers’ behalf. This is not what is intended and instead, these proposals will be automated to remove human components.
  3. The node providers are paid varying rates (the biggest reason is geography). The variance of rates is ok by itself, but the intent is that there would be set and transparent standards (e.g. “Nodes in Germany” vs “Nodes in Chile”) that currently do not exist. These new NNS proposals will create a starting point for transparent & standardized rates for node providers all over the world. More areas will be added, for example.

Table of Standards

Using the historical information from the last 8 months since network Genesis in May 2021, the following standards are being proposed in the NNS proposals:

Region node category XDR reward per month per node
US 0 873.36375
US/FL 0 1087.24875
US/GA 0 1087.24875
US/CA 0 1087.24875
CA (Canada) 0 891.1875
EU (European Union) 0 1087.24875
AP (Asia Pacific) 0 1212.015
US (United States) 2 1604.1375
CA (Canada) 2 1782.375
EU (European Union) 2 2174.4975
AP (Asia/Pacific) 2 2424.03

Please note: This table is for existing (Generation 1) node providers. For the subsequent generation of node providers, the payout rates may change.

NNS PROPOSALS

The relevant NNS Proposals will be added to this forum thread once live.

What we are asking the community

We are asking the IC community to review these NNS proposals, ask questions, vote ACCEPT or REJECT.

11 Likes
2

Relevant Questions

1. Why did it take so long to automate and standardize NNS proposals for node provider remuneration?

The intent was to standardize and automate earlier. The simple reason for the delay is that network stability and decentralizing other parts of the NNS jumped the priority queue once the network became real and live.

As recognized in “The State and Direction of Decentralization & Nodes on the Internet Computer”, there are gaps between the design and implementation and they are being tackled systematically.

2. How was this table of payout rates derived?

The table was derived by looking at the average costs to obtain equipment and run Internet Computer nodes across different regions. The geographical distribution of the nodes is extremely critical to the decentralization goal of IC.

3. Can I vote “REJECT” on these changes?

Yes, one most certainly can. That is the purpose of algorithmic on-chain governance.

4. Where is the documentation about how I can add myself as a node provider?

As stated in “The State and Direction of Decentralization & Nodes on the Internet Computer”, there have been many node providers onboarded, but the experience was not smooth (requiring lots of fixes, patches, and technical know-how). Rather than invest documentation and patches into a system that was unsatisfactory, the team decided to bridge all the gaps between design and implementation. The current plan is to clean up all the technical debt so node providers can add themselves to the network in a frictionless way. The team is 100% focused on improving this so it matches the original goal. As this completes, there will be updated documentation.

5. What is the node decentralization roadmap?

Q4 2021

  • (A) Reproducible builds: Reproducible replica software builds are planned to be available to the public in November 2021. This initiative has its own separate communication because it touches many parts.

Q1 2022

  • (B) Autonomous node deployment and operation: In early 2022, a new deployment process will be introduced based on the reproducible builds (A). Over the subsequent weeks, existing node providers will be asked to update their deployments using the new process.
  • (C) Independent node provider registration: In early 2022, future node providers will be able to register via the NNS frontend dapp. In parallel with the onboarding of the currently backlogged nodes, a limited number of new DCs and nodes will be added to the network using the new deployment method (B).
  • In order to get started with autonomous node deployments (B) before the NNS-driven DC allocation is in place (D), a small set of new node providers will be onboarded using (C).

Q2/Q3 2022

  • (D) NNS-driven DC allocation: The NNS-controlled DC allocation mechanism will be introduced and control all future node additions. With this step and a new node hardware, see (F), an accelerated network growth is unleashed.
  • (E) Availability of node hardware: Different hardware options are currently being investigated. This blocker might be overcome in two steps
    1. Minor extensions to the Genesis hardware setup to overcome current availability limitations.
    2. Larger extensions that incorporate additional hardware technologies such as SEV-SNP.

6. What are the current blockers for anyone to contribute nodes to the IC?

To see updates you can follow the latest demo of milestones for the roadmap.

The following blockers must be addressed before the IC’s network growth can be further accelerated:

  • (A) Reproducible builds: The community, including node providers, shall be able to build the replica software in a deterministic manner. This enables the community to verify that IC update proposals indeed correspond to the source code version specified in the NNS proposal.
    • Completed Q4 2021.
  • (B) Autonomous node deployment and operation: Empower node providers to install and maintain nodes without any support from the ICA.
  • (C) Independent node provider registration: New node providers shall be able to register directly in the NNS frontend dapp. They will be able to submit a proposal to become a new node provider, without the support of the ICA.
    • In progress
  • (D) NNS-driven DC allocation: Node providers can participate in NNS-controlled mechanism to elect new data centers. This process shall incentivise the addition of new DCs that further decentralize the IC’s network. Furthermore, it balances inflationary and deflationary forces.
  • (E) Availability of node hardware: A new generation of ICA-specified node hardware is planned that shall guarantee global availability and provide a better choice between hardware providers. Furthermore, the next generation shall include the SEV-SNP technology to further improve the security of nodes.
4 Likes
3

The conversation leads for any questions are @Luis @samuelburri

2 Likes
4

Furthermore, the next generation shall include the SEV-SNP technology to further improve the security of nodes.

What is the plan to make the already onboarded nodes to support SEV-SNP? Is that possible or only nodes being newly added will support this?

4 Likes
5

The first three node types are running on AMD Rome. They already support SEV-ES. The next node types with AMD Milan will support SEV-SNP.

5 Likes
6

whether ICP can deflate. As the application increases, the reward for the node also increases. as the application increases the Icp burning also increases. Which one will be bigger?

7

Dear Community,

This is Katie Peters from DFINITY. I have been primarily responsible for the Gen-1 Node Providers since Genesis.

Node providers receive rewards (remuneration) for operating node machines that run the IC network. The single source of truth for node provider rewards is the NNS, where changes can only be made through NNS proposals adopted by the IC community.

Gen-1 Hardware was purchased by Node Providers prior to Genesis launch according to the planned usage at that time. We are now two years post-launch, and changes in the roadmap and planning are being realized. We propose another update to this remuneration to realign rewards while still honoring our commitment to these Node Providers.

This document details a proposal to further refine Gen-1 minted rewards so they are aligned with current IC usage. We propose to do this through three actions:

  1. Remove the rewards for type-2 servers from the NNS, for reasons explained below

  2. Create type-1 minted amounts to cover total costs for type-1 servers now that the storage is installed

  3. Reassigning nodes from type-0 to type-1 for all nodes where the node provider paid to upgrade the storage in their nodes

Current status

The NNS currently mints ICP tokens totalling approximately $1.4 million XDR each month for Gen-1 Node Provider rewards. (This represents 22.1% of the total ICP tokens minted by the NNS each month.)

The Foundation has been covering rewards for the additional storage costs incurred by node providers to purchase and install the storage in the interim period between when the storage was purchased and when it was installed. Now that the IC can begin to utilize the storage, these node assignments must be changed from type-0 (no extra storage) to type-1 (additional storage added). We propose type-1 reward values according to the chart below.

Type-1 Remuneration Values

We propose to add new type-1 node rewards according to the following chart. These type-1 values include both the original equipment costs incurred by Gen-1 Node Providers and the varied costs that Node Providers incurred to purchase the additional storage, ship it to their data centers, and get it installed in each server. (Costs for additional storage varied due to taxes, shipping, and installation. ETA: These values also included the Opex which is not changing.)

image
image695×379 11.8 KB

If the proposals are accepted by the community, the new total of ICP tokens minted for Gen-1 NPs would become approximately $1.7 million XDR per month.

Type-2 nodes

Approximately $217k of the total current NNS-minted rewards for Node Providers are for the type-2 subset of Gen-1 nodes.

These type-2 nodes have not yet been utilized by the IC/NNS. It has been determined that the hardware (specified here) is no longer suitable for the original intended usage, and the Foundation is exploring other uses for these servers. We therefore propose to remove the rewards for these nodes from the NNS, and the Foundation will assume responsibility for these Node Provider rewards.

NNS PROPOSALS

The relevant NNS Proposals will be added to this forum thread once live. They will include:

  • A motion proposal first Proposal 122635
  • One proposal to add the type-1 node remuneration table Proposal 122762
  • Proposals to update the affected nodes from type-0 to type-1
  • Proposals to remove the type-2 node rewards from the NNS

What we are asking the community

We are asking the IC community to review these NNS proposals, ask questions, vote ACCEPT or REJECT.

7 Likes
8

Hello Katie,
May I ask how many Gen1 type-2 hardware servers will be removed from the list of unutilised IC nodes (assuming the proposal is passed)?

9

Hello icraus!

It is a total of 114 servers which are type-2.

1 Like
10

Hi Katie, the type-1 remuneration is only for hardwares and hardwares related cost, how about OPEX?

11

The Opex was included in the tpye-0 amounts along with the server costs. We did not take the Opex costs out.

We are adding the hardware costs for the extra storage, so the new type-1 amounts will include Opex costs, server costs and additional storage costs.

1 Like
12

Hi Katie, if I do my calculation with new type-1 rewards, we will receive less rewards than before, is my total monthly rewards going to change?

13

You will not receive less. We are making sure of that! I sent you a message.

1 Like
15

Hi there @Tromix

I removed your post because it was inappropriate in its wording. Please find a way to rephrase your intent (even if its criticism of an idea or process).

yeah, not fun to be a moderator of the developer forum, but we are all working to make this a better community for dialogue.

1 Like
16

(context: user saw my comment above and updated part of it)

@Tromix i appreciate you softening one word in your post (the loudest one, arguably), but i will explain why I still flagged our post:

  1. You quote something that does not exist (as if to pass your opinion as someone else’s). This is something sometimes done by bad actors trying to manufacture fake consent.

  2. You accuse an entity (in this case DFINITY) of minting tokens for node providers, but this is exactly how ICP works (and always has) so it hints at either an unwillingness to learn the protocol, or potentially something worse. It is reasonable anybody wants to change something in the protocol (maybe you want to make an NNS proposal to accept less node providers for example), but the bar for explaining one’s stance is higher in the developer forum.

Any one thing would be on the fence, but the totality of these was too much for me to ignore.

No harm, no foul. Always can try again. That is what this community is about.

Hope that helps.

17

Fine, attempt #2.

Is this proposal asking the user base to increase node provider rewards (monthly token inflation) whilst knowing that every single retail investor and participant in the NNS (sans VC’s, Dfinity, Seed investors, node providers) are deeply in the red and underwater on their investment?

18

Dear Community,

This is Katie Peters again from DFINITY. As you know, I have been primarily responsible for Gen-1 NP rewards since Genesis.

When the Gen-1 NPs purchased their servers prior to Genesis, the reward agreements were set in place for 48 months. The earliest of these final dates are still almost a year away, but we wish establish future rewards as soon as possible.

The long term objective is to have remuneration “based on useful work” for all node machines, which means node rewards are paid out based on the actual contribution to the IC, e.g. the number of blocks created, the size of the blocks created, how many times the node machine has been a block master, in which subnet the node machine is running, etc; regardless of the type of node machine and when the node machine was bought. Since implementing this new approach to remuneration requires extensive discussion within the community as well as time to design and develop, an interim approach is required for the remuneration of Gen-1 node machines for which the node provider agreements will expire.

We have created a wiki page outlining an interim remuneration proposal: rewards for Gen-1 node machines after 48 months.
https://wiki.internetcomputer.org/wiki/Proposed_Gen-1_Remuneration_Model

The proposals will be linked as soon as they are created. We are asking the IC community to review the wiki page, review these NNS proposals, ask questions, and vote ACCEPT or REJECT.

4 Likes
19

Hi Katie,

Thanks for posting and providing clarity. Shouldn’t there be some deliberation before the proposals are posted?

My question centers around what Node Providers who bought more then 42 nodes at the start are to do? What is the plan for them? These are the Node Providers who took the biggest risk to back the network. It is understandable that we want to limit the number of nodes per node provider, but perhaps a couple of new node providers can step in to purchase and take over the nodes for the node providers who have more than 42 nodes. That would help decentralize the network while also not penalizing the Node Providers with a greater number of nodes.

Please let me know what you think of my suggestion.

Best,
David

1 Like
20

Hi Katie,
Is there no difference between type-0 (no extra storage) and type-1 (additional storage added) gen1 nodes anymore in the new proposed rewards? type-1 nodes earned ~50% more rewards for providing the storage. When there is no difference anymore what stops node providers with storage to uninstall the storage and use it elsewhere?
Best,
David

21

Hi Katie and Team,

Thank you for the update - you and the team have clearly put a lot of thought and effort into this - I really appreciate it!

I do think the node providers bring a unique and somewhat different perspective to the discussion (see DavidF above) that would be good to get into the mix along with the objectives of decentralization.

I am trying to figure out the best way for us (the Gen 1 node providers) to provide input and help shape the proposal. I have two broad areas of questions - 1. Process and 2. Substance.

PROCESS

What is the planned timing for the proposal to go live on Gen 1 node provider interim rewards?
What is the best way to provide input on the proposed plan? - I’m sensitive to just having many conflicting comments on a thread either just creates more work for those writing the proposal or good ideas getting left out. Note there are other options vs. just commenting on a thread, such as a group of Gen 1 node holders trying to provide consolidated feedback.

SUBSTANCE

I read through the remuneration model you provided and Bjoern’s thoughtful discussion around network decentralization. I thought as a precursor to the discussion around remuneration it might be helpful to answer a few basic questions about the target network in the 2025 timeframe:

What is Dfinity’s perspective on the acceptable total number of Gen 1 node machines operating in the 2025 timeframe?

[DISCUSSION: We have 1100 current machines and are short type 2s. Bjoern’s analysis suggests a need for 750. How many Gen 1 node machines does Dfinity think need to exit the network? Are you OK with all current Gen 1 node machines staying online in the network? Or do you feel that some number needs to exit? As pointed out in DavidF’s note, several node holders have more than 28 and a few have more than 42 - his question about selling nodes might have a different answer if you are trying to get nodes to exit. ONE NOTE: Bjoern’s analysis did not seem to factor in downtime and nodes offline (significant and growing from what I have seen) - I think all of the numbers need to be increased for cushion/slack]

What is the objective of the decreasing marginal return remuneration scheme - would not a flat payment scheme for at least the first 28 nodes be simpler and accomplish the same objective?

[DISCUSSION: Nodes will stay in the network as long as the present value of the marginal return exceeds the exit/salvage value of the hardware. So the declining nature of the curve up to 28 will not influence whether a node stays online or not.

The declining return remuneration curve does not help decentralization, it only distorts the return profile of node providers based on the size of investment - people who invested in more nodes get a lower average ongoing percentage return than those who invested less to acquire a few nodes. Not sure why this would be an objective of the remuneration structure. A flat payment per node up to 28 nodes (or so depending on desired slack) would seem fairer, easier to implement, and easier to understand. ONE OTHER NOTE: A lot of the hardware likely has value but there is also a lot of SSD memory in the units that could be repurposed in addition to the servers.]

What does Dfinity want to have happen for node providers hosting more than 28? More than 42?

[DISCUSSION: I think a flat rate up to 28 (again, maybe somewhat higher for slack) would be simpler and fairer. There are a couple of subnets where a node provider with 42 units would not harm the Nakamoto coefficient. However, the proposed remuneration scheme suggests a negative operating return above 28 - which would mean that any node provider with more than 28 would shut down those machines. If the network can accommodate a few node providers up to 42 without impact, why not just extend the curve to those nodes as well? Similarly, I had some questions for a node provider with over 42 nodes’ impact on the Nakamoto coefficient. Do extra machines hurt the Nakamoto coefficient or do they just not add any value to decentralization? If fairness is a consideration, and the extra machines do not hurt the network maybe an answer would be just to extend the payment curve for the 2 or 3 node providers that fall into this category. Or, perhaps these excess units could serve as a backup pool for the rest of the network]

Apologies for the length of the note - I appreciate all that the Difnity team does to promote the network and I am incredibly grateful for being part of this journey. Go IC!

David Mark

2 Likes