Proposal: Update Interim Gen-1 Node Provider Remuneration After 48 months

Proposal #135069 — Zack | CodeGov

Vote: Adopted

Reason:
In line with node rewards after 48 month for gen1, NP Decentralized Entities Foundation currently with 0 nodes adds NO xdara for 14 nodes that were taken over from Archery Blockchain SCSp as per this forum post that would have been preferred to be linked to in the summary instead of the general thread, in DC GE1 needed for the HSM less migration.

About CodeGov (click to expand).

CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these topics and Synapse on most other topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron and KongSwap with a known neuron and credible Followees.


Learn more about CodeGov and its mission at codegov.org.

1 Like

Proposal 135069 – LaCosta | CodeGov

Vote: REJECT

As part of the Steps for Gen-1 Node onboarding after 48 months, the Node Provider Archery Blockchain SCSp with 56 type1 nodes is handing over 14 nodes from ge1 DC to the Node Provider Decentralized Entities Foundation, which therefore is creating the Node Operator xdara to hold the 14 nodes in the ge1 DC.
All the required steps were followed:

  1. A Forum post by the NP :white_check_mark:
  2. The self-declaration document was uploaded on the IC Wiki :white_check_mark:
  3. The Node Operator ID proposed is new :white_check_mark:
  4. A statement, published on the IC wiki, signed by both the existing and the new node provider :x:

One of the requirements for the handover it’s a document signed by both parties confirming the handover, an example can be seen in this NP Arceau NP LLC wiki’s.

@Gwojda please consider solving this issues asap. There is still 3 days until the end of the proposal but documents need to be approved once uploaded on the wiki which might take some time.

@katiep Any insights into this?

About CodeGov…

CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these topics and Synapse on most other topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralization of SNS projects such as WaterNeuron and KongSwap with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

3 Likes

@Gwojda Thanks for providing us with this information. Would you be able to provide some proof that the sale has taken place, either through the handover statement signed by both parties (or 3 parties in this case) that @LaCosta has mentioned, or by some other means? I think that’s what’s missing for us to be fully comfortable supporting proposal 135069.

As an aside, could I also recommend including a link to a specific post in these sorts of proposals rather than just the thread as a whole? This isn’t a formal requirement but helps everyone keep track of proposal reviews and discussion.

@katiep Should Extragone also establish a new node provider ID rather than just taking on the one used by the previous owner? This would obviously need 2 extra proposals to remove the old NP and add the new one, and by my understanding is probably needed in order to change the registered name of the NP. (Correct me if I’m wrong on that.)

2 Likes

Greetings!

As work is being done to decrease rewards for Gen-1 and increase the decentralization among Gen-1 NPs, the question arises regarding the rewards amount that node providers would get if they incurred additional costs to relocate their nodes to a new country not yet represented on the IC thereby increasing decentralization.

When Gen-1 and Gen-2 reward amounts were in the earliest stages of discussion years ago, it became clear to everyone that data center costs vary widely around the world. As the topology stands right now, there are already nodes set up in the countries of the world that are most economical and stable.

It is therefore clear that if one of the new NPs who are purchasing “excess” Gen-1 servers is willing to move those nodes to a new country, they would incur:

  • Higher DC costs than any of the current Gen-1 NPs
  • Significant transportation and import costs to move the servers

It is recognized that adding another country to the topology is more desirable for decentralization compared to if the nodes remain running in a country/region that already has many other nodes. We therefore feel that at this time, a slight increase in rewards is justified for a NP who moves nodes to a new country.

We would therefore like to propose a Gen-1.1 rewards value that is a 10% increase over the highest current Gen-1 base reward value, for nodes in a country that is new to the IC, with the understanding that:

  • Due to the recent change from Gen-1 to Gen-1.1, these base rates are 33% less than Gen-1 NPs got originally, so this is still a significant decrease from the Gen-1 amounts.
  • This proposal is only for moves that occur as part of the Gen-1 to Gen-1.1 transition happening in Q4 2024 - Q1 2025.
  • The current highest Gen-1 reward value is 1234 XDR, so the proposed value would be 1357 XDR.
  • The “new country” should not currently have any Gen-1 or Gen-2 nodes running as of December 2024, nor should it be a country that is an EU member.

We welcome any thoughts that the community has.

6 Likes

How do you plan to identity fake kyc and fake ip locations
Are you (dfinity) willing to visit datacenetrs to physically check? Or what
Reward increases incentives new providers to spoof location It’s easier than actually moving nodes

2 Likes

This is a subject that is being actively discussed in the Node Provider Working Group meetings.

Faked KYC certainly seems like the bigger problem.

@LaCosta was kind enough to point me to a geolocation service that appears to be very accurate → https://search.censys.io/

From what understand it relies on a combination of approaches, one of which is an extensive probe network that establishes location based on response latency for requests from a number of geographically distributed servers.

Assuming it can be measured accurately enough, incentivising nodes in more counties definitely sounds good and will help satisfy the IC Target Topology more easily.

If a NP moved two nodes to the same new country, would both nodes qualify for the boosted rewards? Are there any plans for how long the boosted rewards would apply for?

3 Likes

We would support this new rate for any new country (as long as not in the EU, due to the number of EU nodes that we already have) for the full number of nodes that the NP(s) have, and that the rate would be for two years, as are the rest of the Gen-1.1 rates. NPs typically have to sign DC contracts for multiple years, so we have found that, from a business perspective, it is hard to get NPs to sign on (at least now while the IC is still in its infancy) if they cannot make “safe” assumptions of rewards for at least two years.

All NPs know (because we have been open about it) that the eventual goal is to move to a completely different model for rewards that is based on other factors like contribution, but that is still in the future. For the moment, we are supporting this two-years model, which is the case for all of Gen-1.1. (Though all know that changes can still be proposed to the NNS and approved at any time.)

3 Likes

Does this mean that if an NP with 50 nodes moved one of their nodes to a new non-EU country, they’d recieve a reward boost that’s 50X that of a comparable NP with only 1 node (and which has also been moved to a new non-EU country)?

1 Like

Hello timk11!

That is a good question, at the moment, the registered name of the NP is not in the registry. It is still manually done by us, so a proposal is not necessary. If the selling NP is comfortable giving the HSM to the buying NP, we do not know of any reason why they cannot do that, along with the servers. There is only one HSM and when we set them up at Genesis (for Gen-1), no backups were made; therefore no backups are possible without bricking the HSM (which would require a new one anyway.)

For NPs who are doing the newer HSM-less onboarding, I would consider it a security risk to give the (non-hardware) keys to a buying NP, but that isn’t the case for a Gen-1 who is still using the one-and-only HW key that exists.

Thanks for the great question!

@Gwojda I agree that we need to see the handover declaration please. If it’s already uploaded to the forum, please provide the link. Thank you!

4 Likes

Thank you LaCosta,

Here is the history:

  • Archery was a Gen-1 NP prior to Genesis and as such went through the Foundation’s KYC, which we were doing at the time internally. (There was no community or wiki or self-declaration back then.)
  • In 2024, when we rolled out the request for Gen-1 NPs to add the self-declaration when transferring to Gen-1.1 (so their status matched Gen-2), Archery notified us that they were selling their nodes to a different business entity. We told them that, in that case, it made sense for the new entity to do the self-declaration, which they have done here.

You will notice, if you look at the history on the wiki, that none of the Gen-1 NPs had self declarations before 2024 (because they all did KYC internally). Of course, we (DFINITY) no longer do KYC since the IC is are now decentralized and the self-declaration is public.

I agree with you that we do need the handover statement between Archery and Decentralized Entities Foundation, as that is part of the terms that were voted in for selling Gen-1 nodes.

Regarding Extragone, there isn’t any formal requirement through voting or grandfathered rules established at Genesis for a NP to prove where/how they purchased node machines. (In fact, I cannot remember that anyone has requested this information from any NPs of either generation of NPs.) I also don’t think the community has discussed or decided on any rules regarding the transfer of nodes. This is probably something that needs to be decided for future transfers.

So this is obviously an unusual case no matter how you look at it. Extragone has done their self-declaration which is the normal requirement for any new NP, which is the clear requirement.

2 Likes

Hello Lorimer!

I think that I do not understand your scenario. My apologies.

NPs are only allowed 42 nodes, and there are some Gen-2 NPs who have nodes in different countries. The NNS currently calculates each node according to the rate for the country that it is in. If they have 10 nodes in one country and 10 in another, then they receive the rate for the first country times 10, plus the rate for the second country times 10. So the location of a node in one country does not affect the rate of nodes in any other countries.

Each reward rate is per node. So a NP who set up 40 nodes in a given country is going to receive 40x the rewards as a NP who has 1 node in that same country.

Note that DC costs are such that it is next to impossible to earn a profit operating one node in a DC. Even 2 or 3 can be challenging unless you’re already leasing cabinet space for a different business purpose, or you share cabinet space with someone. (This is called co-location in the DC world.) Most DCs cabinet leases cost thousands every month. Thus, NPs typically have to run more than one and split those costs among more than one node.

On the flip side is the consideration that in most DCs, there is an upper limit to the amount of power that can be supplied to a single rack. Therefore once a NP reaches that limit, they have to lease another cabinet in the DC set up connectivity between the two racks/cabinets, etc. which costs extra as far as equipment, base power rates, etc.

ETA: Also, just as some general information, the original Gen-1 type-1 node rewards (here) were originally calculated so that roughly half the amount covered ongoing operational costs and half the amount covered purchase costs (amortized over 4 years). Of course, operational costs have risen over the last four years. This is why the community, NPs, and DFINITY felt that the Gen-1.1 reduction (at the top of this thread) of 33% was reasonable. It was considered that they had paid off their purchase costs already, and yet operational costs had risen over the last four years. (Unfortunately we all know that EVERYTHING has risen in cost over the last four years!) Anyway… I personally feel that, considering the fact that any purchasing NP would be starting over again with purchase costs AND still incurring monthly operational costs, that proposed rate will still be a challenge for them to find a new country wherein they can still make a profit as a business. This is based off of everything that I’ve seen over the last four years.

I hope that answers your question and provides a bit of perspective about the costs and upper/lower limits that NPs are trying to find, in order to host nodes for the IC.

3 Likes

Proposal #135157 Review — Louise | Aviate Labs

Vote: ADOPT
Review:

(Gen-1) Node Provider Allusion is adding a new Node Operator record for 28 nodes in the BR1 DC. This is inline with the steps to migrate from the legacy deployment method, to the HSM-less deployment method being used by Gen-2 NPs.

  • The previous NO record with ID mjeqs shows 28 Gen-1 nodes on this site, which is equal to the allowance being proposed. :white_check_mark:
  • The NP ID belongs to Allusion as they will be retaining the nodes in this data center :white_check_mark:
  • The NO ID being added is new and does not reference any records from other data centers :white_check_mark:
About Aviate Labs

Aviate Labs is a team dedicated to supporting node providers since 2020. Our mission is to make high-performance infrastructure management on the Internet Computer (ICP) as seamless as possible, while adhering to the principles of decentralization.

We are known for our contributions to the ecosystem, including the go-agent and developer work packages on GitHub, as well as the Node Monitor tool, which alerts Node Providers as soon as any of their nodes go down.

In the NNS, our team—Roald, Louise, and Quint—reviews and votes independently on ‘Node Admin’ and ‘Participant Management’ proposals. Currently we don’t vote at all on other proposals.

The Aviate Labs known neuron is configured to follow our team for these topics and other trusted entities for broader proposals. We strive to be a credible and reliable Followee, committed to voting on every proposal and supporting decentralization within the ICP ecosystem.

Proposal #135158 Review — Louise | Aviate Labs

Vote: ADOPT
Review:

(Gen-1) Node Provider Allusion is adding a new Node Operator record for 14 nodes in the BR2 DC. This is inline with the steps to migrate from the legacy deployment method, to the HSM-less deployment method being used by Gen-2 NPs.

  • The previous NO record with ID oorkg shows 14 Gen-1 nodes on this site, which is equal to the allowance being proposed. :white_check_mark:
  • The NP ID belongs to Allusion as they will be retaining the nodes in this data center :white_check_mark:
  • The NO ID being added is new and does not reference any records from other data centers :white_check_mark:
About Aviate Labs

Aviate Labs is a team dedicated to supporting node providers since 2020. Our mission is to make high-performance infrastructure management on the Internet Computer (ICP) as seamless as possible, while adhering to the principles of decentralization.

We are known for our contributions to the ecosystem, including the go-agent and developer work packages on GitHub, as well as the Node Monitor tool, which alerts Node Providers as soon as any of their nodes go down.

In the NNS, our team—Roald, Louise, and Quint—reviews and votes independently on ‘Node Admin’ and ‘Participant Management’ proposals. Currently we don’t vote at all on other proposals.

The Aviate Labs known neuron is configured to follow our team for these topics and other trusted entities for broader proposals. We strive to be a credible and reliable Followee, committed to voting on every proposal and supporting decentralization within the ICP ecosystem.

Proposal #135159 Review — Louise | Aviate Labs

Vote: ADOPT
Review:

Node Provider Arceau NP LLC is adding a new Node Operator record for 28 nodes in the CH2 DC. This is inline with the steps to migrate the nodes from the previous Node Provider Rivonia Holdings LLC.

  • The previous NO record with ID vkwql shows 28 Gen-1 nodes on this site, which is equal to the allowance being proposed. :white_check_mark:
  • The NP ID belongs to Arceau NP LLC as they will be taking over the nodes in this data center :white_check_mark:
  • The NO ID being added is new and does not reference any records from other data centers :white_check_mark:
  • A signed handover statement is present on the wiki :white_check_mark:
  • Hash of the signed handover statement matches what is published on the self declaration page of Arceau NP LLC :white_check_mark:
About Aviate Labs

Aviate Labs is a team dedicated to supporting node providers since 2020. Our mission is to make high-performance infrastructure management on the Internet Computer (ICP) as seamless as possible, while adhering to the principles of decentralization.

We are known for our contributions to the ecosystem, including the go-agent and developer work packages on GitHub, as well as the Node Monitor tool, which alerts Node Providers as soon as any of their nodes go down.

In the NNS, our team—Roald, Louise, and Quint—reviews and votes independently on ‘Node Admin’ and ‘Participant Management’ proposals. Currently we don’t vote at all on other proposals.

The Aviate Labs known neuron is configured to follow our team for these topics and other trusted entities for broader proposals. We strive to be a credible and reliable Followee, committed to voting on every proposal and supporting decentralization within the ICP ecosystem.

Thanks @katiep, this is what I initially expected by I misinterpreted the comment about the full number of nodes that an NP has. Thanks for clarifying.

Doesn’t Allusion have 70 nodes (according to the dashboard)?


Out of interest, if every NP chose the same new non-EU country (increasing the overall country count by one), would they all be entitled to the same reward boost than if they all had chosen distinct new non-EU countries (increasing the overall country count significantly)?

Proposal 135069 | Tim - CodeGov

Vote: Reject

This proposal adds a new node operator ID for node provider Decentralized Entities Foundation for nodes acquired from Archery Blockchain SCSp. However, the required handover statement signed by both parties as specified here has not yet been provided.

About CodeGov…

CodeGov has a team of developers who review and vote independently on the following proposal topics: IC-OS Version Election, Protocol Canister Management, Subnet Management, Node Admin, and Participant Management. The CodeGov NNS known neuron is configured to follow our reviewers on these topics and Synapse on most other topics. We strive to be a credible and reliable Followee option that votes on every proposal and every proposal topic in the NNS. We also support decentralisation of SNS projects such as WaterNeuron and KongSwap with a known neuron and credible Followees.

Learn more about CodeGov and its mission at codegov.org.

2 Likes

Hello again!

Allusion had 70 since Genesis (along with others), but they also fall under the first post at the top of this thread, so they are required to reduce to 42.

The owner of Allusion, Paul, posted here stating which DC they were offloading, and that the date was Jan 31. Therefore, they stopped earning Gen-1 rewards for all 70 nodes as of Jan 31. If they haven’t turned off the extra 42 nodes yet, that’s no issue. It takes some time to do all these things. You’ll notice that the IC isn’t using AN1 because they’re no longer getting rewards. You voted on the proposal to move subnets away from that DC because they were selling the servers.

We are still waiting on the announcement of who has purchased their excess nodes. The buyers do not have deadlines of how fast they have to get the nodes re-onboarded… transfers when you’re dealing with registering business entities and signing or transferring DC contracts can often take many weeks. (Any NP who sells nodes but the buyer wants to keep them running in the same DC to avoid any transportation costs and the work of tearing everything down and setting it all back up is having to navigate that transfer process with the DC, so you might see a lot of this sort of thing this month for a number of the NPs who were hosting more than 42 nodes since Genesis… NPs like Allusion, Blockchain Dev Laps, Rivonia, 162, etc.)

Because historically, rewards have been minted two weeks after the month’s end, all the NPs have to wait until after Feb 13 to start submitting the proposals for the new Gen-1.1 rewards, since the Feb 13 minting is the last one of their 48 months which started prior to Genesis. The chart with dates is much farther up the thread and is on the wiki… that chart was posted for the NP’s benefits to help them navigate this transition time, but you’re welcome to refer to it for understanding if you wish. You’ll see a bunch of those proposals going through in the second half of this month because we (DFINITY) are going to clean up what’s left from the pre-Genesis rewards, but the NPs have to submit the proposals to start the Gen-1.1 rewards for February (which will be covered in the March minting.)

ETA to answer your second question about if all the NPs moved:
Before we considered this question, we already knew from almost all of the Gen-1 NPs that they were not interested in incurring the expense and trouble to move nodes. Almost a year ago, for example, when we were discussing the excess nodes that we had in Europe and the US, I asked a bunch of them if they would be interested in moving their nodes to new countries to help with the decentralization. A number of them got back to me and said that they had looked into the costs, and it just wasn’t worth it. They knew that voters and DFINITY would not support a rate that would make financial sense for them to do it. Cancellation of DC contracts is VERY expensive… often tens of thousands.

In some ways, it would have been preferrable for a whole bunch of them to change countries, but as you can see with the forum posts in this thread, so far, only the new NPs who purchased from us have indicated that they will find a new country. They had no cancellation costs to deal with, and we were shutting down those DCs anyway, so that expensive cost wasn’t factored into any buying/selling agreement, like it has been (I assume) for everybody else. As you can see on this thread, all the other NPs have has said that their servers that they’re keeping aren’t moving. Most of the buyers on here so far have said they’re not moving the nodes either.

Therefore, the position that we are making this proposal is FROM that place… knowing that almost all of the Gen-1 NPs have already committed in DC contracts to remain where they are. This proposal is only if/for the few purchasers who haven’t committed/decided yet. The two who purchased from us are part of that, and I think there are maybe two others who have not annouced if they are moving the nodes. So at most, we have 4 NPs who might take advantage of that. IMO, it would be great if we can get 4 new countries, but I doubt all four will. And this isn’t “unfair” to not announce this earlier because, like I said, I had asked many of them a very long time ago if lesser decrease in rewards would incentivize them to move nodes to a new country, and everyone who got back to me said no.

ETA again… I just realized what you were asking. I suppose, with the way we have worded this, if these remaining four NPs happen to choose to move to the same country, that might be an issue, because that would be a LOT of nodes in that one country. Considering the way topology works, Sven had told me that if two NPs choose the same country, the IC can still use nodes from both NPs well to increase decentralization. Our two purchasing NPs are simultaneously looking for a country to operate out of to help with decentralization, and I don’t know if the remaining two that haven’t announced a location are. But it would be difficult to tell them that they have to communicate with each other and choose different countries. We also can’t pick one of them to choose the first country, then the second to choose the next, etc. As Sven said, even if two pick the same country, it improves decentralization nicely.

But this is definitely something that I have already recognized that somehow needs to be worked out before we open up Gen-3. NPs generally have to order equipment and understand costs months before bringing nodes online. Yet voters will need to approve their locations before they order things? So how do we solve this cart-before-the-horse problem? It seems logical that if a costs and the political climate of a particular country become attractive to NPs, then we’ll have a bunch that want to set up there. How does the NNS choose who does it? How do NPs who are making plans to submit a proposal know if other NPs are about to choose the same place? How do voters know if an NP who submits a proposal months before the nodes go online is actually going to follow through?

Hope that helps!

1 Like

Thanks @katiep, that’s a lot of context. I like context :slightly_smiling_face: and I appreciate you taking the time to go through it

I suspect he’s referring to the fact that the IC Target Topology allows up to two nodes in the same country within the same 13-node subnet (and up to 3 for larger subnets). So if there are two nodes overall in one country (considering the IC network topology as a whole), there are no limitations to the subnets that they are capable of both contributing to (considering just subnet country limits). As soon as you have 3 nodes in the same new non-EU country, the subnets they can take part in are potentially limited by where the other 2 nodes are being utilised. At least this is how I interpret that sentiment.

Sounds like it might be worth introducing a second aspect to the reward structure, given that there are two things to incentivise:

  • NPs that are willing to relocate nodes in the first place
  • NPs that are keen to squeeze out some additional rewards (given that they’re moving anyway) by strategically selecting a country they’d expect few others to join

Perhaps the added complexity isn’t worth it at the moment, given the nuances and context you described. Longer term I think this could be useful though (a flat rate reward to incentivise the move in the first place, and perhaps an additional boost that decays as more nodes join the country).

Thanks @katiep, would you be able to briefly explain the motivation for this restriction?