what are the standards for accurately computing cost(g)?
is any direct labor included in that? overhead? legal costs to defend your node operations if a government, corporation, or data center orders some kind of shut down of your node?
“The rewards are furthermore dependent on estimated capital and operational expenses that vary based on geographies. A table with the concrete numbers follows below.”
how are you getting a “concrete figure” if the sentence before that explicitly states that the basis for the figures are some estimates you are making?
are you using GAAP for the United States and IFS for the ROW?
is there a central entity auditing the accounting records (for all node providers) that the DFINITY Foundation will contract, or is that going to vary by node providers and be their responsibility? will those costs be included in the appropriate parameter? — for the results of the audit, will the firm issuing a qualified statement suffice, or will the standards for that be the same as if the node provider had publicly traded equity in the country in which it is operating? and if a node provider isn’t able to find or contract with an accounting firm that understands cryptocurrency well enough to conduct the audit?
will a record of these audits be made publicly available? will you require node providers to somehow confirm that they are proficient with the guidelines for recording these expenses in the first place? (the source documents for transactions, and so on)
I am seriously considering becoming a node provider.
Two question:
NP/node operator setup DC, install nodes, set up the reward configuration and have them join the network.
A node provider would be setting up hardware in a Data Center somewhere, right?
When I check the dashboard, most Data Centers have 1 node provider and fewer than 2 (e.g. INAP and Flexential). Does this mean that there are few Data Centers where node providers will set up new nodes and we have to wait until a new Data Center is opened or build a Data Center by our own?
who are technically savvy and not afraid to be among the first to run the autonomous onboarding process
I am willing to do it, but I have few skills and a friend will be setting it up, he is knowledgeable but not a specialist. What level of skill is required? Also, are there any instances where the installation and setup of nodes was not completed in the end with the current support system?
Hi @MisterSignal, thanks for your questions. The costs are calculated based on the capex and opex expenses for acquiring and runinng the nodes. Overhead is included (for remote hands), but not the legal costs.
Since the IC is a decentralized network there is no contract between NPs, DCs and the foundation, nor any central auditing entity. However, it is important that the community can verify the identity and credentials of both the Node Provider as well as the DC. For now, the NP needs to submit a self-declaration form that is posted on the internet computer wiki so publicly visible. Happy to hear if you have any recommendations for verification of NPs and DCs in a decentralized network.
Different Node Provider can be in the same DC location, so you are not restricted to setting up your own DC. However, if multiple NPs set up their nodes in the same DC this will not help the decentralization metrics for the IC (the Nakamoto coefficients) as you want to have all nodes in a subnet preferably to be in different DCs. So this is something to take into account when selecting a DC for your nodes.
You require some technical expertise, in particular SRE knowledge and networking knowledge, to set up the nodes. But with the current instructions on the wiki., so far all NPs have been able to set up their nodes, so there are no cases where somebody was not able to complete this.
Thanks for the reply.
Is it possible to select Data Center as Node provider? For example, I live in Japan, but if I want to install a node in a Data Center in Tokyo, will I have to discuss this with the Data Center? Also, is the scope of responsibility between Data Center and Node Provider clarified? For example, who is responsible for the network environment of the DC, such as routers, cables, etc.?
Hi @hokosugi, yes this is absolutely possible. The Node Provider is the investor and owner of the nodes (servers), and is the only one who has access to them and is repsonsible for the availability of the nodes. The Data Center is the location where the nodes are hosted, and you as Node Provider will typically have a monthly/yearly contract with a Data Center for rackspace, networking and bandwidth.
Note that the IC already has nodes in Tokyo, so from a perspective of decentralization it will be important to have new Node Providers in new Data Centers and new countries. This is something the community will look at when voting on new Node Provider Proposals.
This from the Messari report of May 2021 seems relevant:
“The Foundation expects the number of IC data center and node operators to gradually grow post-launch to support the network’s ecosystem of apps (see below for details). The team targets for the network to reach 123 data centers running 4,300 nodes by the end of 2021.” Bootstrapping indeed.
Hi. @SvenF I would like to apply to become a node provider, but I have some questions.
If I want to add a new data centre through NNS, will there be strict criteria to limit the data center, such as tier3, soc2, and so on？
As far as I can see, there are some new node providers that still don’t have any node shown in the ic dashboard. So, is it possible to register as a node provider before the data center and machine has been identified?
When can the node provider start receiving revenue after the new node provider has prepared the data centre and machines? Previously it seemed that unassigned nodes were also rewardable. Is this still the case now for new node providers? Will Dfinity Team limit the rate of increase in the number of rewardable nodes deployed by new node providers?
Will Dfinity Team help new node providers to purchase servers and deploy nodes? Does the Dfinity Team provide one-to-one guidance to Node Providers after they have completed their NNS registration?
Hi @Shuai,
thanks for posting these questions, these are very relevant when considering to onboard as a Node Provider. To answer them:
the main requirement for a data center is that it’s location can be verified by the community once you submit a data center proposal (the second step in the onboarding process). There are currently no requirements in terms of tier3, soc2 but this might change in the future depending on what the community suggested in terms of validation of data centers.
there are some node providers in the process of onboarding, These have submitted the NP proposal (which the community has voted upon) and are in the process of procuring the Gen2 hardware and setting up the nodes. Submitting the NP proposal first and getting it approved before ordering the servers looks to be the most straightforward.
nodes are rewarded for being available in a subnet or to join a subnet. So healthy nodes that are not in a subnet also get rewarded. It is for the community to decide on the limit that is being set on nodes per NP and per country, and this is also reflected in the current remuneration proposal that has been voted upon (within limited remuneration in regions and countries where there are already quite a lot of nodes, and a higher remuneration for setting up nodes in a new region).
the IC is a decentralized network so the NP onboarding, procurement of servers and proposal submissions will need to be done by the future NP itself. Of course, fellow Node Provider and Dfinity team can advise and support but the process itself needs to become fully dencentralized.
My concern is voting on the NNS when I see a Add node operator with a proposer I do not know and always seems to add a proposal in outside business hrs which is a red flag for me and I will always vote No.
Should new nodes be only allow during business hrs?
New Zealand: Sunday
Japan, South Korea: Sunday
China, Philippines, Malaysia, Singapore, Taiwan: Sunday
India: Sunday morning, around 3-5 hours behind Australia’s eastern time zone
United Arab Emirates: Sunday morning, around 7-9 hours behind Australia’s eastern time zone
United Kingdom: Saturday evening, around 9-11 hours behind Australia’s eastern time zone
United States (New York): Saturday afternoon, around 14-16 hours behind Australia’s eastern time zone
United States (Los Angeles): Saturday morning, around 17-19 hours behind Australia’s eastern time zone
Nice work @kvic, turning my half-glib response into a world computer definition of “business hours”.
Thumbs . FWIW when working with Euro area businesses from Australia we schedule meetings around 5pm or 9pm to coincide with first half their work day. For America’s it would be our morning.
Also I agree with your concern about the lack of supporting information about node related proposals. More info and time for NNS voters to understand (even if mundane would be better
I have some questions about the requirements for a node provider.
(I’m afraid these are novice ones, but willing to ask…)
(1) Public /28 IPv4 range and /64 IPv6 range
Does this mean we need to subscribe to this number of global IPs per node with ISP? In other words, if there are five nodes in the data center, does this mean that 16x5=80 global IPv4 addresses are needed?
(2) Is there a minimum number of nodes per data center?
illustrates how DFINITY manages a typical rack of IC nodes (an ‘ICR’). It is published for the benefit of node providers to show one possible implementation of the ICR Gen2 networking requirements.
The most relevant excerpt from the latter page in the first section “Uplink configuration”:
Management Port
Assigned public IPv4 range (min /31): [FILL IN]
Default GW address: [FILL IN]
…
Production Port
Assigned public IPv6 range (/64): [FILL IN]
IPv6 Default GW address: [FILL IN]
(optional) assigned public IPv4 range (min /29): [FILL IN]
(optional) IPv4 Default GW address: [FILL IN]
…
So as I undertand it, each IC replica node uses one IPv6 address (and an “optional” IPv4 address??) to communicate via ICP with the NNS and the subnet it is assigned.
See Gen-2_Network_Requirements#ICR_LAN for the current and future network topologies within a single rack of nodes (“ICR”). The 2x10Gbe interfaces per node will be configured as a single LAG aggregate interface for link redundancy so act as one IPv6 interface.
The requirement for a “Public /28 IPv4” provides 14 usable IP addresses to assign to the nodes.
If this is for the “optional” IPv4 address per production port then the question for DFINITY to clarify is if you actually need an public IPv4 range of this size (smaller ranges of /29 or /30 are more commonly offered even if you lease a full rack) due to their scarcity.
If the “Public /28 IPv4” range is used to assign a public IP address to each management consol (the BMC/iDRAC/iLO embedded management card on each hw node) I would be concerned. I don’t like the idea of having any BMC access (i.e. protocols for the management web UI, KVM, RedFish API, etc ) directly on public IP addresses. Normally these BMC’s should be on a separate private IPv4 /28 range connected using a separate1Gbe switch (also used to access the management port on your main switch for config and monitoring). All access to this private management network should go through a secure firewalled jump-box or private VPN gateway that makes it very difficult for anyone other that the (few) node provider tech staff to access the management ports of all the hardware (servers and switches). This network access to manage hardware should be treated like the physical access required to access the hardware… lock it down, only authorised and verified individual personnel get anything near direct access.
As for your question (2) the minimum replica nodes each data centre and each node operator is allowed to host is restricted by policy voted on by the NNS and would be currently enforced by the IC-OS deployment process (??). I am not sure on the specifics but it depends largely on the decentralisation criteria in play (which is the main topic of this form thread).
The practical limit on how many nodes you can provision in each data centre rack would be about the current 28 node limit per node operator (assuming 40RU per rack minus space for the multiple switches and supporting gear you would need to run that many nodes). Again for IC decentralisation reasons an operator with the financial means to purchase (not lease) that many nodes would find splitting that many nodes across multiple racks and/or data centres is not a huge deal anyway.
@SvenF is any of the above wrong or misleading? I am not the expert here, just what I understand from detailed reading of the relevant wiki and forum pages.
Hi @icarus , I am not a networking expert as well but I think it is a good and exhaustive summary. Networking setup is very different per data center operator, so we are working on a more generic and easy networking guide to add to the wiki. For the networking, the requirement is one IPv6 /64 subnet (each node gets multiple IPv6 addresses) and four IPv4 addresses (for specific use in the future, e.g. boundary nodes). The advise indeed is to not expose the BMC directly to the internet.
Where and how many nodes to set is dependent on the decentralization rates and where the community prefers to have new nodes being set up. The limit is driven mainly determined by the benefit/costs of setting up nodes per country/region, and is reflected in the remuneration table (see Node Provider Remuneration - Internet Computer Wiki).
Thank you very much for the detailed and comprehensive explanation. There were pages on links I had not read and the information there was very helpful. I understood that the /28 (min /29) IPv4 range is necessary for future use as boundary nodes, etc. I totally agree that we should prepare a remote VPN gateway or similar for BMC access.
It is also very gratifying to know that a more general and easy networking guide will be added to the wiki. There seems to be no link to “reference design [3,4]” on the network requirements page, so it would be greatly appreciated if you could add this as well.
. FWIW when working with Euro area businesses from Australia we schedule meetings around 5pm or 9pm to coincide with first half their work day. For America’s it would be our morning.