Node compensation and runaway massive inflation

On the opex cost for an ongoing hosting concern:

I calculate ~$1500 per node. Here’s the rough cost breakdown (generously)

Assumptions:
A1-> node capex: $30000
A2-> CAPEX Amortization Period: 3 years
A3 → CAPEX INTEREST APY: 10%

OPEX COST (PER MONTH): $1500 (APPROX)
NODE PRINCIPAL + INTEREST = $1000 approx
Hosting Cost (50% of P&I ) : $500
INCLUDE RACKING/STACKING, POWER/COOLING, REMOTE HANDS ETC

1 Like

On the plan to 1300 nodes from 500 nodes. I am , bluntly speaking, AGHAST because no appreciation has been given, seemingly, to the load predictions.

A doubling of capacity or more WITH ZERO EVIDENCE of increased compute capabilities needs (followed with detailed calculations) would be LAUGHED OUT at the manager level ; leave besides getting to the executive level; in a normal setting.

This seems like a very expensive science project.

The team REALLY needs to justify the need to the community to MORE than double the number of nodes.

2 Likes

Why Node Fees Are So Expensive

Thanks for providing this information.

NODE PRINCIPAL + INTEREST = $1000 approx

What does this item mean? Does this refer to the node rewards given to the node principal or some cost I’m not aware of?

Typically when you buy an expensive server, you take out a loan which causes you to pay interest on the loan.

So my $30000 server amortized over 3 years =~ 833 per month.

Now the interest on that amortized amount = 0.1×833 = 83

bumping the loan amount to = $916 / month (approx $1000)

This has nothing to do with rewards. Merely the cost of acquiring hardware & repaying the loan on a monthly basis. Hth

Edit: this is exactly why all profitable node providers MUST IMMEDIATELY liquidate their tokens(ICPs or otherwise)…otherwise they do not have a viable business.

1 Like

Yeah, that’s very helpful! thanks.

(A) lowering the rewards being provided per node. $4000 per month is very generous imo.

Actually, based on this I think that estimate may be a bit high.

It seems that monthly USD rewards per node range from $1169 to $3248 depending on where you operate the node. Your monthly operating expenses of $1500 may be too little, too much, or just enough… depending on the region of the world you run the node in.

1 Like

True that cost would vary regionally.

However recall that IC node providers have little wiggle room on the type of hardware they source. So their opex cost of acquiring a server remains within a narrow range.

Because virtually all of the nodes are hosted in the “western” world, the expense of hosting it would also not vary drastically.

This is not consistent with 100000 ICP being provided at a estimated $20 ICP to all (500) nodes. I.e. 2m/ 500 = $4000…unless i am missing something.

Edit: Unless we are paying the 800 IDL nodes ;which would be even more crazy. Because, in that case , we would be paying for capacity that we already had but not deployed

Hmm, I’m getting roughly the same numbers as you. (Actually, it’s a little lower since the price of ICP was closer to $17 with the most recent April node provider payments, I believe.)

@diegop Would you happen to know?

I think two aspects deserve clarification:

  • The IC is already rewarding (most of) the 1’300 nodes we plan to add until EOY, actually all except for DFINITY’s that are not yet deployed. So the rewards in XDR will actually increase only slightly until EOY. (The reason for this is bootstrapping: the IC needed a guaranteed supply of nodes in the beginning; the nodes all exist, they just haven’t been onboarded.)
  • The rewards are rather generous at the moment, which was necessary initially because the NPs were taking significant risk by investing in a project before launch. (They’re not outrageously generous, they just factor in the significant risk of investment.) So I expect lower rewards for future generations of nodes, where risk is considerably lower.

This is the background for the numbers I gave above, and for me not being worried about this at the moment, despite the fact that theoretically this spiral exists.

9 Likes

Thanks for the clarification.

A couple of questions:

(i) If we had already purchased the capabilities of 1300 nodes, why didn’t we deploy all 1300 nodes? Seems a waste of tremendous money ; given that we provided generous rewards for nodes to essentially do nothing for a year. Specifically what does it mean for "the nodes to exist "? That they are simply racked in the data center, not powered on?

(ii) Now that we are onboarding the legacy 800 nodes, i am assuming that they will NOT be conforming to AMD-Milan. So when we say "[quote=“Luis, post:68, topic:9170”]
to ensure that the network is growing with nodes supporting SEV-SNP attestation.
[/quote] ", are we to presume that that growth will be in addition to 1300 nodes?

(iii)For how long are we liable for the cost of first generation nodes? These are the MOST expensive (counter to Moore’s law). Is there a plan to phase them out? Given Moore’s law, we should be replacing these nodes with next generation nodes.

Edit: see answers here (The State and Direction of Decentralization & Nodes on the Internet Computer - #90 by Luis) by Luis.

2 Likes

The first thing we need is clarity about the figures.

  • Official clarification of the node rewards including QoS adjustments in a clear way.
  • Dashboard and spreadsheet showing actual node rewards per day on a per subnet and per node basis so they can be analysed.
  • Dashboard and spreadsheet showing actual cycle and ICP burn per day rather than aggregated. This should be correlated with usage statistics.

The next thing we need is a model burn vs usage as bounded by subnet capacity

  • That is at what level of usage does cycle burn exceed node provider rewards for a single subnet? (Model of different typical applications)
  • Is it feasible for a subnet to become net profitable before performance degrades.

then we should adjust economic parameters and manage capacity to try to ensure that subnets are “net profitable” most of the time By this I mean cycles burned denominated in SDR > node rewards denominated in SDR.

(1) adjust pricing

  • Change pricing of computation, messages and storage and so on in order to ensure that subnets can be net profitable before they are at capacity, while still offering good value to developers. (This can be done in conjunction with a general review of pricing aimed at preventing spam and DoS attacks. It could also be non linear with prices increasing as a subnet becomes congested)

(2) Have a rational approach to managing capacity increases and decreases

  • New subnets should be added when they are needed to cope with usage increases and spikes. However we should ensure that existing subnets are mostly “net profitable” and that burn exceeds node rewards on average.
  • Consider ‘fee market’ type approaches to managing short term peaks in demand. That is it is better for costs to increase during a short term spike than for a subnet to stop functioning altogether.

(2) Consider adjusting how nodes are paid

  • Consider variablize costs somewhat by aligning rewards more closely to usage and QoS performance.
  • Consider making payments continuous based on an automatic in-protocol method so they use the same exchange rate as cycles rather than as a lump sum proposal based on average ICP price.
  • Consider adjusting overall incentives.

Note: the dependancy between (1) & (2). Subnets should be profitable in normal circumstances and certainly well before that threshold for spinning up a new subnet is reached. However onboarding nodes takes time so the threshold at which subnets are added should be chosen so that there is sufficient time to onboard more nodes and spin up a new subnet. We should therefore consider ways in which we can shorten the lead time and make it easier for nodes to be “on stand by” as well as ways of forecasting future usage.

Finally we should come up with a credible plan for the network to become deflationary “ultrasound money”. That is we should develop a model of when usage and subnet numbers will rise sufficiently for the network as a whole to become deflationary with and without taking NNS rewards into account, and a have a strategy for achieving this. This may possibly include reducing NNS rewards but it would likely be hard to get that past the NNS even though it would be in better from a tax point of view as capital gains are typically taxed less than income.

2 Likes

indeed perhaps they see data that we are not made aware of, certainly i understand the need for added capacity.

according to this node providers have received about 1.6M ICP since network inception, where as in the same time frame about 72k ICP has been burned.

Correct me if I am wrong but we are spending about 22 ICP to burn 1 ICP… that doesn’t sound like a viable business model lol.

1 Like

Not all of the burned ICP (72K) are due to compute / storage cost. Most of that (72k) is due to Sonic. I think that real number range is 220-1000 ICP spend to burn 1 ICP for compute/storage.

oh dear that certainly means there is some imbalance that needs to be corrected.

how do we get more sonics :rofl:

Most of that (72k) is due to Sonic.

How do you know that? I’m curious.

looks like the nightmare scenario is unfolding $7 today … god only knows how far all this will sink :stuck_out_tongue_closed_eyes:

let’s hope we don’t need cartloads of ICP to buy bread

Regarding this mythical death spiral

Let’s say that hypothetically, the average ICP price over the next month was $3. That would result in higher node provider rewards than in the past, but the total rewards (node provider + voting) would create approximately the same level of ICP inflation that we had last summer. Even if the average ICP price was $2, total rewards would be only about 8.5% higher than they were last August.

The reason for this is that voting rewards decrease over time (see attached image), so voting rewards are significantly lower now than they were a year ago, and they will continue to decrease over time.

I won’t attempt to perform an in-depth analysis, as @Kyle_Langham so expertly does, but there are other factors at play regarding sell pressure as well, such as the amount of ICP dissolving off of the NNS. As his Supply Analysis Update article describes, we are now entering what he calls “Zone 2”, where the amount of ICP dissolving off the NNS will significantly decrease.

In summary, even if the ICP price were to drop significantly from its current level, down to where some people would be yelling “death spiral”, the total amount of ICP entering the liquid supply each month could very well be lower than it was a year ago.

8 Likes

I should have been more specific. Sonic burnt ICP for cycles to create an AMM. By Jan 2022, they burnt 33k icp.

2 Likes

but what happens when icp gets to 50 cents? 5 cents?

1 Like