How to workaround network throughput limits for high traffic canisters?

As of my understanding from here How would Internet Identity handle a denial of service attack? - #7 by faraz.shaikh I cannot have more than 500 queries / second and 50 updates / second subnet/boundary node and here I have 2 questions:

1)Are boundary nodes 1 per subnet? If not, how do they scale?
2)What are the rate limits for between canisters calls?

Moving on from these questions, the system that I’m building requires having an actor being able to receive a lot more than 500 queries and 50 updates, somewhere between [10^2, 10^3] of those numbers and I will be having this high traffic between canisters as well, is there any workaround for the b nodes / canister rates?

Thank you!

4 Likes

Hey @0xlup3x ,

Boundary nodes serve the entire Internet Computer. According to https://dashboard.internetcomputer.org/, there are currently 12, but there is ongoing work on decentralizing boundary nodes.

I’m not sure if there’s a clear answer. Inter-canister calls aren’t limited by any boundary node limits obviously, but are limited by the actual execution of those calls. So one approach too scaling is to shard your application across subnets.

1 Like

Another approach would be to split your application into multiple canisters. This ha the added benefit of splitting up the concerns of you application, while effectively doubling the throughput and data storage.

I thought that each canister was spread across a single subnet?

I’m not sure I’m following. Yes, each canister runs on a specific subnet and in order to shard across subnets your application needs a multi-canister architecture.

@domwoe

Got it, but don’t application developers currently have no control over which subnet their application gets deployed to?

Also, there a certain benefits to deploying one’s multi-canister application to the same subnet?
Why does it matter if the canister is shared across the same subnet vs. across different subnets?

Thank you @domwoe, deploying on different subnets is actually a wonderful idea.

@icme Tips for deploying to a specific subnet you can basically do a trial and error for deploying on different subnets.

1 Like

To add to the above, inter-canister calls are only limited by execution throughput when both canisters are on the same subnet. So e.g. given very little computation per message (and very little subnet load), you could conceivably execute thousands of them per second per canister.

Actual XNet (cross-subnet) canister-to-canister messages are additionally limited by block size (4 MB) and canister queue size (any 2 canisters can have up to 500 in-flight messages between the two of them). But considering that message size is on the order of 70 bytes plus payload, you could again squeeze quite a lot of them in a block.

(We had some idle chats about not putting message payloads into blocks, only headers and payload hashes; and relying on the gossip protocol plus some additional logic to ensure the payloads are all available to a majority of replicas. But no one has yet spent time thinking it through.)

If you want to go beyond the numbers above in the near future (or be able to handle larger messages / more computation / moderately loaded subnets) you need to consider sharding your application across multiple canisters and/or subnets.

2 Likes

Very insightful information!! In a scenario where my app architecture is sharded over multiple subnets I suppose I would have to make my own load balancer deployed off chain so a client is redirected to an appropiate subnet as canisters calls are limited to their own subnet.

Correct me if I am wrong.

Thank you!

Not sure what you mean by “canisters calls are limited to their own subnet”. There are two kinds of messages: ingress (user to canister) and canister-to-canister. But in both cases you are calling a specific canister, so the message will be routed to the subnet hosting that canister, yes.

As for load balancing, you could either randomly pick one canister (if they are e.g. identical “frontend canisters”) or (in case your data is sharded across multiple canisters) query some index canister for which canister the ingress message needs to be routed to and send it there.

1 Like

Was indeed referring to canister-canister calls.

I would have to shard data on user demand on different subnets but I need to have an external orchestrator that constantly pings subnet canisters for load checking so I can shard it on free nets. Doing a round robin or random sharding won’t work.

maybe it does not make sense without knowing the scope of my system exactly but I don’t wanna spoil it here, haha :sweat_smile:

You shouldn’t have to host anything off-chain. Your “worker” canisters can send usage stats to an “index” canister and your client code can constantly check the index and decide where to take a new user based on that.

I would opt for the second option here. However, 500 query requests/sec for an index canister (i.e. your whole application hits this endpoint a lot) is not nearly enough throughput. Is there an option for a case such as an indexing canister that is read-heavy to be able to specify a higher replication factor (or specific read replicas) so even if update calls are slower on the index canister (b/c more canisters need to reach consensus), there’s a much higher read throughput?

Canister-to-canister calls are not limited to the local subnet. Any canister can transparently call any other canister. The only difference between subnet-local and XNet calls is the additional latency and throughput.

You can run way more (as in orders of magnitude more) than 500 queries per second against a single canister, especially if we’re taking the “which canister hosts data item X” sort of calls.

Queries don’t need to go through consensus, so they’re handled very much the same as your average request on a Web 2.0 service. I.e. you’re only limited by available CPU, memory and bandwidth.

This came up because this is the rate limit an individual boundary node applies to a given subnet.

In my opinion, this problem should not be considered by the business side. It should be the performance expansion that the bottom layer of icp should support. Officials and others say that the business side needs to expand the container to handle the business. I want to know this logic Is it correct? This kind of problem does not need to be considered by the business side under other blockchains.

If the business side needs to maintain the container and the performance, then is the concept of icp upside down? Shouldn’t it be to let the business side get rid of the troubles of operation and maintenance?