"Optimizing ICP Query Calls: Overcoming Performance Bottlenecks in Fetching & Processing Users"

Developers Rust
1

I’m facing a performance bottleneck while fetching and processing 600-700 users from an ICP canister (Rust) via query calls.

  • Issue: Query calls take 2+ minutes, even though they run in parallel.
  • Approach:
    • Fetch users in 6-7 batches (~100 per batch).
    • Process 25 users concurrently per batch.
  • Problems:
    • Fetching & processing is slow.
    • More batches increase execution time instead of reducing it.

Questions:

  1. Why are ICP query calls slow despite parallel execution?
  2. Does ICP have query call limitations affecting performance?
  3. Why does increasing batches increase execution time?
  4. How can I optimize fetching and processing?
2

in frontend this is happening approach regarding how can we optimize this

3

Is it the same situation even if you try fetching via the candid UI?

4

the thing is in candid we pass one principal and its other parameter it is not returning list we wantr user account data of multiple users in frontend to filter the user with hf <1 which is taking time although it is query call

5

The current limits on query execution are as follows:

  1. There’s a limit of 4 queries total per node
  2. There’s an additional limit of 2 queries per canister per node.

With the above in mind, if you are hitting one single canister, then you cannot hope to get a lot more improvement by increasing the number of batches to > 2. Going from 1 to 2 queries you should be able to execute truly in parallel, if you use 6-7 batches as you say, then I expect that a backlog of queries is built up for this canister. If you keep doing this continuously, then it could lead into bigger delays for some queries while they’re waiting for processing in the queue.

That said, I would also not expect delays in the order of minutes. If you’re willing to share the canister id we can try to look into some metrics from our side to see if we can identify some issue.

Regarding recommendations on how to optimize, you can try to scale out to more canisters, I’m not sure if I can give more concrete advice if I don’t see more details about how the calls are made, how the batches are created and sent and so on. But as I said, I would not really expect delays in the order of minutes, that indeed looks a bit suspicious to me.

6

Fetching and processing are two differ things. In what way are you processing during a query? These would be temporary state and memory changes that would be discarded and have to be recalculated each time. Ideally, for speed you need to pre process and have the data ready to go so you can just dump it. That should be fast…and query up to 2mb of data at a time so when you get a slot it all goes in one “page read”. If you have more than 2MB of data you need then it makes sense to paginate.