Update: this case doesn’t result in a canister trapping, but it will exit with an error (can be caught by try/catch)
TLDR: Your canister can error if under 1T when making outgoing calls to a large number of canisters in parallel. This number could be larger depending on the number of calls made in parallel (I believe it is 8B cycles reserved per outgoing call, which gets refunded after the call if not all 8B cycles are used)
I was testing out making calls to a number of canisters in parallel this afternoon and kept hitting a could not perform remote call error.
It took me awhile to figure out why this error was occurring, but I finally figured it out and am posting here in case someone else runs into the issue. (Btw this specific trap error message should probably be updated )
Even though my canister had ~480 billion cycles at the time, I was making ~50 calls in parallel. As a result, each outgoing call to another canister is temporarily charged the maximum number of cycles and held as a reserve while waiting for a response. As a result, around the ~24th parallel outcall, my canister didn’t have enough cycles to pay the outgoing call reserve fee, so the canister trapped with this message.
After bumping up the number of cycles in my canister, all parallel calls succeeded.
The trap here seems to be Motoko specific. According to the IC spec, ic0.call_perform does not trap but rather returns a non-zero error code if a call cannot be made (one of the reasons could be that your canister is low on cycles).
@claudio can you confirm if Motoko traps in this case by choice? It would be better probably to propagate the error code and let the caller figure out what to do (retry, drop the call etc).
@icme can you confirm it was a trap and not an error? I would actually expect this to be reported as an error, not a trap. The difference being that state changes will not be rolled back for the error.
Since Motoko 0.8.0, users have been able to observe and handle ic0.perform_call failures by receiving and handling an error instead of trapping:
I created a little Motoko example here:
function test(100) issues too many calls and exits with an error (reject code 4), not a trap.
function testtry(100) issues too many calls within a try catch and catches the error with Motoko variant #call_error that you can use to detect such failures.
If a local #call_error is unhandled, it gets propagated as an IC reject code 4, with the message text that you are observing.
@dsarlis I think my main qualm here was the nondescript error message. Would be nice instead if it was something like "insufficient reserve cycles - could not perform remote call"
Here’s a proposal that would help with providing better error messages. I can’t say it’s top priority but we’ll try to push it forward opportunistically.
The latest Rust CDK (v0.18.0) now returns a dedicated error variant InsufficientLiquidCycleBalance if the caller doesn’t have enough cycles to perform the call.
)