The Roadmap contains the following. Do you have more information?
Locally timing out canister requests
As opposed to ingress messages, canister-to-canister messages can not timeout at the moment. This feature will introduce the possiblity for the IC to timeout canister-to-canister requests in high-load phases.
Deployed December 2022
I think this means that if a subnet does not accept messages anymore (e.g. because it got stuck), then your inter-canister call would never have received a response. Now, with locally timing out canister requests, you will (after 5 min IIRC) get an error because the request failed.
I’ll try to get someone to confirm/clarify this. EDIT: confirmed
Thank you. I am wondering how the errors are handled and whether the developer or user can be aware of the errors.
Yes, this is possible. For example, in the Rust CDK the call function results in a CallResult which will contain the corresponding reject code and message if delivery failed
Thank you. I am relieved. Do you have any further information on what would cause a timeout?
Each canister has input queues (containing ingress messages and incoming canister messages – requests or responses) and output queues (containing outgoing canister messages – requests or responses). Messages in output queues eventually get routed into streams (going to specific subnets). If the stream already has some amount of messages, a backlog may build up in the canister output queue.
This timeout feature applies only to requests (not responses) in canister output queues. If the message has already been routed into a stream, it can no longer be timed out (regardless of whether the message was received or not by the destination subnet/canister).
In this context, anything that causes a stream to become backlogged (whether it’s simply a large amount of outgoing traffic to the respective subnet; or a stalled stream/subnet) will result in messages staying in canister output queues. If a request has been in a canister output queue for something like 5 minutes, it is timed out. To the canister, this looks similar to what would happen if the request was rejected by the destination canister (different reject code, but delivered in the same way).
Thank you. I understand better now.
If a request has been in a canister output queue for something like 5 minutes, it is timed out.
For example, can I receive a CallResult after 5 minutes?
Is it difficult for developers to test whether they can actually receive CallResult?
In Rust, if you call a function that returns CallResult<R> = Result<R, (RejectionCode, String)> (copied from the cdk-rs call function) there is no way for you to use the result without handling the error case
Thanks for your reply.