We are using ic_cdk::spawn to make inter-canister calls and have encountered some unusual behavior—specifically, it seems that some of the calls to canisters are not being executed. Could you help us understand what might be causing this?
Do ic_cdk::spawn calls get dropped during canister upgrades? For context, we utilize ic_cdk::spawn within periodic timers and ensure that we re-enqueue them in the post_upgrade function.
To clarify, our periodic timer function performs two main tasks:
Calculates the outcome of the bets placed.
Informs the participants of the bet, which involves inter-canister calls using ic_cdk::spawn.
While we observe that step 1 is consistently executed, we are experiencing failures with step 2 in some instances. Any insights you could provide would be greatly appreciated.
Just about anything you don’t personally serialize to stable memory is transient wrt canister upgrades and will be cleared. This includes timers (as you seem to be already aware of), spawned futures, and the necessary bookkeeping to receive in-flight canister calls’ return values. I recommend stopping canisters before upgrading them (dfx canister stop) if you are losing canister call related continuity because of upgrades.
Thank you for clarifying. We do stop canisters before upgrading them. If we stop the canister, will the existing spawned futures still be executed, or could they potentially be dropped?
Stopping the canister will wait for any incoming calls to complete, but other things like timers may not be executed in the meantime. Since you are scheduling calls from timers, if you observe a stop-upgrade-start workflow dropping calls, it is more likely that the timers aren’t successfully being re-enqueued. A reproducible example would help a lot.