How does canister state change when processing multiple messages that await inter-canister calls?

[Severin]

It’s probably easiest if I explain first how my mental model works and then us that to answer the questions.

My mental model:

• Canisters are single-process, but multi-threaded programs
• Every await is a call to thread::yield
• Yields can only happen where an await is placed
• Any state change gets committed when await is called and is visible to every other thread from that point on

@icme

Questions for Scenarios 1-3:

1. As the update calls are processed (in batch?), is there any leakage of the funds variable between update calls due to batch message process interleaving? Does it matter if the funds decrement happens before vs. after the await state commit point in the withdrawFunds() function code?
2. Are there any changes to question 1) above if funds is not a primitive, but an object say Map<UserPrincipal, Funds> and each of the incoming calls is operating on a different user principal’s funds?

1. Yes, state changes are visible to the other messages and permanent unless undone explicitly. The order matters a LOT. I’ll show below with an example.
2. Data type does not matter (unless the data type requires you to switch to async operations)

@icme

Now let’s say Canister A also has an update API appendText(), that takes in a text and appends it to a string.

If I make 5 calls to withdrawFunds() and 5 calls to appendToText(), will the message execution of these calls be “smartly” interleaved knowing that these functions touch separate state, or will each message wait for the previous message to be processed before executing?

AFAIU consensus queues a number of update calls in a fixed order to be executed in a round. Those updates are then run one after another up to the point where an await is called (or the function returns). Then this update call’s execution gets paused, the canister call is put in the right message queue, and the next update call starts executing. Under some special circumstances the awaited call can run in the same block, but you should never rely on that to happen.

So are the calls ‘smartly interleaved’? Maybe if they contain awaits. No state analysis is done AFAIK.

Now for an example how you should handle this. I’ll go with pseudocode to make things easier to write. Take the following code how you should do things:

[1] func withdraw_funds(amount) {
[2]   if(balance >= amount) {
[3]     balance := balance - amount
[4]     result := transfer(amount).await
[5]     if(not result.is_ok) {
[6]       balance := balance + amount
[7]     }
[8]   }
[9] }

What happens if you…

• Remove condition on line [2]
  • User can withdraw any amount they want, even if no funds should be available
• Remove lines [5] through [7]
  • If the transfer fails, no funds are transferred, but they are not in the user’s account anymore.
• Swap order of [3] and [4]
  • If user calls the withdraw function twice with e.g. amount == balance, the first invocation will start the transfer, yield execution, and the second invocation will again send out the funds even though there should be none left to send out. User can withdraw any amount, but needs to time the calls appropriately

Another way to mess up, inserting line [2a] and changing line [6]:

[1] func withdraw_funds(amount) {
[2]   if(balance >= amount) {
[2a]    original_balance := balance
[3]     balance := balance - amount
[4]     result := transfer(amount).await
[5]     if(not result.is_ok) {
[6]       balance := original_balance
[7]     }
[8]   }
[9] }

This introduces a race condition where one invocation may accidentally undo legitimate/important changes made by another invocation