Holding onto motoko promises: ill-scoped await

Developers Language Support
1

I’ve run into this a few times and I can’t find much on the forum or docs so I thought I’d raise it.

I have some code like this:

      let stopPromise = systemCanister.stop_canister(stop_args);

        ignore Timer.setTimer<system>(#nanoseconds(0), func() : async () {
          try {
            await stopPromise;
            req.stopResult := ?true;
          } catch (e) {
            D.print("Stop canister error: " # Error.message(e));
            req.stopResult := ?false;
            
          };
        });

I want to start an async call and go ahead and return quickly(in this case I may be processing many other parallel executions on other canisters) and then I want to make sure no error occurs later on so I have this timer, which I kind of hoped would form a closure and keep the promise around.

The compiler gives me:

ill-scoped await: expected async type from current scope $@anon-func-232.56, found async type from other scope $installProc_start_install
scope $@anon-func-232.56 is lib.mo:232.74-241.10
scope $installProc_start_install is lib.mo:218.128-249.6

It seems I have lost my scope and can no longer await this.

If I want to add the promises to a buffer in my enclosing function I can await them all later without losing scope, but in this case I’m using the generic TimerTool framework and each possible call is getting its own function call and scope with an orchestrator determining how many we can run in parallel.

I see the obvious issue here I’m creating a closure and this could lead to some kind of memory swell if the canister gets backed up with calls or if for some reason the calls never come back (should be impossible?).

I’d much rather stick the promises in a global queue and deal with them all at one time, but that gives me this same kind of scope error.

Is this kind of functionality off the table for technical reasons or because it is complex to implement? Or have I made a bad assumption or syntax error somewhere?

Logically, I see that my timer has not been committed to state because I don’t await a timer creation so I see that I wouldn’t be guaranteed to have this scope in the future if my function traps. Is there an awaitable timer that could create a permanent, guaranteed scope? Maybe I’m the wrong path here?

cc @claudio @ggreif

2 Likes
2

Motoko tries to enforce the condition that you can only await a future in the call that created it.

The problem is that the IC, when returning from a call, doesn’t let you indicate which call you are completing. This meant that if you await a future and then return, the future that you are awaiting needs to have been created in the same call as the await, otherwise the code after the await might wind up completing a different call, the one that created the future, not the one that await it.

Here’s a contrived example that illustrates what could go wrong (if it were allowed by Motoko):

actor Wrong { 

   var o : ? (async ()) = null;

   public func writer() : async None { // never returns, right?
     o := ? (async {
       return; // invokes continuation stored in future
     });
     loop { await async {} };
   };

   public func reader() : async () { // eventually returns, right?
     switch o {
       case null { await reader(); };
       case (? f) { await f; return; }; // stores “reply to caller” in f    
     }
   }  
}

In this code, a call to writer() creates a future and stores it in o and then enters an infinite (asynchronous) loop (calling itself), and should never return (coz it’s looping right?)

reader() on the other hand, enters a loop waiting for o to contain a future, and when it it does, just awaits the future and then returns.

If you call writer() and reader() in parallel, then, naively, you’d expect writer() to never complete, but reader() to eventually return, but instead, writer() returns and reader() fails to reply.

Here, the problem is that the continuation (the code to execute after the await) stored in f (i..e. return), winds up getting executed in the wrong call context, the call context of writer() that created the future, not the context of the function (reader()) that awaited.

Motoko’s type system prevents this by scoping futures, ensuring that you can only await a future in the same call that created it. Under the hood, this is done by assigning fresh scope parameters to every async body and using those to tag the future’s created withing the async’s body. The body can only await futures with the same tag (i.e. scope). Async functions are generic in the scope, so, when used in particular scope, return an async tied to that scope.

We don’t reveal this mechanism to users, other than when reporting violations. But the mechanism is what ensures that you don’t wind up replying to the wrong caller when you do an await.

(BTW I’m not sure the Rust cdk prevents this class of errors, but it might do as a side-effect of Rust’s lifetimes.)

Here’s the Motoko code in Ninja, with scope violations reported:

3 Likes
3

Interesting!

I may be going too deep here, but what is the ‘handle’ that allows one scope(the current) to know which return goes with which call, but is not there later once the scope is destroyed? What keeps you from creating a closure and keeping the memory/context around until something comes back with a some handle? (obviously, a loop here is going to cause you to run out of memory faster).

I wonder how a closure-based language like Azle is handling those(cc @lastmjs). One thing js/ts was always really ‘good’ at was forming closures and keeping them around.

1 Like
4

Actually, the problem is precisely that the IC doesn’t really provide a good enough handle to signal a waiting action to be resumed in the correct context.

In other systems (like an OS), the awaiter would be a thread waiting on a condition variable and woken when the condition is fulfilled or has changed, but the IC doesn’t support that at the moment.

Instead you can just store the code to execute and resume it on the signalling call context, not another signalled thread.

I guess you could argue Motoko is ensuring the signalling “thread” is the same as the “signalled” caller.

1 Like