Motoko Sharable Generics

I’m trying to generalize some watch/notify functionality into a module. I have the following def:

public type Event<X>  = {
            #Create : X;
            #Mutate : X;
            #Delete : X;
    };
public type Notify<X> = shared (Event<X>) -> ();

However, the compiler doesn’t like this

type error [M0031], shared function has non-shared parameter type

Understandable. But, I expected the compiler to instead give me an error at the time of implementing a truly non-shared “Event” for Notify. Which is likely easier said than done.

Is there a good away around this?

1 Like

Not currently, I’m afraid. We have plans to add shared generic types in the future, so that you could write

type Notify<shared X> = shared (Event<X>) -> ()

But unfortunately, that is highly non-trivial to implement for generic functions or classes (as opposed to just types), because it will require constructing and passing types around at runtime. So I fear it will take a bit longer.

3 Likes

I’m trying to perform parallel execution of an async function on each of the elements of a collection (i.e. Buffer<T>) and then to collect the results.

This would behave something like Promise.allSettled() in Javascript.

The code looks something like this

  public func mapParallel<A, B>(as: StableBuffer<A>, f: shared (A) -> async B): async StableBuffer<B> {
    let executingFunctionsBuffer = StableBuffer.init<async B>();
    var i = 0;
    label execute loop {
      if (i >= as.count) break execute;
      StableBuffer.add<async B>(executingFunctionsBuffer, f(as[i]));
      i += 1;
    };
    let collectingResultsBuffer = Buffer.init<B>();
    i := 0;
    label collect loop {
      if (i >= as.count) break execute;
      Buffer.add<B>(collectingResultsBuffer, await executingFunctionsBuffer.elems[i]);
    };

    collectingResultsBuffer;
  };

Attempting to compile returns - type error [M0031], shared function has non-shared parameter type

Note: I’ve been able to get the above code to work and compile if it is not generic, and is specific in the types of A and B.

My specific use case is that I’d like to be able to spin down multiple canisters at the same time

This means for a list of canisters I’d like to do each of the following steps in parallel, then collect the results and move on to the next step.

  1. Transfer cycles from the canisters (execute in parallel, then collect results)
  2. Stop each canister (execute in parallel, then collect results)
  3. Delete each canister (execute in parallel, then collect results)

My current approach is to do this in a for loop awaiting the spin down of each canister one by one, but if one has a fleet of canisters this could take some time to do so.

The benefits of going canister by canister instead of all at once is that if a particular canister fails, it’s easier to troubleshoot and diagnose the issue.

However, I think there’s a use case here for performing parallel async operations and providing a generic interface for doing so, especially since each of the parallel operations for these steps (deposit_cycles, stop_canister, delete_canister would be hitting the management canister, and are therefore is not dependent on the fleet of canisters to be deleted. These deletion calls are then guaranteed to eventually succeed once the requests are sent to the management canister and reach processing status.

Maybe consider pull instead of push? You can call an async function on your own canister with an identifier and then have a query function that reports size and completion of the se

A variation of this is what we are using with dRoute and it only uses one shot calls and never awaits. When the set is done we trigger the continuation. It isn’t great for programmer experience, but it is more true to the actor model that everyone is going to end up realizing is required if you want to do anything of significance with the ic. I’m working on some patterns and clear best practices for making this easier for the dev.

1 Like

This makes complete sense from an use case where one wants to update a number of different canisters and then query to make sure those updates were persisted correctly, but what about the use case where the developer wants to delete a canister and ensure that it no longer exists.

The pull case in the scenarios of parallel canister deletion would then be the equivalent of making one-shot calls to the IC Management canister, and then querying each canister’s status at some point in the future to ensure that it does not exist.

In the case of canister deletion, this involves stopping the canister first. I haven’t tested it out, but what happens if by using one-shot every 1 out of 10,000 times the delete_canister message just so happens to make it to the IC Management canister before the stop_canister message? The canister would not be deleted, I believe it would just be stopped + an error.

A query call in this case is definitely quicker than waiting for the update call, but how do I know if the update or deletion completed? The only way to know when the change has been made is to await the completion of the update calls, in which case why not just await the update calls in the first place? (unless you want to involve heartbeat or some other query trigger for checking).

The stopping and deletion is certainly a tricky situation! Hmmm.

Whoa interesting. I wasn’t aware it was possible to make async calls in parallel with the language features currently available in Motoko.

Yep, I was testing it and just got it working locally the other day - I haven’t tested it on the main net yet though.

That definitely is intended to work. That’s why we have async T as a first-class type of futures.

1 Like

Haven’t gotten a generically typed async function as one of the parameters to work. I tried removing the shared keyword from f in the example I posted above, and received the async has non-shared content type error message.

Was this feature implemented? Parametric polymorphism and async. It seems related (along with the error message).

I’m fine explicitly including the async function inside the parent function being executed, but just figured I’d bring this up in case I’m not using the correct syntax or missing something.

@icme, right, sorry I missed that part. I was only replying to @jzxchiang’s general observation about async.

I’m afraid you cannot yet write something like mapParallel as a generic function, for the reason you guessed. But if you specialise it to concrete types it ought to work. I know, that’s not very satisfying…

2 Likes