Long Term R&D: Motoko (proposal)

Thanks for the suggestions and the encouragement.

Regarding async abstraction.

We were primarily thinking of avoiding the extra yield in examples like:

private func flip() : lazy Bool  = lazy { 
  let blob = await Random.rand();
  blob[0] & 1 == 1 
};

shared func oldCoin() : async Bool {
    await flip();  // does two real awaits.
}

We deliberately don’t want to distinguish internal from external sends, but allowing one to abstract async code into a local function sans penalty would hopefully get you closer to where you want to be.

The problem is that we chose to make every await signal a definite commit point and it’s not clear how to provide a mechanism that is flexible but doesn’t provide concurrency footguns.

+1 on error handling guidlines - many don’t realize that throw doesn’t rollback state and that traps from inner calls are propagated as rejects. There is some documentation Errors and Options :: Internet Computer but not enough detail.

2 Likes

I’d be interested to see a concrete example.

For example,

        let t : Trie.Trie<Nat32,Text> = Trie.empty/*<Nat32,Text>*/();
        let k : Trie.Key<Nat32> = {hash = 1; key = 1};
        let o = Trie.find/*<Nat32,Text>*/(t, k, Nat32.equal);

seems to work, provided you provide an annotation to resolve the overloaded numeric literals in the key value. I’ve inserted the missing, inferred annotations.

Both of those should not be out of reach, given resources.

1 Like

It seems to work for me in VSCode (after a saving the file sans errors). Both on identifiers imported from base and local libraries. Right click the identifier and select goto definition.

This only works on references to library members, not local definitions.

1 Like

Wait, if await flip() now does two real awaits, doesn’t that compound the problem? I thought the goal was to make await flip() do zero real awaits. Not sure I fully understand the example.

I think we may be talking about two different things.

My original example was for a shared (async) function to call another shared (async) function in the same canister without incurring delay from an actual yield. Basically, turning an async function into a sync function.

I think your example is the opposite: turning a sync function into an async function. I’m not really sure the benefit of that, besides being able to throw errors in private functions.

that traps from inner calls are propagated as rejects.

Oh wow, I didn’t know that. I’m guessing that applies to any “lambda” (like those passed to Array.map) that traps. Is the difference between a trap and a reject ever significant? I thought assert traps were a type of rejection (which still rollback state, unlike throws).

I haven’t tried on the most recent Motoko version but for example…

let commentsArr = Iter.toArray(Trie.iter(comments));

Array.mapFilter<(Types.CommentId, Types.CommentInfo), Types.PublicCommentInfo>(
  commentsArr,
  func((commentId, commentInfo)) {
  ...

If I try to get rid of the generic <> for Array.mapFilter, the compiler complains that it can’t infer the type of variable commentId.

But it should be able to deduce it, given that it should know the type of commentsArr. This is where showing the type of an identifier on cursor hover would be useful IMO.

@jzxchiang, Motoko uses an approach called bidirectional type checking. That means, roughly speaking, that type information can either propagate inward-out or outward-in, but it cannot usually flow “sideways”.

It is of course possible to have more general type inference, but in the presence of subtyping, that get’s complicated and potentially surprising very quickly. In particular, it’s not always immediately obvious what the “best” type for something is. In your example, e.g. the result type of the argument function, and thereby the element type of the produced output array.

There has been recent research to extend something like ML-style full type inference to subtyping, but it isn’t quite battle-tested, type errors become more challenging for users, and it’s not directly compatible with a few other features that the Motoko type system has.

1 Like

Yeah, we should probably try to optimise tail awaits if possible, like in the example you gave (assuming you meant async where it said lazy). Or other transformations that do not observably change the actor’s behaviour.

Where I would draw the line is with introducing alternative forms of disguised async. I think that would be premature optimisation in the design of the language, of the kind that causes more harm than good.

Programmers must learn that async is expensive. It’s as simple as that. Consequently, avoid infecting functions with async except where absolutely necessary. Always separate the async communication layer from compute-intensive “business” logic.

Providing fuzzy features that make await cheap in some executions but not in others just makes the cost model more opaque overall, and is counterproductive in the grand scheme of things, because it introduces unexpected performance cliffs. I also bet that many devs will be very confused by a subtle distinction like lazy vs async.

And that’s just performance. What’s even worse is that atomicity, commits and rollbacks would become similarly unpredictable. They’re already is tricky enough to program correctly without further fuzziness.

FWIW, when I think of better abstraction facilities for async, I have in mind enabling more powerful composition of futures and future-based combinators. If those can be optimised, even better, but it should remain semantically transparent.

No, that’s a different problem. The only way that would be possible – without losing scope-based encapsulation – would be some form of mixin composition on actors. But that’s not an easy feature to add.

Most OO languages, e.g. Java, work fine without the ability to put methods elsewhere. In Motoko, you can at least write auxiliary modules.

Sorry, I didn’t meant to write lazy - should have avoided copying from elsewhere but can’t edit the original.
So the example I intended was something like this (the tail call was accidental)

private func flip() : async Bool  { 
  let blob = await Random.rand();
  blob[0] & 1 == 1 
};

shared func Coin() : async {#heads; #tails} {
    if (await flip()) #heads else #tails;  // does two real awaits.
}

I still think it’s highly desirable to let users abstract code that does awaits, without taking the performance penalty of this code.

There are many examples of users using local async functions to abstract code and taking and unwittingly taking a large performance hit as a result, because the only means of abstracting code that communicates is to introduce a spurious additional await.

2 Likes

C# has partial classes than can span several files, thought this was originally introduced in order to separate generated code from user code for e.g. GUI apps.

It would be a shame if splitting your code across files, by introducing async local functions, causes you to take a performance hit. I guess that one could be solved by optimizing tail calls only, but still.

This language feature would massively help with Proposal to Adopt the Namespaced Interfaces Pattern as a Best Practice for IC Developers. Right now a bunch of logic needs to be duplicated to support legacy interfaces and the ability to abstract code into lazy private functions would fix that. Would queries be able to call lazy functions? See the pull request on the 5th post to see how annoying this is if you want two exposed functions to call these same code or have the same query at two endpoints.

@skilsare I had a look at that pull request but I think those occassaion, where you want to factor our some common code in two queries can just handled by introducing a third, local, non-async function, called from both (but perhaps you considered that and reject it).

The kind of examples I was thinking about are described here

But I’ve seen quite a few in the wild now and it pains me to tell users to inline their code.

Another example is the random_maze sample, that I had to rewrite awkwardly using a loop, not recursion, in order to be able to generate random numbers on demand without running like a drain.

1 Like

FWIW, when I think of better abstraction facilities for async, I have in mind enabling more powerful composition of futures and future-based combinators. If those can be optimised, even better, but it should remain semantically transparent.

Agreed. I think something like a JS-style Promise.all would be very useful, at a minimum.

I still think it’s highly desirable to let users abstract code that does awaits, without taking the performance penalty of this code.

OK, this example makes more sense. Actually, even if flip is public instead of private, it’d be great if no performance penalty is incurred if flip is called from another method in the same canister.

Proposal is live: Internet Computer Network Status

It looks like someone significant voted against Motoko? If that is you, I’d be curious to hear your justification for voting against this proposal?

1 Like

I know that this is a little bit late; but why would a developer want to learn a new language (i.e. Motoko) for programming IC; when you would have a portable-to-other platform language (Rust)?

I have programmed in both (Motoko & Rust) and I don’t really see a clear distinguishing killer use case for new developers to learn a completely new language which they can take nowhere else.

1 Like

That shouldn’t matter once we have blockchain singularity, right? :wink:

2 Likes

From my experience, Rust is a huge blocker to adoption. It is …opinionated?.. to a significant degree and many competent developers(me included) bounce off of it. Motoko is a bit more ‘high level’ and approachable by your run of the mil Javascript dev that wants to get into writing smart contracts for the IC.

As an aside, I’ve actually had a couple of conversations about trying to find some alignment around using Motoko across a broader set of blockchains…specifically…ETH2 once they figure out how the hell they are actually going to do smart contracts on ETH2.

5 Likes

You want to say: don't await flip();. (Sorry, I’ll shut up now. :wink: