∞ Day - Origyn Motoko Gift 3 - Kusanagi -or- What have you done to my beautiful language?

Today is ∞/∞/2^3 - Infinity Day - Origyn has 3 Gifts for the Motoko Community!

Gift 3 - Kusanagi - a little language that compiles to Motoko - It’s just motoko

@rossberg , @claudio , @matthewhammer - Please don’t kill me.

We all know what a great language motoko is. It is designed with the IC in mind and makes writing clear smart contracts with a high level language more possible for beginners than RUST.

…but sometimes…it is hard…and verbose…and takes up a lot of room on the screen.

And since the compiler is written in oCAML it can be very hard to contribute to.

Kusanagi is an attempt to fix that.

Back in 2009 a little language called coffeescript emerged to try to expose the “gorgeous heart” of javascript. It pioneered things like fat arrow, the existential operator, deconstructed arguments, and a ton of other features that eventually made their way into ECMAscript. CoffeeScript has faded from memory, but if you were one of the folks privileged enough to use it, you felt like it made you a faster, better developer(and it can make sure you produce lint-able, best practices code).

Kusanagi is CoffeeScript for Motoko. It is a significant whitespace, no semi, no braces, easy-to-add syntactic sugar engine that transpires into motoko. It is ultimately just motoko and in fact, you can mostly just start with motoko and start taking things out and it should transpile nicely.

What can it do?

Handle nulls a bit cleaner - null soaks:

let x = item?.child?.thing

to

let x = do?{item!.child!.thing};

Adds the take operator

let x = take item?.child?.thing, return #err("was null!")

to

let x = switch(do?{item!.child!.thing}){case(null){return #err("was null!")};case(?val){val}};

Adds the match operator

let x = match(aResult, #ok(aResult), return #err(debug_show(aResult)))

to

let x = switch(aResult){case(#ok(aResult)){aResult};case(_){return #err(debug_show(aResult))}};

Adds Fat arrow functions

let addOne = (x: Nat) : Nat =>
      x + 1

to

 let addOne = func (x: Nat) : Nat {
      x + 1;
    };

Adds cleaner object definition:

let x =
      item = "1"
      item2 = take item2, "default"

to

let x = {
      item = "1";
      item2 = switch(item2){case(null){"default"};case(?val){val}};
    };

Back ticks let you put traditional motoko in:

module
    let x = "1"

`
    let y = "2";
`

The fat arrow and null soaks were added in the last 24 hours based on feedback from the community. So the hope here is that we can move faster to make the language more effective…and then good ideas can be migrated to motoko proper.

Oh…and the parser and transpiler are all written in a form coffeescript called hera which is a peg.js competitor so js devs can jump in and try adding features themselves(a little parser framework upfront work will be required…but you don’t have to learn oCaml). I’ll let the dev describe this more formally.

Here is the take definition:

"match" ExpNullary:exps ->
    var exp, pred, def;
    if (Array.isArray(exps)) {
      if (exps[0] === "(") {
        exp  = exps[2][0];
        pred = exps[3][2];
        def  = exps[4][2];
      } else if (exps[1] === "(") {
        exp  = exps[3][0][0];
        pred = exps[3][1][0];
        def  = exps[3][2][0];
      }

      if (exp != undefined && def != undefined) {
        return ["switch(", exp, "){case(", pred, "){", exp, "};case(_){", def, "}}"]
      }
    }
    return $skip

All of the work to this point has been done by @DanielXMoore with a bit of funding and guidance by the Origyn Foundation. We now want to open it up to the community as you all will be able to pound on this much more efficiently than we can.

You can try the language out at: Kusanagi

File issues at GitHub - DanielXMoore/kusanagi: CoffeeScript style syntax for Motoko language

There are certainly some bumps in it but we feel like we’re at about v0.9.0 and you all can help us get docs built(hopefully we can get to something like https://coffeescript.org/), edge cases covered(porting the base libraries might be a good start), and integrated with a build pipeline.

We hope we can hand this over to the community to improve and maybe pick up a DFINITY Grant for Daniel to increase his ability to contribute and move things along.

21 Likes

:rofl: :rofl:

But in all seriousness :purple_heart: optional chaining (null soaks). Does it infer the new coalesced type? (i.e. opt typeof thing). That would be some great syntactic sugar to not have to nested pattern match all the way through.

1 Like

Of course not!

With Azle coming out, there has been an adversarial narrative brewing on Twitter I’ve noticed recently, and to which I have even recently responded.

To reiterate, I think the more languages we have in the mix, suiting people’s existing tastes and existing cognitive frameworks, and helping all of us evolve together, the more we are delivering on the big promises of web3 via the IC. It’s a story about interoperability, interdependence and partnership, not of singular dominance.

Having said that, I have concerns and questions. Looking forward to discussing more.

My biggest concern – How are type errors from Motoko communicated meaningfully back to the Kusanagi programmer when the transpiler tool produces an ill-typed Motoko program from an Kusanagi input program?

To step back, Motoko, Wasm and Candid each have their own type system that is as much a part of its “core language design” as its concrete and abstract syntax, where all are actually somewhat co-designed.

What I call “a typed language” is a fairly high bar, but given that we are talking about Motoko (and in the context of Wasm), I think that high bar is warranted. In particular, Motoko programs enjoy a type system that prevents Wasm type errors from arising in well-typed Motoko programs.

So, for this tool to be considered a language in this more rigorous sense of " typed programming language", it should also have a way of checking programs before it transpiles them into Motoko, and should also avoid Motoko output programs with type errors. (And if it’s not a “typed programming language”, fine, but again, how are type errors from Motoko communicated meaningfully back to the Kusanagi programmer?)

Until there is a type system for Kusanagi surface syntax, could we all this tool what it currently seems to be: A “token-based preprocessor”, similar to what the C preprocessor is for C?

Please (please!) correct me if that’s mistaken, but that’s my impression from glancing over the source briefly.

8 Likes

Great question! We’ll need some kind of language server that is running in the background and transpiring the thing. I’m pretty sure this already existed for coffeescript, so hopefully we can follow the pattern and plug in the same flow so that a transpiler line error can be highlighted on the proper .moku line. @DanielXMoore Do you know of a pattern we can use here?

Until there is a type system for Kusanagi surface syntax, could we all this tool what it currently seems to be: A “token-based preprocessor”, similar to what the C preprocessor is for C?

Pardon the vocabulary…I think because coffeescript didn’t have types they were a bit liberal with “language”. This thing is a transpiler and not a full language.

I think that is fair…this thing produces Motoko and if the Motoko doesn’t compile, you get no wasm. Now there is certainly an issue where we could say what if it produces the WRONG motoko. We need more tests for sure. There is a limited set of translations, so hopefully, we can get 100% coverage.

5 Likes

As an aside…this came out of some discussions about doing a typed coffeescript on the coffeescript github repo.

2 Likes

Thanks for the introduction @skilesare!

The current implementation is a very thin parser that inserts implied braces, parentheses, semi-colons, etc. The take, match, and null soak features work by adjusting the token trees directly. If anyone has questions about the architecture or would like to contribute feel encouraged to ask me and I’ll explain as best as I can. Even suggesting a feature or providing an example of something that parses incorrectly is useful.

The current workflow requires .ku -> .mo -> .wasm. The types would be passed through exactly as written. I’ve kept the same type annotations in Kusanagi as Motoko, the goal being 1:1 congruence. To create the .wasm the transpiled Motoko source needs to be compiled as well. As for reporting errors, syntax highlighting, and IDE support no work has begun there yet but it’s definitely something I’d like to do if there is interest.

I hope you enjoy Kusanagi!

3 Likes

So does that mean I cannot use take as a variable or function name then? Bummer.

What happens if a programmer imports and uses List.take, or a similarly-named function, from a library?

Presumably something goes wrong, but I wonder what kind of error? Parse error, or some strange error from a mistranslation, depending on how take is being used?

Here’s a small working example.

Same question for the other new operators that arise as new keywords (match, etc.)

Discussing with @rvanasa, we have another syntax proposal for the take operator, which we agree is useful.

// short circuits the `let y` line when foo() is null and 
// the return sub-expression runs, skipping other code later.
let x : Nat = (baz.foo() : ?Nat) !or return 42 

// uses the value 37 when is null, does not skip later code.
let y : Nat = (baz.bar() : ?Nat) !or 37

// rest of code, blah blah blah
let z : Nat = ...

Rationale – What makes these operators strange and a bit tricky to reason about is that they alter the control flow of the program during an error. To make that behavior more obvious, it makes sense to reuse the syntax for or, which already does “short circuiting” and affects control flow. We are using !or for this operator that attempts to unwrap an option, and runs the right-hand-side only when this operation fails.

2 Likes

This is an interesting project and I’ll definitely keep an eye on it. For now I’m not completely sold on it though, there are a couple things I don’t like, namely: no curly braces, whitespace and take syntax, I think Kusanagi’s potential could have been much bigger if it aimed to be the Typescript of Motoko rather than Coffescript.

Happy to change it if we have a better option. It just made sense to me. If you need a take I think you can do take( and the parser will leave it alone.

let x : Nat = (baz.foo() : ?Nat) !or return 42

This is awesome as long as I can replace 42 with return #err(“thing”) and not have return always called. That was an issue I was having with a number of other constructs. In the past that thing on the other side of the or was always executed and only if the first was false.

func (y : ?Text) : Result.Result<Text, Text>{

    let x = y !or return #err("cant be null");  //with just or the return statement runs always and the function is always #err("cant be null")

   return #ok(x);
}

3 Likes

You’ll be happy to know that you can generally use braces and semis if you want to.

Semis are not an issue, I like them being up to the dev like in JS.

Any examples on when they can’t be used?

Also just to be more specific on what I meant, I think this could be more succesful it it were a superset of Motoko rather than a similar but different language that gets transpiled. It’d help a lot with adoption.

I could understand if there were some design choices in the Motoko syntax so bad you’d not want them in Kusanagi, but in the current implementation if you add curly braces back, Kusanagi is pretty much Motoko with added syntactic sugar, now I don’t know what’s in store for the future but whitespaces and no curly braces isn’t enough to justify such a choice imo. And yeah I guess you could use the backtick, but having 2 similar but different languages in the same file feels weird and leads to fragmentation, so again I’d reconsider this choice unless there aren’t good reasons behind it.

I guess my main question would be: why invent ad-hoc syntax extensions when a library function can do almost the same? For example, isn’t take just Option.get?

4 Likes

I think it is very close to this. It is almost completely backwards compatible. It just gives us a place to try new features out quickly.

Example:

Paste the Blob.mo from base into Kusanagi and it transpires without changes:


import Prim "mo:⛔";
module {

  /// An immutable, possibly empty sequence of bytes.
  /// Given `b : Blob`:
  ///
  /// * `b.size() : Nat` returns the number of bytes in the blob;
  /// * `b.vals() : Iter.Iter<Nat8>` returns an iterator to enumerate the bytes of the blob.
  ///
  /// (Direct indexing of Blobs is not yet supported.)
  public type Blob = Prim.Types.Blob;

  /// Returns a (non-cryptographic) hash of 'b'
  public let hash : (b : Blob) -> Nat32 = Prim.hashBlob;

  /// Returns `x == y`.
  public func equal(x : Blob, y : Blob) : Bool { x == y };

  /// Returns `x != y`.
  public func notEqual(x : Blob, y : Blob) : Bool { x != y };

  /// Returns `x < y`.
  public func less(x : Blob, y : Blob) : Bool { x < y };

  /// Returns `x <= y`.
  public func lessOrEqual(x : Blob, y : Blob) : Bool { x <= y };

  /// Returns `x > y`.
  public func greater(x : Blob, y : Blob) : Bool { x > y };

  /// Returns `x >= y`.
  public func greaterOrEqual(x : Blob, y : Blob) : Bool { x >= y };

  /// Returns the order of `x` and `y`.
  public func compare(x : Blob, y : Blob) : { #less; #equal; #greater } {
    if (x < y) { #less }
    else if (x == y) { #equal }
    else { #greater }
  };

  /// Creates a blob from an array of bytes, by copying each element.
  public let fromArray : [Nat8] -> Blob = Prim.arrayToBlob;

  /// Creates a blob from a mutable array of bytes, by copying each element.
  public let fromArrayMut : [var Nat8] -> Blob = Prim.arrayMutToBlob;

  /// Creates an array of bytes from a blob, by copying each element.
  public let toArray : Blob -> [Nat8] = Prim.blobToArray;

  /// Creates a mutable array of bytes from a blob, by copying each element.
  public let toArrayMut : Blob -> [var Nat8] = Prim.blobToArrayMut;

}

For Array.mo I had to change one place where func i {} was valid…and we can likely get that fixed ← @DanielXMoore motoko-base/src/Array.mo at master · dfinity/motoko-base · GitHub

No, because

let x = Option.get(thing, return #error("it was null")); 

instant returns every time and will not let the program pass.

Basically @quint told me that my switch statements looked horrible in our origyn Nft. We spent some time trying to figure out how to make it pretty, but just couldn’t figure out a way. That’s how we ended up with this.

1 Like

Sure but I assume full backwards compatibility isn’t one of the goals for Kusanagi considered the existence of backticks.

That’s great and might be one of the best ways to indirectly improve Motoko’s syntax, just think that giving up full backwards compatibility only to remove curly braces doesn’t seem worth it, if you have some features planned in future releases that might break it or want to keep the door open just in case then that’s fine too, just wanted to understand the thought process behind the choice.

Do you have plans for publishing this project as an NPM package? It looks like kusanagi is currently available!

I think you can do this

let x = switch (thing) { case (?v) v; case (_) return #error("it was null") };

well, less elegant but still possible

Well…the goal was I wanted to add take and match and didn’t know oCaml so this was the result. I didn’t target backwards compatibility but @DanielXMoore has done this a couple of times and he prioritized it. Historically, coffeescript was pretty rabid about “coffeescript is JavaScript” and I’d imagine we’d keep the same here. It is one of the reasons why coffeetypescript os taking a while…everyone wants to keep js compatability.

“Kusangi is motoko” and we’ll try to keep it that way.

1 Like

Yes…and that is all over our code. Switch statements everywhere.