Introducing WASI for IC

wasm-snip can sometimes help simplify things when imports show up but those code paths aren’t used.

4 Likes

The goal is to allow building canisters that contain wasm32-wasi target dependencies, but otherwise you are developing the canister by the usual canister development flow.

3 Likes

As far as I know Motoko does not compile to wasm32-wasi (it also doesn’t have FFI). My initial focus is low-level languages like Rust and C++.

There is definitely a --wasi-system-api flag for outputting to wasi. I’d say that @claudio or @matthewhammer would have more info.

If there are a few small things that could be added to get motoko into the mix it would be really helpful.

Great! Many thanks! I might have more questions when I start working on the fd_ functions.

I also don’t know if this has any crossover with what @v1ctor has done on Assigned: ICDevs.org - Bounty #34 - Wasmer Motoko - $10,000, but if it does it would be nice to build on the work that has already been done over there. I think this is likely the other side of the coin from this work, but I want to make sure you guy know about each other and can compare notes.

1 Like

Thank you for the information, I will take a look at that project and see if there is something in common that can be reused. At the moment I don’t have a clear picture how it would work in Motoko, but supporting wasm32-wasi is a good sign!

One important detail I’m still missing from the workflow described in the OP is this: at the end of the day, any program useful on the IC will need to interact with the IC’s system API in some way, i.e., at least be able to receive and respond to ingress messages. That is rather different from how programs interact with more traditional environments. What is the plan for bridging that gap?

1 Like

I for one would love to write Motoko programs that Don’t need the IC system APIs…or at least that have the details abstracted away into a runtime that lets me do strongly typed async, actor-based programming on any platform and have other programs import my code and use it for other non-ic based tasks…of course, I need some way for that program to tell the outside world that something interesting happened.

8 Likes

Incorporate a build step to my command line tool to replace WASI dependencies.

Will you support the CLI on Linux, Mac and Windows?

That would be important so we can include it as a step inside our tooling which supports all platforms.

We write our CLI in Python and for now distribute via PyPI.

One important detail I’m still missing from the workflow described in the OP is this: at the end of the day, any program useful on the IC will need to interact with the IC’s system API in some way, i.e., at least be able to receive and respond to ingress messages.

I’ve been chatting with @sgaflv offline about this project. One thing that’s not clear from the description and may be a source of confusion is that the project is not about taking any WASI binary and automatically generating an IC canister out of it.

The actual goal is to allow an existing IC canister to use libraries that support wasm32-wasi as the compilation target (but don’t support wasm32-unknown-unknown). In other words, when the project is done, developers should be able to compile IC canisters to wasm32-wasi and with some post-processing run the canister on the IC. The canister would still need to use the System API to interact with the IC.

9 Likes

I think that this is great tldr.

However to make it into a useful wasm32-wasi compilation target, it seems to me that some of implementing functions would need to use underlying IC calls (such as random_get()).

So really the question is which of the functional groups (outlined above) which will be really implemented as MVP and which would left for another day.

1 Like

In case this is helpful:

1 Like

Yes, the command-line tool will be cross-platform

2 Likes

Yes, you are right. The wasm-wasi functions will be implemented on top of the System API. The plan is to implement all of the functions except for the sockets. As I mentioned in the project description, the initial implementation of the random_get() will be based on a pseudo-randomness and will not be secure.

1 Like

Thanks! I was considering a similar implementation. However, the canister might call random_get() during initialization before calls to the management canister are allowed. @ulan mentioned to me that this problem may be solved in the future by supporting raw_random() in the System API as a synchronous call.

3 Likes

Yes, I saw. However, you seem to imply that this only affects the initial implementation. But I wonder how this can be worked around at all, given that proper randomness requires an asynchronous call to the Management canister. I doubt that is going to change.

4 Likes

@ulan mentioned to me that this problem may be solved in the future by supporting raw_random() in the System API as a synchronous call.

But I wonder how this can be worked around at all, given that proper randomness requires an asynchronous call to the Management canister. I doubt that is going to change.

The execution layer of replica gets random bits as input in each round [code]. These bits are passed to the management canister to handle raw_rand() requests. I think we could use some of these bits to provide randomness to canister message execution synchronously. Having random bits during canister installation would be useful in many cases. We definitely need to check this with security/crypto experts because I am not sure if this would break some security properties of raw_rand().

5 Likes

I’m very excited about this effort, I imagine this will help Azle and Kybra tremendously in the future as we attempt to provide support for as many npm and PyPI packages as possible.

As for the randomness, it would be fantastic to be able to retrieve randomness synchronously during init. In the mean time, we’ve overcome this issue in Azle and Kybra by setting a timer of duration 0 during init and post_upgrade that immediately retrieves randomness and seeds an rng that we hook up to our custom_random function for the rand crate to then use under-the-hood.

I’m not sure if this technique could be used for this system API functionality, but it seems to be working very well so far.

9 Likes

That seems like a great technique, especially since you say it actually works.

Do you test for edge cases that have races?
(a need for randomness before the timer-fed seeding step is done?)