I for one would love to write Motoko programs that Don’t need the IC system APIs…or at least that have the details abstracted away into a runtime that lets me do strongly typed async, actor-based programming on any platform and have other programs import my code and use it for other non-ic based tasks…of course, I need some way for that program to tell the outside world that something interesting happened.
8 Likes
Incorporate a build step to my command line tool to replace WASI dependencies.
Will you support the CLI on Linux, Mac and Windows?
That would be important so we can include it as a step inside our tooling which supports all platforms.
We write our CLI in Python and for now distribute via PyPI.
One important detail I’m still missing from the workflow described in the OP is this: at the end of the day, any program useful on the IC will need to interact with the IC’s system API in some way, i.e., at least be able to receive and respond to ingress messages.
I’ve been chatting with @sgaflv offline about this project. One thing that’s not clear from the description and may be a source of confusion is that the project is not about taking any WASI binary and automatically generating an IC canister out of it.
The actual goal is to allow an existing IC canister to use libraries that support wasm32-wasi as the compilation target (but don’t support wasm32-unknown-unknown). In other words, when the project is done, developers should be able to compile IC canisters to wasm32-wasi and with some post-processing run the canister on the IC. The canister would still need to use the System API to interact with the IC.
9 Likes
I think that this is great tldr.
However to make it into a useful wasm32-wasi compilation target, it seems to me that some of implementing functions would need to use underlying IC calls (such as random_get()).
So really the question is which of the functional groups (outlined above) which will be really implemented as MVP and which would left for another day.
1 Like
In case this is helpful:
1 Like
Yes, the command-line tool will be cross-platform
2 Likes
Yes, you are right. The wasm-wasi functions will be implemented on top of the System API. The plan is to implement all of the functions except for the sockets. As I mentioned in the project description, the initial implementation of the random_get() will be based on a pseudo-randomness and will not be secure.
1 Like
Thanks! I was considering a similar implementation. However, the canister might call random_get() during initialization before calls to the management canister are allowed. @ulan mentioned to me that this problem may be solved in the future by supporting raw_random() in the System API as a synchronous call.
3 Likes
Yes, I saw. However, you seem to imply that this only affects the initial implementation. But I wonder how this can be worked around at all, given that proper randomness requires an asynchronous call to the Management canister. I doubt that is going to change.
4 Likes
@ulan mentioned to me that this problem may be solved in the future by supporting
raw_random()in the System API as a synchronous call.
But I wonder how this can be worked around at all, given that proper randomness requires an asynchronous call to the Management canister. I doubt that is going to change.
The execution layer of replica gets random bits as input in each round [code]. These bits are passed to the management canister to handle raw_rand() requests. I think we could use some of these bits to provide randomness to canister message execution synchronously. Having random bits during canister installation would be useful in many cases. We definitely need to check this with security/crypto experts because I am not sure if this would break some security properties of raw_rand().
5 Likes
I’m very excited about this effort, I imagine this will help Azle and Kybra tremendously in the future as we attempt to provide support for as many npm and PyPI packages as possible.
As for the randomness, it would be fantastic to be able to retrieve randomness synchronously during init. In the mean time, we’ve overcome this issue in Azle and Kybra by setting a timer of duration 0 during init and post_upgrade that immediately retrieves randomness and seeds an rng that we hook up to our custom_random function for the rand crate to then use under-the-hood.
I’m not sure if this technique could be used for this system API functionality, but it seems to be working very well so far.
9 Likes
That seems like a great technique, especially since you say it actually works.
Do you test for edge cases that have races?
(a need for randomness before the timer-fed seeding step is done?)
We have not. Our RNG will use a seed of 0s until the timer callback executes. So I believe it’s possible that message could get in before the callback executes. Nothing would crash or panic necessarily, but those values could possibly be predicted until the seed is changed by the callback.
Makes sense.
If the initializer gives their own temp seed, one could use that and then overwrite it with the better one. Would that work? As you say, it wouldn’t be normally needed, but maybe would help cover this edge case if it were a security concern for anyone auditing the way the code works. WDYT?
In any case, thanks for for this clarification.
1 Like
That makes sense to me, if the init and post upgrade had a parameter for some random bytes you should be able to do that pretty easily.
Ooh, I think we could fix the issue for post upgrade actually by saving the seed in stable memory during pre upgrade and writing it back in post upgrade. It’s probably only necessary then during init.
1 Like
Cool! Then it’s not really an issue (since initialization only happens once, the chance of “attack” then seems very controllable and minimal).
2 Likes
Any update on when we’ll get WASI on the IC?
1 Like
Does anyone know why candid won’t compile for wasm32-wasi? Won't compile for wasm32-wasi · Issue #426 · dfinity/candid · GitHub
Seems like a problem in the num-bigint crate, but I haven’t been able to figure it out yet.
1 Like
I’ve found a workaround to the issue, but it should probably be addressed in Candid before Wasi can go to production on the IC: Won't compile for wasm32-wasi · Issue #426 · dfinity/candid · GitHub
1 Like
@sgaflv I just wanted to say you’re doing an awesome job! I have the polyfill working with Kybra and we’re compiling to wasm32-wasi which is just amazing.
2 Likes