Introducing WASI for IC

Developers
41

This is helping us to hopefully enable a huge chunk of the Python stdlib. Also wasm32-wasi I hope helps us overcome some pesky issues related to enabling randomness and time in our dependencies.

2 Likes
42

@lastmjs Thank you for your feedback. I am glad you were able to compile the project and it looks promising to you.

I think the project is almost ready. Iā€™ve managed to run the rusqlite inside a canister. There are a few polyfill functions that still need to be implemented as well as more testing and better documentation are pending. It will probably take a few more weeks to complete.

Thank you for looking into the candid issue, I vaguely remember seeing the error message some time ago, but I donā€™t remember how I fixed it. I will look into it.

For others who are also interested in trying this out:

I look forward to hearing feedback!

8 Likes
43

I have a question about seeding the randomness. If you look here: ic-wasi-polyfill/lib.rs at main Ā· wasm-forge/ic-wasi-polyfill Ā· GitHub

it is using rand::rngs::StdRng::seed_from_u64. Why are you using from_u64 and not from_seed which would allow passing in a vec or randomness like we get from the management canisterā€™s raw_rand? Thatā€™s how weā€™ve been doing randomness, and Iā€™m not sure if I can just reduce that randomness down to a u64.

Iā€™m trying to hook this up with our current implementation that uses a timer of delay 0 after init and post_upgrade to seed good randomness.

44

Yes it seems like if you use seed_from_u64 then we only have 64 bits or 8 bytes to work with, but raw_rand gives us 32 bytes, so we canā€™t take advantage of all of the randomness possible on the IC. Seems like this is weakening the security.

45

Also is it safe to call ic_wasi_polyfill::init multiple times? Weā€™re calling it once in init and post_upgrade with a seed of 0, and then once in that timer callback of delay 0 with an actual seed value.

46

it is using rand::rngs::StdRng::seed_from_u64. Why are you using from_u64 and not from_seed which would allow passing in a vec or randomness like we get from the management canisterā€™s raw_rand? Thatā€™s how weā€™ve been doing randomness, and Iā€™m not sure if I can just reduce that randomness down to a u64.

Yes it seems like if you use seed_from_u64 then we only have 64 bits or 8 bytes to work with, but raw_rand gives us 32 bytes, so we canā€™t take advantage of all of the randomness possible on the IC. Seems like this is weakening the security

Yes, you are right! It is a good idea to use from_seed instead. For now it was done using u64 for simplicity.

2 Likes
47

It is not save to call init multiple times, because we might do something that only works once in that method. I will add an extra function to be able to update the random seed instead.

1 Like
48

Okay perfect! Thank you

49

Great job @sgaflv! I have a question for you.

Recently, I managed to port Oxigraphā€™s RDF database to the IC, as you can see at ic-oxigraph. The original oxigraph library can run in JS environments using js_sys, so Iā€™m pretty sure it doesnā€™t target wasm32-wasi by default.

As you can see from the README, what basically needed to be ported were the random number generator and the Date::now() functions, and it was not that hard.

Iā€™d like to understand if thereā€™s a way to use your wasi2ic tool to make it more stable while also benefit from future updates and improvements.

As a general overview btw, I believe having RDF databases on the IC can help people create and deploy decentralized (knowledge) graphs from which multiple project could benefit.

2 Likes
50

I would love this FYI, been dealing with this problem since Sudograph in Summer 2021

1 Like
51

@sgaflv @ulan or anyone else, I canā€™t seem to get the stable filesystem to work. Iā€™ve tried some very basic reading and writing across deploys in Azle and in a very simple Rust canister and it just doesnā€™t work. Iā€™ve opened an issue in the ic-wasi-polyfill repo: Basic stable filesystem read and write not working Ā· Issue #13 Ā· wasm-forge/ic-wasi-polyfill Ā· GitHub

Please help, getting close to providing Azle with a Node.js fs implementation hooked up to stable memory.

52

Hi Jordan, sorry for the inconvenience. I think I found the problem, I am working on the fix now.

2 Likes
53

Should be working now, just take the latest ic-wasi-polyfill version 0.3.13.

2 Likes
54

Perfect! Thanks for the fast turn around, I will try it out and report back.

55

FYI so far it is working, we havenā€™t done extensive testing but the initial problem is gone. Thanks!

1 Like
56

Hey @sgaflv, ic-wasi looks great! I just tried running the demo with the Boa JS engine. This is exactly what I need to be able to run untrusted JS code for my upcoming project CATTS ā€“ Composite Attestations Engine. In this I need to allow the user to define ā€œprocessorsā€ with arbitrary code that processes and transforms some JSON data.

1 Like
57

waoo ! amazing ! Thank you for your great product!
Is this on github opensource yet ? even its in progress.

58

Hey, curious why youā€™re using Boa and also why not Azle in this case? You can just eval code in Azle.

Besides that though, I highly recommend QuickJS over Boa. We used Boa for a very long time in Azle, and swapped it for QuickJS back in August/September 2023.

Boa has various bugs (though we have also found two major bugs in QuickJS that have been fixed fortunately), is very inefficient, and is generally immature compared with QuickJS.

I especially recommend wasmedge-quickjs which is what we use to provide a lot of the Nodejs APIs in Azle.

The end goal is SpiderMonkey or V8, SpiderMonkey will probably come first.

1 Like
59

CATTS looks awesome! I was just listening to some GreenPill podcasts about it and envisioning this a very similar same service on the IC. What a joy to find it already underway!

I scanned the docsā€¦one thing I was thinking about is that with these services that have API endpointsā€¦does that break some kind of trust at any point? I wonder if it would be better to validate against a couple of rpc gateways the actual storage or function call data using EVM RPC Canister - #30 by domwoe to get better trust that it is on chain(or better yet if we had an onchain copy of ETH history we could query like the BTC UTXOs. I guess it would probably get too big for some of the L2s.

(This doesnā€™t have to do with wasi so maybe we should move to a different thread, but this is a great use case to make to the community that we need further development of these services. The IC is a perfect L2 relayer if we can get reliable data!)

1 Like
60

The short answer is ā€¦ I like Rust, for all the reasons people in general like rust. :grinning: No garbage collection, strong typing, no undefined etc. So I prefer building in Rust. But, these small code snippets, the ā€œprocessorsā€ will be written by anyone. JS in that case I believe to be the most accessible language.

Thanks for the tips regarding Boa and the other JS engines. I noticed that SpiderMonkey supports compiling to wasi, perhaps Iā€™ll give that a try. QuickJS I plan to look at as well.

3 Likes