Hi graial, congrats on your ambition.
That said, 85 MB is way beyond what compression alone will fix
The right approach here is to upload Wasm chunks and then use the chunked install: IC management canister | ICP Developer Docs
With this, you can have up to 100MiB (uncompressed). Given that you got an import error and not a module size error, the tooling you use is probably already doing that for you. So the Ruby interpreter will fit. Unfortunately I don’t know anything about Ruby’s execution model, so some of my assumptions below might be flawed.
how the memory gets persisted from run to run. for example, does the IC runtime send a progression of kill messages that increase in severity as it approaches the DTS limit?
Wasm modules that are usable on the IC don’t have a main function, they are “reactor” modules (not technically quite true because “reactor” is a Wasi term whereas the IC does not natively support Wasi. But it’s fine to think about it this way). That means that code does not simply run at all times like e.g. a webapp or some other process might. Instead, code is being run only when somebody calls a canister function (the public interface of your app). Such functions have to terminate within reasonable time though, so we have limits in place.
For web requests this should probably not be an issue. I would ignore this problem for now. Focus on whether you can compile the Ruby interpreter (or whatever you are using) to a Wasm module that the IC can accept.
The canister state between function calls is persisted. I.e., if function call 1 changes some global variable, then function call 2 will be able to observe that change (unless you use “query” in which case the state is rolled back). From this perspective a canister almost looks like a process that is frozen until a function is called, which it executes to completion and then goes back to being suspended - but the “process state” meaning its internal (global) values are retained.
There are no kill messages because the IC tracks a function’s execution time via “instructions” which basically count the number of Wasm operations that have been executed in your function call so far. If it exceeds some limit, the IC aborts the function and rolls back the state. Never mind DTS, limits, cycles and all that, it’s not important here.
do think that IC is gonna need to define a target-triplet
In the interface specification Canister interface (system API) | ICP Developer Docs , we detail what interface a canister Wasm module has to satisfy. It’s not Wasi, it’s its own standard. So any Wasm module compiled to adhere to the Wasi standard (like you are presumably doing) must polyfill the Wasi interface with the IC’s system interface. But they are not congruent. Some features are just not supported on the IC, like sockets, randomness, files, … There are workarounds for some of them though.
Ultimately though, if your runtime/interpreter relies on syscalls for e.g. sockets, it’s a non-starter on the IC (currently. We do want to bring Wasi2 to the IC, but this will take a while).
Polyfill means: If your compiler emits Wasm code that imports “some_unix_syscall_time” or perhaps “some_js_thing_y”, the IC cannot satisfy these imports. But you can link (or transform) your module using a polyfill library, which instead imports e.g. “ic0.time” and points “some_unix_syscall_time” to this import. For the calling code it looks like it’s calling the intended “some_unix_syscall_time”, but it’s just relayed to the actual import that the IC can provide. This won’t work for some things.
Anyway, that’s the reason for your import errors. The only imports we allow are the IC’s system API. In order to use those sensibly, you probably want an SDK for your language. That is: Ruby FFI (foreign function interface) bindings that wrap these system functions and some logic that inserts them in the right places. It’s a big topic…
I would try to answer these questions in this order:
- Can you compile Ruby (user program plus its runtime or interpreter) to a “reactor” conforming to wasip1? (I think you have already done this)
- If yes, can you polyfill all necessary imports or does it have requirements we’ll never satisfy? This might involve tuning how exactly the Ruby->Wasm compile emits code, check out the tooling and its options (like mitnick suggests).
- How can you use exernal “C” symbols in Ruby? Can the Ruby->Wasm compiler transform them into well-typed Wasm imports? I.e., how to do FFI in Ruby?
- Try to write a minimal SDK that lets you call a canister function (using the IC syscalls msg_arg_data_size, msg_arg_data_copy, msg_reply_data_append, msg_reply)
Any final pointers on where I might look to get more info & documentation about the IC runtime/build target?
The interface spec. IC interface specification | ICP Developer Docs
Good luck.
