Arbitrary Code Execution guard considerations - "controllers only"?

Hey y’all.

As I continue to evolve my ruby-on-icp demo and quickstart, I find myself reaching for gems like rack and roda, so I can handle some basic requests, query params and so on.

As an interpreted language, it seems to me that installing the gems into stable memory at /data/lib gets my around the wasm size and instruction count limitations.

Put another way, the wasm holds the ruby interpreter (including rust-cdk), and the gems at /data/lib. Only pure ruby gems, nothing compiled or containing c-extensions.

One consequence of this is that That means a call to put can add code the canister will then run, so I need to protect this method call. Right now, I’m approaching this as follows:

  1. Only a controller may write. Any other caller gets a refusal (the call traps and nothing is written).
  2. Writes stay in /data/lib. A path with .., an absolute path, or anything that resolves outside it is refused.

Some questions I have that I’d love to hear some community feedback on:

  • Is “controllers only” appropriate & enough? The asset canister has its own permission model separate from controllers. Would a separate permission for installing code (granted by a controller) be closer to what people expect?
  • Adding gems at runtime voids the value of the module hash. With Installed code, module_hash and a reproducible build verify the Ruby runtime, not the app on top of it, and installing code doesn’t go through install_code, so it leaves no entry in the canister history. Does the canister history (especially/specifically install & update) play a big role in any of your canister lifecycle management? I haven’t yet looked into what it’d take to try n register my own log entries at that level, or like post addition info to install_code. Is there a convention for this already?
  • Upgrades. Installed files survive an upgrade because they live in stable memory, but icp-cli’s Candid compatibility check only sees the packed interface. Has anyone dealt with code that changes outside the upgrade path, any encountered issues?
  • Prior art. Has any other canister on a runtime loaded code sent after deploy? How is it working for you?

Between PocketIC and the quickstart, things seem to be running fairly well so far in the limited functionality I’ve verified.

Cheers!

you know, it never occurred to me to run ruby on icp. But now that you’ve accomplished this, I see the vision more clearly. thanks for sharing your work here.

Cheers, I’d say this is still mainly an academic exercise. I need to come up with new stuff to present each month at our meetups.

But I also have a reasonably tight little use-case in mind to take the thing live on. So it’s been a fun little exploration.

Interesting, if Ruby can run reliably onchain and have the code swapped out on the fly then this will be very useful for numerous things. Can I ask if you’ve run into any of the following problems? Writing Smart Contracts on ICP in Go – A First Step - Developers / CDK - Internet Computer Developer Forum

If not, I recommend testing for them.

Limiting code upgrades to controllers sound sensible, or your could make if configurable (maybe with controllers just being the default).

Adding gems at runtime voids the value of the module hash

You could expose an endpoint that provides the hash for the currently uploaded script, or something similar. Better yet, you could allow the whole script to be downloaded and read by whoever is interested.

Greetings Lorimer.

We manage to avoid most of the issues in that go thread with the interpreter approach. :wink:

The determinism (#5) concerns are handled pre-packing/release via the TDD in my CI pipeline. Each new version has a new set of challenges which highlight the new features that ship (and guard against breakage of earlier added features). There were some hairy bits that I never would have solved without AI assistance but i’d call the build system early, but mostly stable at this point.

the wasi surface (#4) is handled via the community provided ic_wasi_polyfill. If new imports come available later, I’ll look into shipping a new release. I know wasm builders everywhere are looking forward to WebSockets support.

I’ve been keeping a notebook in the repo to track how the build grows as I pull in extra stuff. Upstream ruby releases minimal and full. The most important addition from full is OpenSSL. I’m about 85% sure it’s gonna fit (barely) alongside the rust-cdk. Maybe even this week! :crossed_fingers:

Your suggestion gives me the impression that you or others might someday want to be able to put little worker scripts on such a canister and communicate with other canisters to respond.

If I get your thinking right, it seems to me that the target canister needs some basis for assessing and trusting/whitelisting the canister (that it understands might to change its contents/runtime at any time).

For establishing integrity and the chain-of-trust of the ruby canister, I’ve got a few thoughts. I’d love to hear your impressions.

What the IC already covers. My quickstart has a packer task which packs the base-application Ruby files into the prebuilt base Wasm released by ruby_wasm_builder. This is what needs to fit inside the 12Mb limit. So both of those are hashed and available for inspection.

So module_hash (from canister_info) covers this packed code. What it doesn’t cover is gems installed later through update calls. Those only take effect at the next upgrade and boot, so each change still shows up in the canister’s history.

So the divergence from the base-application on upgrade is gonna be a combination of gems and app-code. That’s what a receiving canister would need to decide whether to accept messages from.

The plan. The developer (controller) chooses one of four disclosure modes in the canister’s interface file (packed, so the hash covers the choice):

  • closed (default): nothing beyond what the IC already gives
  • hash: the SHA-256 of every installed file that this boot loaded, plus anything pending, marked as pending
  • gems: the hashes plus the installed gems themselves
  • open: the gems plus the installed app, so others can directly review the installed app code.

The chain-of-trust. The installer generates the report and hash at boot, before any installed gem loads. These get then sealed in Rust and served directly by the C shim, never by Ruby. So an installed gem can’t redefine the endpoint to lie about itself.
I can release such a canister open-source so that basic packed-application is verifiable. That means a receiving canister only needs to decide whether to trust the installed-application.

This seems to expose the possibility of a malicious canister messaging a canister, then immediately running upgrade. So then the receiving canister approves based on checking canister_info on the pre-upgrade version. The upgrade would shows up in the history, but if the receiving canister already granted a whitelist policy, it is exposed. Closing that gap would probably need some additional thought and may be hard to close without some kind of trust judgement of the controller of the sending canister.

Some points to ponder:

  1. Would you allow a third-party controlled canister to message one of your canisters based on self-reported, sealed hash as described above?
  2. Have you seen upgrade race conditions like I described in the wild?
  3. What alternative configurations could you imagine using besides controllers-only?

Cheers

I just realised I linked to the wrong post. That one was about onchain compilers but there’s another post somewhere by @quint that lists the difficulties and gotchas of onchain interpreters - I can’t seem to find it though (maybe I’m misremembering).

How do you handle randomness? The runtime for the scripts will have a bunch of built in library functionality, and if any of those aren’t reliably deterministic then it’s essentially a bug unless its accounted for and polyfilled to provide the same behaviour in a deterministic way.

I think there are lots of potential use cases, such as RTS games where behaviour of a player’s entities is defined by code that can be edited easily. Or LLMs running via the upcoming ICIG (Intelligence Gateway), that can build their own dapps simply by spawning runtime canisters and injecting code (no need for off-chain compilers) etc.

I think I’d need a very specific example to think this through properly, but I think the general solution is that if a caller needs to trust some mutable script running onchain was the script that was responsible for a response or an action, then the caller also needs to trust whoever has the power to mutate that script. That’s the same situation for normal canisters (those can be upgraded at any time too, which is why blackholing is the ideal, and reputable well-protected DAOs are the only real alternative - ones that are actually meaningfully decentralised).

You could probably devise some sort of signing strategy where the script runtime signs responses based on the script that produced it (then you just need to trust the runtime and who/whatever controls that). I expect that to be unnecessary complexity though.

I think you are talking about randomness returned from system calls to the IC runtime. Yeah?
I haven’t come across such situations before. The runtime is ruby and I expect it to be reliably deterministic. But i’m happy to try and exercise the build with some more examples/challenges.

Right, I’m obviously new here but I was assuming that by the time a dapp is going to start accepting messages from a 3rd-party, they’d need to have thought through some of these kinds of upgrade, decide whether to trust scenarios.

Ya I see 2 basic options:
a. whitelist a particular canister for certain mutations. With some kind of policy for what to do if the canister changes.
b. do a request-based auth like bearer tokens

Cheers

I don’t know much about Ruby but I assume there will be a whole host of operations that depend on the runtime providing certain functionality. If you’ve not done so already I’d recommend cataloging all of those and testing that they work as expected when running onchain (rather than stalling due to lack of determinism).

I agree. That is certainly an important thing to consider. Thanks for sharing your thoughts on this topic.

This is probably true at the gem/application level. But to just have the ruby interpreter walk through a script and execute instructions, I’m not seeing issure anything so far.

My custom installer only loads the gems for now. Loading the application code is still on my to-do list.

I’ve listed out a whole bunch of functionality in the features page of the demo app. So you can check out and execute the available primitives.

https://xcdhl-gaaaa-aaaal-as2jq-cai.icp0.io/features

The only way to really know is to try though. Want to throw a common ‘randomness’ challenge/annoyance that you’ve encountered on-chain?

As mentioned I don’t know enough about Ruby, but I expect in a script you can write something like rand(). I expect that would break consensus unless you intercept the call and hook it into ICP’s native randomness implementation.

This is the most obvious potential issue. But I suspect there would be more.

My base module is already hooked into __ic_custom_random_get and ic0.timevia the polyfill.

So consensus has been running and verified, via pocket IC mainly, and limited on-chain testing from day 1.

One open issue I do have on that front, is that the RNG seed is provided at canister_init (orcanister_post_upgrade) and persists for the entire 40B cycle.

Long-story short. randomness cannot yet be relied upon. There are two major consequences right now:

a. the output is potentially guessable by an external caller for any given cycle. The rng seed depends on timestamps which are publicly knowable, I presume that’s a potential attack surface other IC canisters also need to guard against.

b. rand() currently returns the same result on each query within a 40B cycle.

I haven’t done a deep dive into how to resolve these yet but have been looking into reseeding from the management canister’s raw_rand

Perhaps I’ll also generate an in-canister secret along the lines of secret_key_base to salt the RNG. To increase the workload required from an external guesser.

For b, I imagine that other apps also need to solve this issue in a consensus safe maneer so I’d guess the rust-cdk exposes functions for me to call here. But haven’t gone looking yet.

Cheers