Exploration: calling Rust functions from Motoko

Developers
1

Hey, Motoko and Rust developers!

The Languages team at DFINITY is exploring several approaches to make it easier to call Rust functions from Motoko canisters. This forum post summarizes the current status for Motoko-Rust interoperability and gives anyone who is interested a place to voice opinions and feedback.

Currently, the simplest way to call Rust functions in Motoko is to use two different canisters on the same subnet, one written in Rust and the other in Motoko. This has several advantages (good support for multi-canister projects in dfx, data structure conversions handled via Candid interface) but also has various drawbacks (async/await syntax, difficult state management, and possibly the introduction of canister commit points).

Although complicated to do by hand, it’s also possible to manually recompile the Motoko compiler and runtime system with custom Rust language bindings. This provides a foreign function interface with static linking, making it possible to use both languages in the same WebAssembly module. However, this currently has major limitations in practice, especially when trying to allocate memory from a Rust function.

The goal of this exploration is to reduce the learning curve and mitigate the drawbacks for each of these approaches.

Here is a summary of recent progress:

  • Canpack: a solution for Motoko developers to use Rust libraries via a generated sibling canister
    • Add Rust crates using a [rust-dependencies] section in your mops.toml file (example).
    • It’s also possible to create Motoko libraries which depend on Rust crates without requiring additional setup on the canister end.
    • This is an experimental tool; whether this becomes an official project depends on developer interest.
  • GitHub PR: Motoko compiler support for custom runtime system (RTS) functions
    • Removes the need for a recompiled moc binary to add Rust bindings (still requires a custom RTS).
    • Low-level capability which we can build upon with additional tooling over time.

If you have a specific use case for calling Rust from Motoko (such as needing a cryptographic function or parser which isn’t yet available in the Motoko ecosystem), please feel free to let us know so we can prioritize the overall most useful solution.

As a quick side note, another possible approach to communication between Motoko and Rust is the Wasm component model specification. It’s currently undecided whether this will be supported in the replica, so this exploration focuses on solutions with incremental value at each step towards the long-term goal of seamless Motoko-Rust interoperability.

Looking forward to hearing your thoughts on this topic!

Cheers,
~ Ryan

7 Likes
2

Ryan,

This is really fantastic advancement and I appreciate you putting things together to help us understand how to best use these.

One thing that helped me understand what is going on here is that calls to a composite query on the same subnet are happening pretty much in line with processing. I am curious if there is any cycle overhead from context switching(I’m guessing memory isn’t shared between the canisters so If I want to regex 2Mb of text I’m going to be spending some cycles) and if control flow is handed over to a scheduler or not. Or put another way, when I await a composite query, can something else run before I get control back or not? I think the answer is that you should assume something can, but maybe functionally, in the current replica, it doesn’t work that way?

In any case, this greatly increases the number of use cases for Motoko. I think there may be a bit of a bootstrap to ‘canpack’ some of the most popular and useful libraries. Ultimately it would be awesome if those could just bootstrap from mops as well.

4 Likes
3

It would be fun if we had some kind of service that would let you do an init like this and it would deploy the certified canpack wasm to your subnet if it doesn’t exist or return to you the canister where it does exist. Cycle management and access lists would need to be considered as well. :thinking:

     stable var regEx : RegEx.Service = actor("aaaaa-aaa");

     public shared init() : async () {
           cycles.send(2000000000);
           regEx = actor(await CanPackService("regex#v0.2.3");
     }
1 Like
4

Yep, that is correct if I understand your question. As a reference point for anyone reading this, we use a similar pattern in Motoko Playground (here) to call Rust functions from the Motoko canister pool.

Great idea; noted!

5

I know there was some talk within the Motoko team around having multiple actors within a canister that could communicate with one another.

I don’t know what the technical lift would look like, but a separate sub-actor within a canister (in a single language i.e. Motoko/Rust) would be a really nice abstraction to have (as well as not needing to make an inter-canister call just to use a regex library).

5 Likes
6

This would be a fantastic abstraction. For this to work in an intuitive way, it would most likely require support in the replica for the Wasm component model. I’m seeing this as good a concrete use case to increase the priority of adding this capability to the Internet Computer.

1 Like
7

I just want to be able to use Rust libs within Motoko. The inter-canister is nice but I can see many use cases where you want to call a rust func to generate some hash or something. Bridging libs with rust would be nice, this will allow a vast amount of useful code to be used within Motoko.

Elixir can call Rust functions by leveraging the interoperability provided through Native Implemented Functions (NIFs) and Rust’s Foreign Function Interface (FFI). This process essentially involves two main steps: exposing Rust functions in a way that they can be called from external code (like Elixir), and then loading these functions into the Elixir application.

3 Likes
8

I see what you’re saying and completely agree. The caveat with Elixir NIFs and other similar approaches is that they use dynamic linking, which isn’t (yet) supported for Wasm modules on the IC. It’s possible to simulate dynamic linking using multiple canisters on the same subnet, although this of course has various limitations (see the summary below).

We are currently also looking into a way to support FFI using statically-linked Rust code. This makes it possible to synchronously call Rust functions at the cost of increased complexity and (in some cases) memory safety issues. This is a universal challenge with statically-linked FFI implementations, so my goal is to at least make this available for use cases where it’s worth the extra complexity.

Here is a quick summary of the tradeoffs for each approach:

  • Canpack
    • Pros: convenient usage; works with Mops; usable today
    • Cons: limited IC-specific capabilities; requires async context; commit point when calling a function
  • FFI (with static linking)
    • Pros: direct access to Motoko memory from Rust; highest-performance option
    • Cons: complexity from memory management; sensitivity to choice of GC
  • Wasm component model
    • Pros: overall the best solution with the fewest compromises
    • Cons: may take at least 1-2 years for this to become available (currently not a planned feature)
2 Likes
9

Would that allow you to use any language since it is Wasm. So you would be able to use Azle in JS as well?

10

That is correct. With the component model, it would be possible to use essentially any combination of Motoko, Rust, JS, and any other Wasm-compiled language in the same canister. We would most likely start by supporting Rust + Motoko and then roll out support for other language integrations over time.

For anyone reading this, now is the best time to chime in with your opinion so we can decide how to prioritize this feature.

If you’re interested, please vote on this topic on the ICP Developer Feedback Board:

1 Like
11

:exploding_head: This feature would be extremely powerful. Specially for us in Motoko land.

1 Like