Request: Publish `ic_state_machine_tests` as a separate package and provide documentation/guidance on how to use it

Developers Rust
1

As I’ve been experimenting with ic-repl to figure out a way to test our multi-canister backend, I’ve been made aware that there exists another library called ic_state_machine_tests that’s used by Dfinity internally for integration tests.

It can be found here and example usage found here

Although I’m trying to mimic how it’s been used in the above illustrative example, I believe it would be immensely helpful for Rust canister devs if this package were published separately to crates.io and had some documentation/guidance on how to use/consume it.

We could then use this to write integration tests for our own canister backends written in Rust.

Thoughts?

2

Hi @saikatdas0790!

I’m the original author of ic-state-machine-tests.

I believe it would be immensely helpful for Rust canister devs if this package were published separately to crates.io and had some documentation/guidance on how to use/consume it.

I agree that having this library on crates.io would be great, but publishing it is infeasible:

  1. The library depends on a lot of internal code (hundreds of packages), which we’ll have to publish and keep up-to-date.
  2. The library is somewhat fragile to internal refactoring. For example, the execution environment is planning to remove the option of non-sandboxed canister execution, which will break everyone’s code. We can easily update all internal clients when such perturbations happen, but not external ones.
  3. Generally, the code inside the IC repository is not guaranteed to build without installing special tools and libraries. I made quite some effort to build ic-state-machine-tests without such tools, but it’s an uphill battle because of point 1.

There is hope, however. After chatting with the II team, who are heavy users of ic-state-machine-tests, we decided to turn the library into an executable suitable for testing. We’ll add a thin HTTP layer to make the state machine look like a replica, but without all the complexity of the P2P and Consensus layers, providing a deterministic execution model and APIs to manipulate the internal state, e.g., shift time.

This solution addresses most of the previously mentioned issues and enables you to write tests in any language with an IC agent (TS/JS, Rust, etc.).

Unfortunately, I don’t have an ETA for this solution yet.

6 Likes
3

Got it. For now, what would be your suggested way of setting up integration tests for a multi canister backend that does cross-canister calls with canisters written in Rust? Would you not suggest ic-state-machine-tests

I was told here that it would be the path of least resistance for the time being. Would you suggest otherwise?

Related forum thread here

4

Hi Roman,

Any updates on this at all?

With the II being migrated/refactored to a more scalable architecture, wanted to check if there has been any movement on this since you mentioned them being the largest consumers of ic-state-machine-tests

5

Hi! The migration work doesn’t affect the canister source code or testing infrastructure. For now, we are only moving the canister to a different subnet. Nothing has happened on the state machine testing front so far, unfortunately.

1 Like