What are the best tools to test canisters written in Motoko. What I‘ve tried so far is surprisingly big of a pain

I‘ve started to use ic-repl but I‘m wondering what else is out there?

1 Like

I think these are some good resources:

5 Likes

In addition to these, you can also try ic_state_machine_tests. Here is an example: cdk-rs/e2e.rs at main · dfinity/cdk-rs · GitHub. You will have to write the tests in Rust, but it works for Motoko canisters as well. The benefit is that it is much faster than making calls to the replica.

Regarding ic-repl, may I know your pain points?

3 Likes

I second this. I use ic_state_machine_tests and it’s a joy to use as compared to anything on Motoko. Although the execution times could definitely be lowered. The test suite run on our Github CI takes around 20 minutes to run :sweat_smile:

If anyone is up to writing down a best practice article on testing using the various tools available, we can bundle up a bounty for this.

1 Like

I could take a stab at this for writing tests for Rust projects, both unit and integration, but it will have to wait till the 19th. We have a grant milestone check in coming up, and there’s so much stuff we need to complete :sweat_smile:

1 Like

Idk what kind of tests you want , but you can check this latest one from kyle.

If you like the speed of ic_state_machine_tests, you can also try drun, which is running in our Motoko CI. It’s like a CLI version of the state_machine_tests:

The best way to get drun is probably to run nix-shell in the motoko repo. If there is more interests in this, we can spend time to improve the UX.

1 Like

I prefer a language centric way like ic_state_machine_tests. That way, it integrates very well into existing Rust testing tooling, and all I need to run is cargo test and my entire test suite including the integration tests are run.

I was using ic-repl before that but I prefer ic_state_machine_tests owing to not have to run the tests manually. And also having all the power of a programming language like Rust as opposed to a custom scripting language

1 Like

Wasn’t ic-repl a CLI version of state_machine_tests? How is this different?

ic-repl runs on replica directly. Both state_machine and drun run on the execution layer of the replica, which is much faster, because messages don’t go through consensus.

1 Like

We have a large number of motoko tests written in motoko at GitHub - ORIGYN-SA/origyn_nft: The Public Facing Origyn NFT Reference Implementation.

If you follow the pattern there it works quite well. Make sure you use the “args”: “-v --compacting-gc -no-check-ir”, in your dfx.json for your test canisters or they will take too long to build.

4 Likes

Ideally, you’d want to abstract as much logic as possible outside of the API and into modules that you can unit test more thoroughly. Then your API testing becomes moreso integration testing.

By having your API import function “routines” from an external module, this saves you the headache of writing more integration tests with ic-repl than you absolutely need to, especially since the more canister tests you write the longer you have to wait for your tests to complete (generally).

3 Likes

Thank you very much I will experiment with it, I have barely any rust skills though, unfortunately.

The pain points probably are rather on me than on the tool, I’m not an extremely experienced dev so I was just not sure exactly what it is that I’m writing. It felt like another thing with its own synthax and way of interacting with the canisters.

But I find it generally hard to deal with state across several canisters and the nuances of local vs. mainnet, so my way of thinking about testing canisters could generally be better…

Thank you! slightly scared of getting into rust though :wink:

Thank you very much! That could in the end be what would feel the most natural to me, nice.