What are the best tools to test canisters written in Motoko. What I‘ve tried so far is surprisingly big of a pain

[Mathias]

I‘ve started to use ic-repl but I‘m wondering what else is out there?

[paulyoung]

I think these are some good resources:

Unit Testing in Motoko

Motoko: How to write unit tests?

We are still experimenting with the best way to test Motoko and canisters. There are several tools around today that can help with unit testing: Built-in assert: see examples from the base library: motoko-base/test at master · dfinity/motoko-base · GitHub The Motoko matcher library: GitHub - kritzcreek/motoko-matchers. It’s best to test with library code, i.e. without service actors. ic-repl: GitHub - chenyan2002/ic-repl. This is designed at the canister level, so that you can use the tool to…

Motoko: How to write unit tests?

In addition to the approaches listed by @chenyan above, there is also the approach to use the ic-hs IC simulator as a library, and use Haskell to script the testing of the compiled canister. As it works with the compiled wasm, it works with both Motoko and Rust canisters. This is the approach chosen by the Internet Identity project:

[chenyan]

In addition to these, you can also try ic_state_machine_tests. Here is an example: cdk-rs/e2e.rs at main · dfinity/cdk-rs · GitHub. You will have to write the tests in Rust, but it works for Motoko canisters as well. The benefit is that it is much faster than making calls to the replica.

Regarding ic-repl, may I know your pain points?

[saikatdas0790]

I second this. I use ic_state_machine_tests and it’s a joy to use as compared to anything on Motoko. Although the execution times could definitely be lowered. The test suite run on our Github CI takes around 20 minutes to run

[domwoe]

If anyone is up to writing down a best practice article on testing using the various tools available, we can bundle up a bounty for this.

[saikatdas0790]

I could take a stab at this for writing tests for Rust projects, both unit and integration, but it will have to wait till the 19th. We have a grant milestone check in coming up, and there’s so much stuff we need to complete

[h1teshtr1path1]

Idk what kind of tests you want , but you can check this latest one from kyle.

Testing Your Canister With Vitest

[chenyan]

If you like the speed of ic_state_machine_tests, you can also try drun, which is running in our Motoko CI. It’s like a CLI version of the state_machine_tests:

dfinity/motoko/blob/master/test/drun-wrapper.sh

The best way to get drun is probably to run nix-shell in the motoko repo. If there is more interests in this, we can spend time to improve the UX.

[saikatdas0790]

I prefer a language centric way like ic_state_machine_tests. That way, it integrates very well into existing Rust testing tooling, and all I need to run is cargo test and my entire test suite including the integration tests are run.

I was using ic-repl before that but I prefer ic_state_machine_tests owing to not have to run the tests manually. And also having all the power of a programming language like Rust as opposed to a custom scripting language

[saikatdas0790]

Wasn’t ic-repl a CLI version of state_machine_tests? How is this different?

[chenyan]

ic-repl runs on replica directly. Both state_machine and drun run on the execution layer of the replica, which is much faster, because messages don’t go through consensus.

[skilesare]

We have a large number of motoko tests written in motoko at GitHub - ORIGYN-SA/origyn_nft: The Public Facing Origyn NFT Reference Implementation.

If you follow the pattern there it works quite well. Make sure you use the “args”: “-v --compacting-gc -no-check-ir”, in your dfx.json for your test canisters or they will take too long to build.

[icme]

Ideally, you’d want to abstract as much logic as possible outside of the API and into modules that you can unit test more thoroughly. Then your API testing becomes moreso integration testing.

By having your API import function “routines” from an external module, this saves you the headache of writing more integration tests with ic-repl than you absolutely need to, especially since the more canister tests you write the longer you have to wait for your tests to complete (generally).

[Mathias]

Thank you very much I will experiment with it, I have barely any rust skills though, unfortunately.

The pain points probably are rather on me than on the tool, I’m not an extremely experienced dev so I was just not sure exactly what it is that I’m writing. It felt like another thing with its own synthax and way of interacting with the canisters.

But I find it generally hard to deal with state across several canisters and the nuances of local vs. mainnet, so my way of thinking about testing canisters could generally be better…

[Mathias]

Thank you! slightly scared of getting into rust though

[Mathias]

Thank you very much! That could in the end be what would feel the most natural to me, nice.

[Mathias]

@skilesare may I ask; Does “test_wallet” exist simply to call the relevant canister from different principals?
I like the pattern, it would be nice to use it if my project gets bigger. But I was also wondering if there are other ways of calling the canister from different principals in Motoko?

[skilesare]

Exactly right. The testWallet acts as another address on the network. It is a stand in for a user outside the network. Think of it as someone calling a function from plug.

[Mathias]

Thank you, that makes sense.
Your codebase is amazing to learn from btw!