Introducing "canbench": a benchmarking framework for canisters

Developers Programs & Applications
1

Hey everyone!

I wanted to share with you a little tool that we’ve been working on and just open-sourced. It’s called canbench and it’s a benchmarking framework for canisters.

canbench is a tool that we built organically when we were trying to understand the performance of canisters and libraries that we were building. Once setup, it allows you to effortlessly monitor performance regressions as well as analyze the performance of your code.

Here is an example of what the canbench CI integration looks like (screenshots below):

Performance changes are detected:

image
image1834×1294 223 KB

No performance changes are detected:

image
image1818×1180 175 KB

Currently canbench only supports Rust canisters, but support for other languages can easily be introduced. Let us know if this is something you’d like to see. Happy benching :weight_lifting_man:

12 Likes
2

Hey @ielashi,

this is awesome, and as such, should be added to GitHub - dfinity/awesome-internet-computer: A curated list of awesome projects and resources relating to the Internet Computer Protocol :wink:

Would you mind doing a PR?

2 Likes
3

Anyone got any ideas of what might be causing this error?

“error parsing result of benchmark test. Result: module”

1 Like
4

Sure, will do :slight_smile:

It looks like there’s an error when executing the benchmark. Can you share the benchmark code and/or wasm module that’s causing this error?

1 Like
5

I am getting an error when trying to run canbench

Error executing benchmark create_account_bench. Error:
ingress Completed: Reply: 0x4449444c016c01b3c4b1f204680100010a00000000000000000101
ingress Err: IC0503: Canister rwlgt-iiaaa-aaaaa-aaaaa-cai trapped explicitly: failed to decode call arguments: Custom(Cannot parse header 

Caused by:
    binary parser error: io error)
Err: IC0304: Attempt to execute a message on canister rwlgt-iiaaa-aaaaa-aaaaa-cai which contains no Wasm module
2 Likes
6

Apologies for the slow response here. Is there code you can point me to to triage the problem? My github username is “ielashi” in case you’d like to share access to your code.

1 Like
7

repo

1 Like
8

Hi @ielashi, Do you know why I am getting this error?

1 Like
9

@zohaib29 Is your canister taking in init args? canbench doesn’t currently support deploying a wasm that has init args. If yes, can you try removing them and see if that fixes the issue?

We should be able to add support for specifying the init args in canbench.yml - it’s not that much work.

2 Likes
10

Yes, after removing init args it worked. Thanks!

1 Like
11

Thanks for the feedback. I added two issues to improve canbench in that regard.

2 Likes
12

Update: The newest version of canbench (version 0.1.4) now supports canisters with init args. See here for the documentation. Big thanks to @frederikrothenberger for adding this feature!

cc @zohaib29

3 Likes