Canbench: State Persistence, Caller Configuration, and Extended Metrics in canbench for ICP Canisters

Developers Rust
1

Hi everyone,

I’m currently using canbench to benchmark my ICP canisters and have a few questions that I couldn’t find clear answers to:

1. State Persistence

If the canister’s state persists between the execution of different benchmark functions, then the data stored by one function (like store_user) should be accessible by another (like get_user).

  • Is this typically the case?
  • Does canbench reset the canister state between benchmarks, or does it run them sequentially with the same state?

2. Caller Configuration

When I call a main function from benchmark function:

  • Who is the default caller (i.e., the principal ID)?
  • Is there a way to configure or set a custom caller in the benchmark context, either through code or a canbench configuration option?

3. Benchmark Result Configuration

By default, canbench only shows three results:

  • instructions
  • heap_increase
  • stable_memory_increase

Is there any way to configure canbench to display additional metrics such as execution time, cycles used, or other performance details?

  • Are there any flags or config files I can modify to get more comprehensive results?
1 Like
2
  1. State persistence is generally considered bad practice. Tests should be independent, and their execution order should be arbitrary. Stable memory could be persisted, see the crate’s documentation.
  2. The default caller is anonymous, and it is not currently configurable. Please note this is a query call, which also addresses the previous question.
  3. A pricing calculator is available and may be useful. Given an estimated execution rate of 2 billion instructions per second, developers should prioritize optimizing instruction count and memory usage. Note that optimizing execution time will not result in cost savings.
1 Like
3

Hi,

Thank you for your valuable response. I am encountering an issue where none of the printing statements seem to work within canbench bench functions. I have tried the following:

eprintln!("Error occurred: {:?}", error);
ic_cdk::println!("Starting register_user benchmark");
println!("Starting register_user benchmark");

However, none of these statements produce any output in the logs or the debug console. Could you please help me understand why this is happening and suggest a way to enable logging or debugging for bench functions in canbench?

4

The discussion is in this thread.

1 Like