Exciting changes coming to your local development experience

Developers
1

Hello everyone!

My name is Jason, I am the Engineering Manager of the SDK team at DFINITY. I’d like to announce an exciting change we’re bringing to dfx!

Previously, when running Internet Computer locally, dfx would use a configuration called the System Subnet to deploy canisters and execute their code. This has worked well for the most part, but we have heard from developers that many were faced with challenges when they were ready to migrate to mainnet.

This is because the System Subnet configuration is much more lenient:

  • No cycles accounting is done
  • Function calls have more generous limits
  • There is a higher maximum limit on canister .wasm size

The result is that once code developed locally gets deployed to mainnet, many users hit errors such as “cycles limit exceeded” with no clear indication as to why.

We’re happy to introduce two new subnet types to dfx - Application and VerifiedApplication. Here, we’ll focus on the Application subnet type. This subnet configuration is much more strict - apart from a few bells and whistles, it is essentially the same configuration that is used in mainnet. You should now see errors much earlier in your development lifecycle, and that should lead to fewer errors in production!

Moving forward, this is the default behavior when using dfx. You can always revert to the old behavior if you encounter any issues with the new configuration. The subnet type can be configured using the defaults.replica.subnet_type key in dfx.json. The possible values are system, application, and verifiedapplication.

You may also now encounter a point where you must top up your canisters when running Internet Computer locally. This makes for a fine opportunity to learn how to deal with these types of errors before promoting your code to mainnet.

To top up your canisters locally, you can leverage the “fabricate-cycles” command, providing an amount in ICP or cycles:

dfx canister fabricate-cycles

So to recap, the application subnet introduces the following changes to your local development experience:

  • The same instruction limits for messages
  • Proper cycles accounting

This is available as of version 0.10.0-beta.1. Try it out for yourself and let us know what you think! As always, if you encounter any issues, please let us know by dropping a line here in the forum or flagging an issue in our GitHub and we will get right on it.

We’re excited about what this change brings, and we hope you will be too. Let’s build!

22 Likes
2

This is excellent! About Proper cycles accounting, does this mean we can measure cycle usage locally? I am very interested in benchmarking my code, I want to be able to call a query or update method and have returned to me the exact cycles usage during that call. Is this possible?

5 Likes
3

Hi @lastmjs. This functionality is not available yet, but we are working on it! It’s queued to enter our SDK roadmap.

4 Likes
4

You could check the cycle balance before and after the call to know the cycle usage.

We also have an CLI tool to instrument the canister similar to how we profile cycles in the Playground: GitHub - dfinity/ic-wasm: A collection of libraries and tools for transforming Wasm canisters running on the Internet Computer. But it needs more tooling support to make it user friendly.

5 Likes
5

Great to hear! If it helps to know how to prioritize it, benchmarking is something I want for Azle (TypeScript CDK) and Kybra (Python CDK) before releasing a 1.0 for both.

2 Likes
6

Hey Jordan, we have been developing a really useful Rust CDK for canisters. I would love to discuss it with you, as there may be some interesting use cases for your CDKs.

2 Likes
7

Awesome, could you DM me on OpenChat, Telegram, or Twitter? @lastmjs

1 Like
9

Any chance we could get a god mode subnet type with a massive wasm size and massive cycle limit. This is necessary for testing. You have to reference classes and instantiate them from test runners and you quickly get above the 3.7MB size limit for wasm. It would be great if there we a place to configure that.

3 Likes
10

Is this kept up to date as the execution team updates the costs of various system calls etc?

Also, can this (be made to) replace motoko/wasm-profiler at master · dfinity/motoko · GitHub so that we don’t maintain multiple copies of the same idea?

11

This is really exciting! It seems like 0.10.0 will be a big release.

1 Like
12

Once the replica implements ic0.performance_counter, we can make sure the cost are in sync.

I hope eventually we can replace wasm-profiler, but we probably won’t implement support for WASI API, as this library is for canister only.

1 Like
13

I am not sure if using ic0.performance_counter will be as good as what we have now: With wasm-profiler, the code that does the profiling and printing/storing of values is not profiled; if we use ic0.performance_counter the profiling code will itself be included. Maybe the overhead can be somehow be measured and subtracted again, though. We’ll see.

14

Good point, especially with storing logs in stable memory, the recorded cost can be high. We can let users to choose which approach to use. The only benefit of using performance_counter is that we can stay in sync with the cost table used by the replica.

15

dfx 0.10.0 is ready for manual installation and testing. You can upgrade to/install it with DFX_VERSION="0.10.0" sh -ci "$(curl -fsSL https://smartcontracts.org/install.sh)"

16

Now that ic is public, maybe there is a hope that the instrumentation code can be it’s own crate, and flexible enough so that your ic-wasm tool can simply depend on it as a library? The wasm-profiler initially was a copy of the instrumentation in the replica, so maybe that’s possible?

Until, of course, the instrumentation and cost accounting is no longer a pure function and starts to depend on subnet configurations etc, then things become more messy.

17

I noticed in the dfx 0.10.0 release notes that:

New feature: Install arbitrary wasm into canisters
You no longer need a DFX project setup with a build task to install an already-built wasm module into a canister ID. The new --wasm flag to dfx canister install will bypass project configuration and install the wasm module at . A DFX project setup is still recommended for general use; this should mostly be used for installing pre-built canisters. Note that DFX will also not perform its usual checks or API/ABI/stable-memory compatibility in this mode.

Do I still need to specify the canister name in dfx.json to install the canister using this flag? I’m not quite sure what the purpose of this flag is. If I reference the wasm in my dfx.json with some name (e.g. Foo), can I then import it from my other (Motoko) canister using the import Foo "canister:Foo" syntax?

18

It was mentioned on the call that ingress message size was a limitation of Wasm. But I know that an application subnet we can install a binary is up to 3.7 MB. What is the actual top limit that the wasm can take in a single ingress message?

19

You only need the canister name in dfx.json in order to use the short name; you can use a principal whenever you like. The purpose of this flag is to allow you to do things like install II with a single command instead of it being part of your project setup. There are a couple more features before that exact workflow becomes a reality but --wasm should still be useful in its present form.

20

Is there a way to fabricate cycle deductions? I’d like to drain all of my canister’s cycles to test error handling.