So excited for this!!!
Hey! April is now over, any update on this?
Will be part of the new CDK release: export candid by chenyan-dfinity ¡ Pull Request #386 ¡ dfinity/cdk-rs ¡ GitHub
2 Likes
To clarify, will this allow developers to remove the candid property from their dfx.json file?
What weâre after is removing that requirement and having dfx automatically retrieve Candid from Rust (and custom canisters that use Rust), unless the developer explicitly specifies a Candid file.
The main goal is that developers no longer need to use the candid_method attributes to generate the did file.
You can use the generated did file as the final did file, so the candid field in dfx.json will be optional. Itâs still recommended that the developer provides the final did file for better readability. dfx build can check if the provided did file is out of sync with the generated did file.
1 Like
So will we still have to run wasmtime or something ourselves to get the generated did file? Or will we simply be able to upgrade dfx and remove the candid property? It looks like thereâs still commands that the developer will need to run to generate this?
Just trying to understand, will dfx be able to generate and find the did file on its own? Will it be automatically included in the public metadata section?
Yes, dfx will run wasmtime and store the did file in the metadata if the canister type is Rust. The build script is for custom type projects.
1 Like
Okay I understand, and thatâs great. Can we get the same functionality somehow for custom canisters that use Rust? Azle and Kybra create Rust canisters, but we use the custom type to insert our own build process. Otherwise we wonât be able to take advantage of this.
1 Like
The build script in that PR should work for custom canisters?
But that build script requires wasmtime and ic-wasm, both dependencies we would need to figure out how to install automatically for Azle and Kybra devs. These dependencies and extra build steps cause complications for our CDKs, if dfx can do it all that provides a really nice experience for us.
Has this functionality been released yet?
And just bumping this question, will we be able to use the same Rust functionality in custom canisters? Itâs very desirable for us to not have to run our own wasmtime or other infrastructure to generate the did file for our users, we would love for dfx to do all of this for us.
The goal is to support Rust canister type in dfx. If the custom canister is based on Rust, you can include the script listed in the PR in your own build process without waiting for the dfx release.
There are still a few design decisions we want to make in that PR. Would be good to know how your use case fits in.
- Generating did file requires building a binary either in Wasm or in the host architecture. There are pros and cons to both options: 1) running in Wasm requires wasmtime, which dfx will provide in its cache. The build time is shorter, since we only need to build for one architecture; 2) running in host architecture doesnât require wasmtime, but the build time is longer. However, if you already have unit tests running, you are already building in the host architecture, so there is no extra build time needed. We may end up supporting both depending on the use cases.
- The current PR requires putting
ic_cdk::export_candid!()at the end of the main file, which looks weird and is easy to forget. We are trying to find a better way to do this.
1 Like
Our use case is a general-purpose build step abstracted away from the developer, and simpler than what we have now.
Weâve already been doing this in Azle and Kybra, compiling for wasm32-unknown-unknown in Azle and now Wasi in Kybra. I think itâs better to compile for Wasm and not the host environment, weâve run into some strange issues while compiling for the host environment, and especially considering the various different host environments it seems much safer and simpler to compile for wasm32-unknown-unknown or Wasi. Please keep in mind Wasi as that support is coming for the IC soon, and we had to do some extra work to run the Wasm binary plain when compiled to Wasi.
Just to make sure, will this work for custom canisters? If it wonât work for custom canisters then it is not useful to Azle and Kybra. We are already running our own build step that compiles the canister to Wasm and runs it through a local Wasm VM (Nodeâs Wasm VM for Azle and Wasmer for Kybra). The reason I am pushing for dfx to handle this is to reduce the complexity of our build systems and having to install a Wasm VM generally across all possible systems that our devs may have.
I just ran into this problem on Mac M1âs with our new version of Kybra using Wasmer, it simply broke because Wasmer was not supported on those chips. Also our work-around is to add to the post_install script a call to the canister to get the Candid file, but this doesnât allow an easy way to put the candid into the Wasm metadata, because weâve already deployed the binary.
dfx handling all of this for custom canisters will be very nice and solve issues weâve been dealing with for a long time.
We need this to work for the custom canister type as well, using the script is not useful to us as weâre already doing that and the main purpose of this functionality for us is to not have to run our own process to generate the Candid file.
Also, can we remove the need to provide the Candid property in dfx.json? Thatâs another goal we have, the dev should not have to provide that. Motoko has this dx and it is very nice. If they want to override the Candid they can provide a path to a hand-written Candid file, otherwise I donât see why they should need to deal with this IMO unnecessary complexity.
1 Like
weâve run into some strange issues while compiling for the host environment, and especially considering the various different host environments it seems much safer and simpler to compile for wasm32-unknown-unknown or Wasi.
Do you have an example where things go wrong? cargo test is building binaries for the host environment. As long as the platform is supported by Rust, we should be fine?
Just to make sure, will this work for custom canisters?
The short term focus is for Rust canister type only. For custom canisters, itâs unclear how dfx can help. Itâs possible that the canister is not even based on Rust. Whatâs the problem of adapting the build script, which is essential what dfx will do behind the scene?
The reason I am pushing for dfx to handle this is to reduce the complexity of our build systems and having to install a Wasm VM generally across all possible systems that our devs may have.
Does it help if wasmtime is packaged in dfx, which can be called from .cache/dfinity/version?
We need this to work for the custom canister type as well, using the script is not useful to us as weâre already doing that and the main purpose of this functionality for us is to not have to run our own process to generate the Candid file.
You have your own build script anyway. Whatâs the difficulty in adding a few more lines?
Also, can we remove the need to provide the Candid property in dfx.json?
Yep, candid field will become optional.
1 Like
It was a while ago and the situation was rather complicated, somewhere in our dependencies, possibly with ic-cdk or similar crates, compiling on my laptopâs architecture broke but compiling to wasm32-uknown-unknown did not. We tracked it down to compiling for two different architectures.
Itâs not too difficult to add an option to dfx.json to allow custom canisters to do this right? If a custom canister is actually a Rust canister under-the-hood, like all Azle and Kybra canisters are, then we still would want to take advantage of this feature.
Adding a flag/property to dfx.json to turn this on optionally seems like a great solution to me.
The build script is the whole reason we want this functionality in the first place, we donât want to have to run that build script and install wasmtime onto the developerâs machine automatically. We are trying to build a general-purpose build process that works for everyone, and dealing with wasmtime/wasmer adds unnecessary complexity if dfx can just do it for us. We want to remove as many dependencies as possible, so that the dev only has to install dfx and Azle/Kybra.
Weâve already had issues with wasmtime, I had to remove it in favor wasmer when we switched to Wasi, and wasmer is currently broken on Mac M1s. Now Iâm stuck again, because getting the Candid out of post_install causes problems since the canister has to be entirely deployed first.
The issues with this are many and itâs caused us major setbacks.
Yes this helps a lot, but allowing custom canisters to turn on this option seems the more ideal solution so that we can remove this code entirely from our build process.
Actually this might not help, as I mentioned above I switched from wasmtime to wasmer because of issues with wasi, but perhaps youâve overcome those issues in the cdk. There were some problems with the ic-wasi-polyfill, wasi2ic, or something like that. I assume youâve overcome those in the cdk but weâll have to see.