Due to the prerequisite of building the canister before generating declarations for Motoko projects with dfx generate, sometimes it takes a really long time to generate declarations for larger projects (sometimes up to a minute, and this is for a powerful M2 chip Macbook with plenty of RAM). Especially if we’re importing nns canisters and generating declarations for those canisters as well.
Is are there any improvements that I can make, or that can be made on the dfx side to speed up declaration generation so that it takes 5-10 seconds, instead of a minute or so? Can the declaration generation process be parallelized for all the canisters? Is building the canister necessary?
Maybe in addition to the question, I have noticed that no matter what there is a recompile on build even if the source files have not changed. Maybe that could also help.
I do a deploy and a generate (and a build before all that), but they each recompile
Assuming that there is a did file available you can run didc bind, or?
In my opinion, canister repos should always ship with hand-written did files in them. And CI should check that it matches the implementation. Then it’s easy and fast to derive bindings from them without having to build the canister.
Just checked things on my end with -vvvv. Generation of type declarations happens quickly, but the bottleneck is during build.
Our main canister takes 20-30 seconds to build, we have 6 canisters in total that are a part of our application.
We also have staging versions of some of these canisters, so a declaration file gets generated for both the main canister and the staging canisters, meaning we’re needing to build twice even though the declarations are identical between prod and staging wasms.
So all in all, each time we refresh declarations this takes 90-120 seconds.
Finally, the process for building and generating declarations for each of these canisters happens in sequence rather than parallel.
So a few suggestions off the bat on how this could be improved:
Dfx can identify if two canisters have the same settings in dfx.json (same file path, relevant settings) and just build and generate declarations once
Canister builds happen in parallel instead of sequence.
Just with these two changes implemented, I’m guessing this would cut the generation declaration time from 90-120 sec to 20-30 sec.
It would also be interesting to explore improvements that could speed up the build time of Motoko canisters. If we could get it down to 3-5 sec, this would help us iterate quicker (specifically when writing integration tests), and would definitely save us a few dollars on our CI bill
Can you use dfx … -vvvv to to see if you are invoking moc via the mop ‘moc-wrapper’. My hunch is that this is slowing down the dependency analysis because it invokes moc using bash and node rather than directly. It will do that for each dependency of the canister (though not shown in -vvvv log)
I also noticed a recent hack is causing dependency analysis to happen twice, once before and another time during the actual build.
