We have a large project that has been refactored a little bit into a good number of type, interface, etc modules.
Our wasm is getting big. When you drag it into the wasm editor it seems that there is a bunch of candid that is repeated over and over:
This candid text seems to be repeated 32 times in the file. It would be great to:
Does that code use actor classes? Each class will contain its own description of its candid interface. Also, there may be some duplication of types in the stable variable sections too
If possible, can you share the wasm for me to look at? Or point at the repo?
It is the same repo as last week. I’ll dm the branch.
You can just build one item in dfx and then inspect the wasm.
@ZhenyaUsenko Tried replacing some of the actor references with interface references that had imports and the file got BIGGER. Then he ran the distilled candid through the did → motoko to get rid of the imports and it got bigger again.
Actually, I totally forgot (or did not know) that we have a compiler option to omit metadata sections, e.g. --omit-metadata candid:args (or perhaps --omit-metadata "candid:args") should do just that.
Probably worth an experiment to see if it cuts down on binary size significantly, but omitting other meta data might make upgrading a little scary… (since dfx will have nothing to check against).
[nix-shell:~/motoko/doc]$ moc --help
...
--omit-metadata <name> omit icp custom section <name> (candid:args or candid:service or motoko:stable-types or motoko:compiler)
...
no_args - 11_256 KB
candid:args - 11_254 KB
candid:service - 11_150 KB
motoko:stable-types - 11_086 KB
motoko:compiler - 11_254 KB
Thanks, but what are those numbers exactly? The total number of bytes in each kind of section or something else? The last one (motoko:compiler) should be much, much smaller
Wasm size in KB (updated the comment)
I rechecked motoko:compiler. Got the same size
Isn’t that 11MB way over the 2MB (or is 3MB?) binary size limit, regardless of metadata size?
It is 3.1 MB when gzipped
We are getting close to the limit of where we’ll be able to deploy. It would be good to find the culprit so we can engineer for it.
I’m thinking we may need to end up having “factory” canistors that only reference one actor and are asked to deploy and transfer control to our management canisters so that it doesn’t have to reference the actors directly. This would be unnecessary complexity but may become necessary unless we can identify an optimization.
I was thinking of suggesting that solution too, but now that I’ve managed to build your project and look at the wasm, I’m beginning to wonder if we don’t actually have a bug in the code generator.
Your code seems to import 6 or so actor classes, non-recursively as far as I can tell, and that doesn’t seem to match the number of meta-data sections I’m seeing in the binary. So I’m wondering if we are indeed duplicating string constants, perhaps by forgetting to purge the constant pool when we do a recursive build of an actor class.
I’ll investigate some more.
Sorry to be so slow.
-c
Ok, I think I know what might be going on.
I suspect the compiler is linearizing the dependencies between libraries for the entire project, but when it compiles an imported actor class, it’s including all the (preceding) libraries in the global order, not just this class’s actual dependencies, leading to duplication of code.
So when you import a sequence of actor classes, each class will import all of the preceeding ones, not just the ones it actually needs.
This is pretty bad, but won’t be a super-quick fix.
@nomeata does that seem like a reasonable diagnosis?
Ok, I think I actually have a fix that works. Might make it into 0.8.1.
It reduces the size of the wasm to 3MB, which gzips (after wasm-shrink) to 600K.
Not too shabby (assuming it still works correctly).
The binary also now has the expected number of meta-data sections (just one per imported actor class plus the one for the main actor, for each sort of section).
@ZhenyaUsenko @skilesare thank you very much for taking the time to report this!
bugfix: fix for quadratic import of actor classes by crusso · Pull Request #3758 · dfinity/motoko · GitHub is the work in progress PR fixing this.
Celebrate the victories!!!
Moc 0.8.1 has been released and should fix this issue.