Wasm module contains a function that is too complex

Developers
1

At Demergent Labs we’ve just started working on Kybra, a Python CDK for the IC.

While trying to get RustPython to compile and run within a canister, I ran into the issue with the Wasm function limit: Wasm module exceeding maximum allowed functions

Luckily that limit is about to be raised to 50,000 functions.

I upgraded to dfx 0.12.0-beta.2 to see if the limit was raised locally. It may have been, because I’m getting an entirely new but very troubling error: Wasm module of canister rrkah-fqaaa-aaaaa-aaaaq-cai is not valid: Wasm module contains a function that is too complex

What can be done about this? Keep in mind that I’m trying to incorporate a complex software project into the IC that I do not have the most direct control over, RustPython. RustPython compiles to Wasm and just as I’m trying to deploy it to the IC I keep hitting these issues.

5 Likes
2

But I can and have forked RustPython. I would love some help tracking down which function is too complex, and I’m hoping we can remove this limitation in the future.

1 Like
3

The error appears to originate here:

1 Like
4

@lastmjs this limit was added to protect against situations where Wasmtime would take too long to compile the canister’s Wasm module. I think they’ve since made improvements to the compilation speed so we’ll look into raising the limit.

We’re also pushing out an improvement to the error message so that it reports how much the module exceeds the limit by. If you can share the module you’re currently working with we can also use it to see how much of an increase in the limit would be needed, or if it can be modified in some other way to make your code work.

5 Likes
5

That would be great! It might also be useful if the error message could give some information on which function.

I’ll push some code up in a couple hours and share the link.

2 Likes
6

How should I share the module with you? Do you want the Wasm binary?

7

Yeah just the .wasm file would be good.

8

I think I’ve been able to track down the crate that the function originates from, it’s the LALR parser generator: GitHub - lalrpop/lalrpop: LR(1) parser generator for Rust

9

Yep, I found it. It’s an absolutely humongous function that is auto-generated

10

Okay, so lalrpop is generated a Python parser used by RustPython. Part of the parser has a function that is about 6000 lines long, and it has a match with almost 900 arms.

Is the Wasm function complexity limit just based on the function body, or does it somehow follow the function statements as they call into other functions? I would guess it’s just based on the function body.

I would love the limit to be raised to have this issue just disappear, but I guess I’ll try manually modifying the function to pull pieces out into other functions to see if that works. If I’m headed down the wrong path I’d love some guidance. Thanks!

11

So the function complexity refers to the Wasm function, not the original Rust function. I’ve seen cases where this error appears after wasm-opt, probably due to function inlining. So lalrpop parser may not be the real cause (Candid parser is also using lalrpop, and it seems to work fine inside the canister).

3 Likes
12

lalrpop’s very large generated function was indeed the cause. I refactored the function and everything works now.

13

This is still a problem though, because I had to manually edit a generated file. Should I try not running ic-cdk-optimizer? Our binaries with RustPython are ~7mb without optimization though.

14

I would guess that the Python lalrpop generated code is more complicated than the Candid code, it’s an entirely different full-featured programming language

15

I guess it all depends on how the rust compiler generates wasm code. Using .gz without optimizer, or tuning the compiler optimization level is also worth trying.

1 Like
16

Yep, it’s just based on the function body.

It is possible that ic-cdk-optimizer is pushing the lalrpop generated code over the limit because of function inlining, and I’d be curious to see if your canister still hits the limit when you don’t run it through ic-cdk-optimizer, but that shouldn’t be our long-term solution.

17

I’ve reached out to you on Discord and am providing the source code and Wasm binary. Hopefully you can glean what’s going on. Refactoring the match in the __reduce function (the match with over 800 arms) does solve the problem.

18

I’m not sure it will make any difference but I think ic-cdk-optimizer has been deprecated in favor of ic-wasm.

19

I’ve been playing with that too

20

So I have some new information. When I use ic-wasm shrink, I get the too many functions error:

"Wasm module of canister rrkah-fqaaa-aaaaa-aaaaq-cai is not valid: Wasm module defined 13760 functions which exceeds the maximum number allowed 10000."

When I use ic-cdk-optimizer I get function is too complex error:

"Wasm module of canister rrkah-fqaaa-aaaaa-aaaaq-cai is not valid: Wasm module contains a function that is too complex"

When I refactor the extremely large function and use ic-wasm shrink I get the too many functions error.

When I refactor the extremely large function and use ic-cdk-optimizer everything works.

3 Likes