The current Dfinity Ledger wasm is too large, even after minimisation, to be deployed to the IC for testing purposes. Does anyone have a version of the ledger wasm that is under the 2mb limit? After optimising, I am seeing a file-size of roughly 2.5 mb.
Cheers
Hi @Maxfinity !
We optimized the size of the Ledger canister in scope of the “Ledger canister fit for dapps” feature (The Internet Computer’s Post-Genesis R&D Roadmap | by DFINITY | The Internet Computer Review | Feb, 2022 | Medium). The Wasm module should now be below 2MiB (this commit did the trick if you’re curious).
Could you please try following the local setup instructions but use a fresh commit from the IC repo in the first step (e.g., export IC_VERSION=c7ec4881cf8db619aab86a148010b74f58e3ef37)?
Hello Max,
For canister wasm larger than 2mb (egress message size limit) and smaller than 5mb (intra canister message limit tbc) –
You might be able to deploy the canister via a canister;
Here is an example (not really well documented) about how to achieve this: GitHub - ORIGYN-SA/large_canister_deployer_internal: Install wasm code to a canister larger than 2 mb by chunking it and deploying it via a canister in the same subnet
large_canister_deployer_internal/index.js at master · ORIGYN-SA/large_canister_deployer_internal · GitHub
Make sure that the “larger canister deployer” canister and the target canister are in the same subnet;
Hi Roman, none of these commits work unfortunately - the first is unfortunately slightly too large and the second has an invalid gz format.
The data I see is a bit different, did i miss something?
Hi @Maxfinity!
none of these commits work unfortunately - the first is unfortunately slightly too large and the second has an invalid gz format.
I looked into that issue, it turns out that there was recently a refactoring in the CI pipeline that introduced a glitch making the canister larger again.
You can use this commit hash to get a reasonably small ledger canister: 77cecc596216a584174106733b40513645641fef
Thanks for reporting the issue and sorry for the inconvenience.
Why does this page again point to a ledger canister that is too big and not deployable? This is still a year after launch and yet the public docs do not have a canister that is deployable locally…despite 3 interventions from this forum thread…
I recommend an automated check before tagging.
You can gzip it and deploy it now. Hopefully, dfx will do this for us soon? Maybe? Please?
This step was missing from the installation instructions:
Configure your replica to run a `System` subnet. Modify `dfx.json` to include:
{
"defaults":{
"replica": {
"subnet_type":"system"
}
}
}
Adding it ASAP.
Hey Max,
I noticed this a while ago myself. This is because dfx 0.10.0 uses an application subnet with stricter limits by default. I added a note in step 9 of the docs yesterday. But it’s probably better to include this as a default step as @Severin has done in the Github readme.
@roman-kashitsyn shared this concern about doing it on behalf of users.
Perhaps an entry in dfx.json to easily enable or disable it (depending on the default) would be a good compromise.
Has this issue been resolved? I still get errors for .wasm binaries. The gzip version did not work for me either, as specified in the Ledger setup on smartcontracts.org, but then I went into the examples on github and gave chmod 777 permissions to ledger.private.did/-public.did/-.wasm. I made dfx.json as per the updates in 0.9.3:
{
“canisters”: {
“ledger”: {
“type”: “custom”,
“wasm”: “src/ledger/ledger.wasm”,
“candid”: “src/ledger/ledger.private.did”,
“remote”: {
“candid”: “src/ledger/ledger.public.did”,
“id”: {
“ic”: “ryjl3-tyaaa-aaaaa-aaaba-cai”
}
}
},
“musall”: {
“main”: “src/musall/main.mo”,
“type”: “motoko”,
“dependencies”: [ “ledger”]
},
…
And still I get this error:
Building frontend…
Creating UI canister on the local network.
Jun 26 17:08:48.650 WARN s:tsgfc-bkhjx-johkh-hawro-us53x-5bp2c-gxlgy-tu66c-gamsr-gidrw-kae/n:7yn2i-be2ey-7sgvq-2lajz-qedq5-oivmi-w3rdc-wdzla-s3phy-nwyv3-iqe/ic_execution_environment/scheduler At Round 31 @ time 2022-06-26 17:08:47.939551 UTC, the resulted state after execution does not hold the in-out cycles invariant: cycles at beginning of round 99695011921726 were fewer than cycles at end of round 199695011921726, messaging: {“round”:31,“canister_id”:null,“message_id”:null}
The UI canister on the “local” network is “rkp4c-7iaaa-aaaaa-aaaca-cai”
Installing code for canister ledger, with canister ID rrkah-fqaaa-aaaaa-aaaaq-cai
Error: Failed to install wasm module to canister ‘ledger’.
Caused by: Failed to install wasm module to canister ‘ledger’.
Failed to install wasm in canister ‘rrkah-fqaaa-aaaaa-aaaaq-cai’.
Failed to install wasm.
The replica returned an HTTP Error: Http Error: status 413 Payload Too Large, content type “”, content: Request 0x857cfbf34c9b4976d13f08d13f161266ec34dd710039d696ff23b74ac5d39e2a is too large. Message byte size 2494197 is larger than the max allowed 2097152.
Does anyone know a work around please? This same issue of size is observed with II canister, where I get a error 5 code, and can’t get the build done. I assumed it was my ubuntu, then I upgraded and its still there. Then I tried on a Mac-m1, and still this error. I also tried to build the interface myself using didc bind ledger.did -t ts > blablabla, and I can then call the functions on frontend, but still I am not able to complete my mocking of the Ledger/II in this way. Any advice would be appreciated. Thanks in advance.
PS: also note the project I have worked in the Hackathon was able to run on my team mates machine without problems. So its quite strange, and hence my immediately changing the local machine setup.
IIRC a replica version that accepts gzipped files only entered dfx with 0.10.0. Can you try with a later version that 0.9.3?
Thanks for replying: I’m using dfx 0.10.1 at present. So I guess dfx versions is not the issue. As an aside note: I suspected this so I tried to dfx toolchain install 0.10.0, and 0.9.3, to see if versions were the problem, but when looking into .cache I found that my versions included a full installation with moc of 0.10.1 (default) and only empty 10.0 and 9.3. (if you know how have to multiple dfx setup send that link please). Have you got any other suggestions of how I can get unstuck, please let me know, I have the xcode, cli-tools setup as well. And my ubunti is upto date. I also develop in wat2wasm on my Ubuntu to call from nodejs and I can easily do this. What could possibly be stopping this build. Shall I build the entire ledger and II with moc?
If you want to use a different version of dfx, it’s best to install it using DFX_VERSION=0.10.0 sh -ci "$(curl -fsSL https://internetcomputer.org/install.sh)". Substitute version numbers as needed. I honestly don’t know how the toolchain command really works.
Do you have the ic-cdk-optimizer installed? If not, try cargo install ic-cdk-optimizer and then run it over the wasm first. Does that maybe help?
How did you get the ledger.wasm ?
Did you use the following as described here ?
export IC_VERSION=dd3a710b03bd3ae10368a91b255571d012d1ec2f
curl -o ledger.wasm.gz https://download.dfinity.systems/ic/${IC_VERSION}/canisters/ledger-canister_notify-method.wasm.gz
gunzip ledger.wasm.gz
I’m just asking because there are different versions of the ledger.wasm floating around. This specific version is optimized to be small enough to be deployed on an application subnet. In earlier versions of dfx the included replica was configured to act like a system subnet which is less restricted than application subnets.
So I work in two projects( lets say Musall and MusicNova), both giving problems locally with II and Ledger wasm on ubuntu and Mac-m1, but installed differently in each. What I described above is for Musall, where I used the dfinity/examples/motoko/defi/ ledger versions repo at: examples/motoko/defi at master · dfinity/examples · GitHub (its up to date). I just copied these and installed as described in the release notes for 0.9.3. I might be wrong but these wasm and their associated .did are legitimate since I checked them line for line with the .did descriptions for II and nns-ledger on canLista. Now for MusicNova the II is installed via curl call in dfx.json as described on GitHub for the official II →
"canisters": {
"internet_identity": {
"__0": "The development build of Internet Identity. For more information, see https://github.com/dfinity/internet-identity#build-features-and-flavors",
"type": "custom",
"candid": "internet_identity.did",
"wasm": "internet_identity.wasm",
"__1": "There is no standard way to pull remote canisters, so instead we have a dummy build script that",
"__2": "simply downloads the Internet Identity canister. See also: https://github.com/dfinity/sdk/issues/2085",
"build": "curl -sSL https://github.com/dfinity/internet-identity/releases/latest/download/internet_identity_dev.wasm -o internet_identity.wasm"
},
.....
So I am pretty sure we have the correct II and Ledger wasm.
I have the latest rustup, rustc, cmake, wasm-wasi, they all work individually when tested on a simple hello-world file reader. The MusicNova gives this error exactly when building, please note I do have the ic-cdk-optimizer installed before running dfx …(cargo install ic-cdk-optimizer) →
Building canisters...
Executing: cargo build --target wasm32-unknown-unknown --release -p nova_one
Compiling nova_one v0.1.0 (/Users/lee/dfinity/reward-system/src/nova_one)
Finished release [optimized] target(s) in 5.37s
ic-cdk-optimizer not installed, the output WASM module is not optimized in size.
Run `cargo install ic-cdk-optimizer` to install it.
Building frontend...
Executing 'curl -sSL https://github.com/dfinity/internet-identity/releases/latest/download/internet_identity_dev.wasm -o internet_identity.wasm'
/Users/lee/.cache/dfinity/versions/0.10.1/base/Int.mo:65.5-65.18: warning [M0154], field hashNat8 is deprecated:
This function may be removed or changed in future.
/Users/lee/.cache/dfinity/versions/0.10.1/base/Int.mo:78.5-78.18: warning [M0154], field hashNat8 is deprecated:
This function may be removed or changed in future.
/Users/lee/dfinity/reward-system/src/fileupload/main.mo:20.23-20.32: warning [M0154], field hash is deprecated:
For large `Nat` values consider using a bespoke hash function that considers all of the argument's bits.
/Users/lee/dfinity/reward-system/src/fileupload/main.mo:143.39-143.51: warning [M0154], field append is deprecated:
`Array.append` copies its arguments and has linear complexity; when used in a loop, consider using a `Buffer`, and `Buffer.append`, instead.
Creating UI canister on the local network.
Jun 27 09:22:13.269 WARN s:wqsem-lwzmj-wa3ia-y3mqb-axtgs-vohbc-ycl3b-efl3o-aoo5r-e73q3-tae/n:uagvz-aior3-6gydl-qibur-dd6ew-wyipd-k7wk6-egodi-424h2-zrmah-iae/ic_execution_environment/scheduler At Round 67 @ time 2022-06-27 09:22:12.589350 UTC, the resulted state after execution does not hold the in-out cycles invariant: cycles at beginning of round 99595008205052 were fewer than cycles at end of round 199595008205052, messaging: {"round":67,"canister_id":null,"message_id":null}
The UI canister on the "local" network is "rno2w-sqaaa-aaaaa-aaacq-cai"
Installing code for canister fileupload, with canister ID rrkah-fqaaa-aaaaa-aaaaq-cai
Installing code for canister internet_identity, with canister ID ryjl3-tyaaa-aaaaa-aaaba-cai
Error: Failed to install wasm module to canister 'internet_identity'.
Caused by: Failed to install wasm module to canister 'internet_identity'.
Failed to install wasm in canister 'ryjl3-tyaaa-aaaaa-aaaba-cai'.
Failed to install wasm.
The Replica returned an error: code 5, message: "Canister ryjl3-tyaaa-aaaaa-aaaba-cai trapped explicitly: Custom(Cannot parse header
Caused by:
binary parser error: io error)"
…/end of log with errors
Also, I always start with a clean build from scratch, these are my alias from .bashrc:
alias dremove='rm -rf ./.dfx/local/'
alias dkill='killall dfx replica'
alias dst='dfx stop'
alias dd='dfx deploy
alias dsc='dfx start --clean'
alias dca='dfx canister create --all'
alias db='dfx build --all'
alias dci='dfx canister install --all'
alias dcanister='dremove && dsc --background && dca && db && dci && dd'
alias moc="$(dfx cache show)/moc --package base $(dfx cache show)/base"
Sorry for all the logs and commands, I wish to provide more info, as it might help diagnose the error.
I will give this a go, may I put it into my dfx.json or just run in console?
thanks for all feedback. The funny thing about the optimiser is that I build it with cargo install and gives a message of successful, yet when I cargo build I get this → ignoring invalid dependency ic-cdk-optimizer which is missing a lib target. On crate.io its marked with a red notice, that its not a library, but either way it should still install. I tried various ways to build it but always get some error which I assume is incorrect since I get a success message. here is the .toml →
[lib]
crate-type = ["cdylib"]
[dependencies]
candid = "0.7.14"
ic-cdk = "0.5.1"
ic-cdk-macros = "0.5.1"
serde = "1.0.137"
[build-dependencies]
walkdir = "2.3.2"
syn = { version = "1.0.96", features = ["full"] }
ic-cdk-optimizer = "0.3.4"
which shouldn’t be necessary since I already cargo install ic-cdk-optimizer = “0.3.4”. If I run it again I get →
Updating crates.io index
Ignored package `ic-cdk-optimizer v0.3.4` is already installed, use --force to override
as expected.
In the console. You have to manually download (and unzip) the ledger.wasm. Make sure that downloading worked. I have issues with copy & paste because it prefixes the curly braces with backslashes and then you download a 404 doc instead of the ledger wasm.