Introducing ic-nix, a ready made, cross-platform, and bleeding edge dev environment!

  • Want to build latest of replica, motoko, dfx, etc. straight from source? Check!
  • Wonder where to get pre-built binaries? Check!
  • One liner to start canister development without waiting for official release? Check!
  • It is cross-platforms ready! Native on m1 macbooks, anyone? Check!
  • All system canisters and did files in one place? Check!
  • Looking for obscure binaries such as ic-prep or ic-admin? Check!

For example, to start canister development with platform-native dfx binaries in less than 10 seconds! (network conditions apply):

nix-shell https://github.com/ninegua/ic-nix/releases/latest/download/dfx-env.tar.gz

This will download pre-built binaries if your platform is supported by Github CI. But also offers an option if you decide to build from source (which may take quite a while)!

Because Github has yet to support aarch64, so for m1 macbook the only option now is to build from source (add --arg force true to the above command). So not quite ready made, but still it absolutely works! :grin: Edit: I uploaded pre-built binaries for aarch64 in this release. Enjoy!

New releases will automatically be made on a weekly basis. Or you can make PRs to have github build for you!

For non nix users, you can still check out the binaries nevertheless, if you are curious.

21 Likes

Just curious, do people in the community ever build the replica themselves? I can’t imagine why they would do so (even for node providers). In any case, thanks for sharing!

It is mostly because I had my 5 year old surface book replaced by a m1 macbook :grin:

But also I think it is good to get the knowledge out there “anyone can build all binaries”.

5 Likes

Hello Paul, this is a neat project … we are having issues trying to get a docker to run the prebuilt aarch64 binaries on an ubuntu:latest docker running on a M1 chip … the dfx start works and launches a replica and icx-proxy nicely, but when launching even the “dfx new hello” project with a “dfx deploy” it crashes the replica and starts to “stop” and then “start” after trying to create the default identity and wallet … here is the output:


[nix-shell:~]$ cd hello/

[nix-shell:~/hello]$ dfx start --background
binding to: 127.0.0.1:43861
Oct 16 19:08:25.513 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some("/nix/store/aiz4fvbd72yl3z9abb26brjrjv6adjpx-dfx-env/bin/replica"), replica_version: "0.8.0", log_level: Warning, debug_overrides: [], cargo_bin: "cargo", cargo_opts: "", state_dir: "/home/icpipeline/hello/.dfx/state/replicated_state", http_listen_addr: 127.0.0.1:0, http_port_file: Some("/home/icpipeline/hello/.dfx/replica-configuration/replica-1.port"), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: "/home/icpipeline/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool", crypto_root: "/home/icpipeline/hello/.dfx/state/replicated_state/node-100/crypto", state_manager_root: "/home/icpipeline/hello/.dfx/state/replicated_state/node-100/state", registry_local_store_path: "/home/icpipeline/hello/.dfx/state/replicated_state/ic_registry_local_store", unit_delay: None, initial_notary_delay: Some(600ms), dkg_interval_length: None, detect_consensus_starvation: None, consensus_pool_backend: Some("rocksdb"), subnet_features: SubnetFeatures { canister_sandboxing: false, http_requests: false, bitcoin_testnet_feature: None, bitcoin: None }, ecdsa_keyid: Some(EcdsaKeyId { curve: Secp256k1, name: "dfx_test_key" }), subnet_type: Application, bitcoin_testnet_uds_path: None, canister_http_uds_path: None, _state_dir_holder: None }, Application: starter
Oct 16 19:08:25.513 INFO Initialize replica configuration "/home/icpipeline/hello/.dfx/state/replicated_state/ic.json5", Application: starter
Oct 16 19:08:26.128 INFO Executing "/nix/store/aiz4fvbd72yl3z9abb26brjrjv6adjpx-dfx-env/bin/replica" "--replica-version" "0.8.0" "--config-file" "/home/icpipeline/hello/.dfx/state/replicated_state/ic.json5", Application: starter
Oct 16 19:08:26.748 ERRO s:cyuno-ysfo5-vyjnf-wg5cg-slonx-45dwi-hxtvq-guekf-uzxij-7fjta-nae/n:4pp23-zpnro-pilqs-g7ocq-4w2gj-3drhv-wy3o7-2eggg-6djqd-3axuo-aae/ic_canister_http_adapter_client/ic_canister_http_adapter_client Unable to connect to the canister http adapter. No UDS path provided.
Oct 16 19:08:26.748 WARN s:cyuno-ysfo5-vyjnf-wg5cg-slonx-45dwi-hxtvq-guekf-uzxij-7fjta-nae/n:4pp23-zpnro-pilqs-g7ocq-4w2gj-3drhv-wy3o7-2eggg-6djqd-3axuo-aae/ic_p2p/advert_utils AdvertRequestBuilder::new(): advert_config = Some(GossipAdvertConfig { best_effort_percentage: 20 })
version: 0.9.0
 Oct 16 19:08:26.940 INFO Log Level: INFO
 Oct 16 19:08:26.946 INFO Starting server. Listening on http://127.0.0.1:8000/

[nix-shell:~/hello]$

[nix-shell:~/hello]$

[nix-shell:~/hello]$

[nix-shell:~/hello]$ dfx deploy
Creating the "default" identity.
WARNING: The "default" identity is not stored securely. Do not use it to control a lot of cycles/ICP.
To create a more secure identity, create and use an identity that is protected by a password using the following commands:
    dfx identity create <my-secure-identity-name> # creates a password protected identity
    dfx identity use <my-secure-identity-name> # uses this identity by default

  - generating new key at /home/icpipeline/.config/dfx/identity/default/identity.pem
Created the "default" identity.
Creating a wallet canister on the local network.
thread 'MR Batch Processor' panicked at 'called `Option::unwrap()` on a `None` value', memory_tracker/src/lib.rs:282:73
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 Oct 16 19:08:34.996 WARN Internal Error during request:
hyper::Error(
    Io,
    Os {
        code: 104,
        kind: ConnectionReset,
        message: "Connection reset by peer",
    },
)
Error: Failed to get wallet canister caller for identity 'default' on network 'local'.
Caused by: Failed to get wallet canister caller for identity 'default' on network 'local'.
  Failed to get wallet for identity 'default' on network 'local'.
    Failed to create wallet for identity 'default' on network 'local'.
      Failed while installing wasm.
        The replica returned an HTTP Error: Http Error: status 500 Internal Server Error, content type "", content: Internal Server Error

[nix-shell:~/hello]$ Oct 16 19:08:35.158 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some("/nix/store/aiz4fvbd72yl3z9abb26brjrjv6adjpx-dfx-env/bin/replica"), replica_version: "0.8.0", log_level: Warning, debug_overrides: [], cargo_bin: "cargo", cargo_opts: "", state_dir: "/home/icpipeline/hello/.dfx/state/replicated_state", http_listen_addr: 127.0.0.1:0, http_port_file: Some("/home/icpipeline/hello/.dfx/replica-configuration/replica-1.port"), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: "/home/icpipeline/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool", crypto_root: "/home/icpipeline/hello/.dfx/state/replicated_state/node-100/crypto", state_manager_root: "/home/icpipeline/hello/.dfx/state/replicated_state/node-100/state", registry_local_store_path: "/home/icpipeline/hello/.dfx/state/replicated_state/ic_registry_local_store", unit_delay: None, initial_notary_delay: Some(600ms), dkg_interval_length: None, detect_consensus_starvation: None, consensus_pool_backend: Some("rocksdb"), subnet_features: SubnetFeatures { canister_sandboxing: false, http_requests: false, bitcoin_testnet_feature: None, bitcoin: None }, ecdsa_keyid: Some(EcdsaKeyId { curve: Secp256k1, name: "dfx_test_key" }), subnet_type: Application, bitcoin_testnet_uds_path: None, canister_http_uds_path: None, _state_dir_holder: None }, Application: starter
Oct 16 19:08:35.159 INFO Initialize replica configuration "/home/icpipeline/hello/.dfx/state/replicated_state/ic.json5", Application: starter
Oct 16 19:08:35.159 INFO Executing "/nix/store/aiz4fvbd72yl3z9abb26brjrjv6adjpx-dfx-env/bin/replica" "--replica-version" "0.8.0" "--config-file" "/home/icpipeline/hello/.dfx/state/replicated_state/ic.json5", Application: starter
Oct 16 19:08:35.717 ERRO s:cyuno-ysfo5-vyjnf-wg5cg-slonx-45dwi-hxtvq-guekf-uzxij-7fjta-nae/n:4pp23-zpnro-pilqs-g7ocq-4w2gj-3drhv-wy3o7-2eggg-6djqd-3axuo-aae/ic_canister_http_adapter_client/ic_canister_http_adapter_client Unable to connect to the canister http adapter. No UDS path provided.
Oct 16 19:08:35.717 WARN s:cyuno-ysfo5-vyjnf-wg5cg-slonx-45dwi-hxtvq-guekf-uzxij-7fjta-nae/n:4pp23-zpnro-pilqs-g7ocq-4w2gj-3drhv-wy3o7-2eggg-6djqd-3axuo-aae/ic_p2p/advert_utils AdvertRequestBuilder::new(): advert_config = Some(GossipAdvertConfig { best_effort_percentage: 20 })
Stopping icx-proxy...
Stopped.
version: 0.9.0
 Oct 16 19:08:35.775 INFO Log Level: INFO
 Oct 16 19:08:35.775 INFO Starting server. Listening on http://127.0.0.1:8000/
thread 'MR Batch Processor' panicked at 'called `Option::unwrap()` on a `None` value', memory_tracker/src/lib.rs:282:73
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

That is an interesting setup. So you run a native aarch64 docker on m1 chip?

What is the command line you used to start the nix-shell? BTW. thanks for trying out this project!

1 Like

I was able to reproduce the bug under aarch64 linux. Will try out a fix. Stay tuned!

2 Likes

thanks … let me know if I can help… also did nothing special to start it … just replaced the url with the aarch64 branch as you recommended in your amendment:

nix-shell https://github.com/ninegua/ic-nix/releases/download/20220519-aarch64/dfx-env.tar.gz

I made a fix but it will take a couple days to trickle down to the public repository of ic. Likely in next week’s release of ic-nix (dfx-env) we will have it.

1 Like

@mbeekman The problem should have been fixed in the latest release:

nix-shell https://github.com/ninegua/ic-nix/releases/download/20221024/dfx-env.tar.gz

Please try it out :slight_smile:

2 Likes

Awesome sauce! … I will try it and circle back with the result here

1 Like

NICE work Paul

not a dev but i got this up and running on my m1 mbp… we only need a couple more layers of abstraction and some co-pilot AI and I just may learn to “make” a program yet. :slight_smile:

2 Likes

Just closing the loop here … it came up with another thread on gihub about dfx … thanks again Paul … your works is appreciated…