Errors creating a canister with 'dfx deploy'

I’m trying to set up a project locally, following the quickstart guide but I’m getting errors at the ‘dfx deploy’ stage.

Firstly, I was being told my account needed to be an administrator (which it was), but I seem to be able to get past that step. The very next part, though…

I get this message in the shell:
“The replica returned an HTTP Error: Http Error: status 400 Bad Request, content type “text/plain; charset=utf-8”, content: Failed to connect to host: Connection refused (os error 61)”

And at this point the other shell window, running the server goes nuts and gives an error every few milliseconds along the lines of “Couldn’t find transcript at height 200 with finalized height 199…”

From that point ‘dfx start’ always throws the height error above until I start clean. But it never gets past the bad request error.

Anyone come across this before or have any troubleshooting steps I can try?

Thanks

2 Likes

Slightly different but similar error next attempt:

“The replica returned an HTTP Error: Http Error: status 400 Bad Request, content type “text/plain; charset=utf-8”, content: Failed to connect to host: Internal error: connector has been disconnected”

And in the replica terminal"

“Mar 12 10:01:31.666 WARN s:fscpm-uiaaa-aaaaa-aaaap-yai/n:uuafg-yid6r-7twlj-jasi4-5tzba-xpd4b-qrqkq-tzu4k-rn3op-r7o7y-nae/ic_consensus/block_maker Cannot propose block as the locally available validation context is smaller than the parent validation context (locally available=ValidationContext { registry_version: 1, certified_height: 7, time: Time(1615543291652678000) }, parent context=ValidationContext { registry_version: 1, certified_height: 99, time: Time(1615543240160242000) })”

1 Like

Try running dfx start --clean, instead of dfx start, this will clear the local cache. Or you could manually delete the .dfx directory in your project. See if that helps the issue.

7 Likes

Thanks for the suggestion, Ori. I have tried both of those things several times. It still throws up the http error sadly.

Bumping this up as I’m still having the same issue on the latest beta version of DFX - just slightly further down the line now.

Canisters get created, but in the tutorial when I run
"dfx canister call explore_hello greet everyone" or sometimes even at the canister create part I get:

The replica returned an HTTP Error: Http Error: status 400 Bad Request, content type "text/plain; charset=utf-8", content: Failed to connect to host: Connection refused (os error 61)

And in the dfx start window, the replicas keep restarting on a different localhost. I try to start clean but the same thing happens.

If I manage to get through the build and deploy process quickly, it works for a while, but the dfx build operation seems to always restart on a different host after a short amount of time (a few minutes).

1 Like

What’s your development environment, if you don’t mind sharing?

I’m on macOS but it is an old Macbook. Running High Sierra (can no longer get the newest OS).

Just trying to follow the tutorials using terminal locally. I can’t use the ic network to deploy now, since I don’t want to pay for cycles to do the tutorials.

Not sure what exactly you need to know that might help.

Thanks

I was thinking things like the machine spec. RAM and free disk space (ssd or hdd). But I’m a Windows/Linux user so don’t know what the requirements are for a Mac, but maybe some on the forum has relevant experience.

Honestly it’s just so frustrating. I get I’m not using brand new hardware, but I should expect to be able to deploy a project locally.

The MO of the ICP for a talented developer anywhere to be able to deploy code comes into question, given they’ll also likely be using any hardware they can get…

cd ~
rm -rf ~/.cache/dfinity
sudo rm /usr/local/bin/dfx
DFX_VERSION=0.7.0-beta.8 sh -ci "$(curl -fsSL https://sdk.dfinity.org/install.sh)"
export TMPDFXDIR=$(mktemp -d)
cd $TMPDFXDIR
dfx new hello
cd hello
dfx start --background
dfx deploy
dfx canister call hello greet "Bob"
dfx stop
cd ~
rm -rf $TMPDFXDIR

Can you run the above line by line, and confirm if you still see the issue?

Hi @prithvi. I’ve just tried and the same thing happens, after a few minutes. I can execute the canister call once or twice, but then the dfx server errors and keeps resetting itself to a new replica number on the localhost.

Thanks for trying that out!

the dfx server errors and keeps resetting itself to a new replica number on the localhost.

When the replica stops and restarts, are you still seeing logs like what you reported earlier?:

“Mar 12 10:01:31.666 WARN s:fscpm-uiaaa-aaaaa-aaaap-yai/n:uuafg-yid6r-7twlj-jasi4-5tzba-xpd4b-qrqkq-tzu4k-rn3op-r7o7y-nae/ic_consensus/block_maker Cannot propose block as the locally available validation context is smaller than the parent validation context (locally available=ValidationContext { registry_version: 1, certified_height: 7, time: Time(1615543291652678000) }, parent context=ValidationContext { registry_version: 1, certified_height: 99, time: Time(1615543240160242000) })”

Or something else entirely?

2 Likes

Thanks for responding. It looks different this time - but can’t guarantee this is always the error that shows. I’ve copied the output between the last successful canister call and the dfx stop:

 ✝  T/tmp.6g3jvPCG/hello   master  May 15 18:27:07.320 CRIT s:3pirb-s2tvo-khiz5-bsyhq-xzhlg-nzqt4-7w2c5-j3zvz-vrjk6-wql5b-bae/n:6rtfv-4rjch-wd6d3-cjsbd-xubrv-nwajn-3jvhw-d3bvp-naxrs-ctcyg-7ae/ic_state_manager/ic_state_manager Failed to make a checkpoint @100: IoError { path: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/state/tip", message: "Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)", io_err: "Operation not supported (os error 45)" }
thread 'MR Batch Processor' panicked at 'Failed to make a checkpoint @100: IoError { path: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/state/tip", message: "Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)", io_err: "Operation not supported (os error 45)" }', src/lib.rs:1729:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
May 15 18:27:07.542 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some("/Users/mattdomm/.cache/dfinity/versions/0.7.0-beta.8/replica"), replica_version: "0.1.0", log_level: Warning, cargo_bin: "cargo", cargo_opts: "", state_dir: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state", http_listen_addr: V4(127.0.0.1:0), http_port_file: Some("/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/replica-configuration/replica-1.port"), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool", artifact_backup_dir: None, crypto_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/crypto", state_manager_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_staStarting webserver for replica at "http://localhost:52793"
binding to: V4(127.0.0.1:8000)
replica(s): http://localhost:52793/
May 15 18:27:18.959 CRIT s:3pirb-s2tvo-khiz5-bsyhq-xzhlg-nzqt4-7w2c5-j3zvz-vrjk6-wql5b-bae/n:6rtfv-4rjch-wd6d3-cjsbd-xubrv-nwajn-3jvhw-d3bvp-naxrs-ctcyg-7ae/ic_state_manager/ic_state_manager Failed to make a checkpoint @100: IoError { path: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/state/tip", message: "Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)", io_err: "Operation not supported (os error 45)" }
thread 'MR Batch Processor' panicked at 'Failed to make a checkpoint @100: IoError { path: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/state/tip", message: "Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)", io_err: "Operation not supported (os error 45)" }', src/lib.rs:1729:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
May 15 18:27:19.146 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some("/Users/mattdomm/.cache/dfinity/versions/0.7.0-beta.8/replica"), replica_version: "0.1.0", log_level: Warning, cargo_bin: "cargo", cargo_opts: "", state_dir: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state", http_listen_addr: V4(127.0.0.1:0), http_port_file: Some("/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/replica-configuration/replica-1.port"), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool", artifact_backup_dir: None, crypto_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/crypto", state_manager_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_staStarting webserver for replica at "http://localhost:52796"
binding to: V4(127.0.0.1:8000)
replica(s): http://localhost:52796/
dfx canister call hello greet "Bob"
May 15 18:27:32.454 CRIT s:3pirb-s2tvo-khiz5-bsyhq-xzhlg-nzqt4-7w2c5-j3zvz-vrjk6-wql5b-bae/n:6rtfv-4rjch-wd6d3-cjsbd-xubrv-nwajn-3jvhw-d3bvp-naxrs-ctcyg-7ae/ic_state_manager/ic_state_manager Failed to make a checkpoint @100: IoError { path: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/state/tip", message: "Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)", io_err: "Operation not supported (os error 45)" }
thread 'MR Batch Processor' panicked at 'Failed to make a checkpoint @100: IoError { path: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/state/tip", message: "Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)", io_err: "Operation not supported (os error 45)" }', src/lib.rs:1729:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The replica returned an HTTP Error: Http Error: status 500 Internal Server Error, content type "text/plain; charset=utf-8", content: IO error: Connection reset by peer (os error 54)
 ✘ ✝  T/tmp.6g3jvPCG/hello   master  May 15 18:27:32.719 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some("/Users/mattdomm/.cache/dfinity/versions/0.7.0-beta.8/replica"), replica_version: "0.1.0", log_level: Warning, cargo_bin: "cargo", cargo_opts: "", state_dir: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state", http_listen_addr: V4(127.0.0.1:0), http_port_file: Some("/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/replica-configuration/replica-1.port"), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool", artifact_backup_dir: None, crypto_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/crypto", state_manager_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_staStarting webserver for replica at "http://localhost:52801"
binding to: V4(127.0.0.1:8000)
replica(s): http://localhost:52801/
dfx stop
Stopping the replica...
Stopped.
 ✝  T/tmp.6g3jvPCG/hello   master  cd ~
1 Like

Ok, that’s unfortunate.
Based on the log output, I’m suspicious about the version being run…

from the logs you provided, here’s how ic-starter configures the replica

:latin_cross:  T/tmp.6g3jvPCG/hello   master  May 15 18:27:32.719 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some(“/Users/mattdomm/.cache/dfinity/versions/0.7.0-beta.8/replica”), replica_version: “0.1.0”, log_level: Warning, cargo_bin: “cargo”, cargo_opts: “”, state_dir: “/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state”, http_listen_addr: V4(127.0.0.1:0), http_port_file: Some(“/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/replica-configuration/replica-1.port”), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: “/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool”, artifact_backup_dir: None, crypto_root: “/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_state/node-100/crypto”, state_manager_root: "/private/var/folders/ym/bps2v83504lcmrlng_lzjrqr0000gn/T/tmp.6g3jvPCG/hello/.dfx/state/replicated_staStarting webserver for replica at “http://localhost:52801

It’s missing a few things since we’ve changed the flags sent to ic-starter (as of 0.7.0-beta.1)
For example, my output for ic-starter looks like

May 15 18:49:05.826 INFO ic-starter. Configuration: ValidatedConfig { replica_path: Some(“/Users/prithvishahi/.cache/dfinity/versions/0.7.0-beta.8/replica”), replica_version: “0.1.0”, log_level: Warning, cargo_bin: “cargo”, cargo_opts: “”, state_dir: “/private/var/folders/l5/rhsc__9j79b_jrk2dl8k8d940000gp/T/tmp.yECDevMl/hello/.dfx/state/replicated_state”, http_listen_addr: V4(127.0.0.1:0), http_port_file: Some(“/private/var/folders/l5/rhsc__9j79b_jrk2dl8k8d940000gp/T/tmp.yECDevMl/hello/.dfx/replica-configuration/replica-1.port”), metrics_addr: None, provisional_whitelist: Some(All), artifact_pool_dir: “/private/var/folders/l5/rhsc__9j79b_jrk2dl8k8d940000gp/T/tmp.yECDevMl/hello/.dfx/state/replicated_state/node-100/ic_consensus_pool”, artifact_backup_dir: None, crypto_root: “/private/var/folders/l5/rhsc__9j79b_jrk2dl8k8d940000gp/T/tmp.yECDevMl/hello/.dfx/state/replicated_state/node-100/crypto”, state_manager_root: “/private/var/folders/l5/rhsc__9j79b_jrk2dl8k8d940000gp/T/tmp.yECDevMl/hello/.dfx/state/replicated_state/node-100/state”, registry_local_store_path: “/private/var/folders/l5/rhsc__9j79b_jrk2dl8k8d940000gp/T/tmp.yECDevMl/hello/.dfx/state/replicated_state/ic_registry_local_store”, unit_delay: None, initial_notary_delay: None, detect_consensus_starvation: None, consensus_pool_backend: Some(“rocksdb”), state_dir_holder: None }, Application: starter

You’ll notice consensus_pool_backend: Some("rocksdb")
The commands to manually delete the cache and reinstall dfx should have sorted it out. I’ll have to look further

Can you try to create the project in non tmp dir, somewhere in home and confirm you don’t see

“Failed to convert tip to checkpoint to 0000000000000064 (err kind: Other)”, io_err: “Operation not supported (os error 45)” }

1 Like

Thank you for getting back.

I have set up the project in home, and exactly the same thing is happening.

This error is showing:

1 Like

Ok, I’ll need to circle with other folks to see what could be happening. Nothing stands out to me.

2 Likes

Thank you again.

Please do keep me updated and let me know if I can help with testing. I’m really keen to get going with Motoko but this is holding things back.

1 Like

@canicodenow While we’re investigating further, can you try dfx start --emulator as a workaround. This runs an emulator instead of the actual replica and for all intents and purposes should unblock you for local development.
Also we released 0.7.0, try to install that and run it once more (as it has a new replica version)
Lastly, please run diskutil info / | grep "File System Personality" and paste the output

1 Like

Thanks for the workaround - that does appear to work.

I have installed version 0.7.0 (was already running the beta) and it still isn’t working, I’m afraid.

Here is the requested output:

   Device Identifier:        disk0s2
   Device Node:              /dev/disk0s2
   Whole:                    No
   Part of Whole:            disk0

   Volume Name:              Matt's Mac
   Mounted:                  Yes
   Mount Point:              /

   Partition Type:           Apple_HFS
   File System Personality:  Journaled HFS+
   Type (Bundle):            hfs
   Name (User Visible):      Mac OS Extended (Journaled)
   Journal:                  Journal size 40960 KB at offset 0xe8e000
   Owners:                   Enabled

   OS Can Be Installed:      Yes
   Booter Disk:              disk0s3
   Recovery Disk:            disk0s3
   Media Type:               Generic
   Protocol:                 SATA
   SMART Status:             Verified
   Volume UUID:              A1841240-428C-3B11-B826-EF973F0033DB
   Disk / Partition UUID:    EAFD8EE2-CD55-4919-BBF7-F55C5A94ED15
   Partition Offset:         209735680 Bytes (409640 512-Byte-Device-Blocks)

   Disk Size:                459.2 GB (459248103424 Bytes) (exactly 896968952 512-Byte-Units)
   Device Block Size:        512 Bytes

   Volume Total Space:       459.2 GB (459248103424 Bytes) (exactly 896968952 512-Byte-Units)
   Volume Used Space:        429.8 GB (429750845440 Bytes) (exactly 839357120 512-Byte-Units) (93.6%)
   Volume Free Space:        29.5 GB (29497257984 Bytes) (exactly 57611832 512-Byte-Units) (6.4%)
   Allocation Block Size:    4096 Bytes

   Read-Only Media:          No
   Read-Only Volume:         No

   Device Location:          Internal
   Removable Media:          Fixed

   Solid State:              No
   Hardware AES Support:     No
1 Like

@canicodenow do you have ~/.dfx running on the root HFS+ partition? Could you maybe also share the output of df and diskutil list to double check if you’re not sure?

1 Like