Hello everyone, dfx 0.26.0-beta.1 is available for testing. You can read the release notes here.
To switch to the beta release, use dfxvm default 0.26.0-beta.1.
If no bugs show up we plan to cut the full release next week.
9 Likes
adam@Ripper:~/projects/dragginz/backend$ dfx canister create --all
Error: Failed to create canister 'cdn_bucket'.
Caused by: Failed to construct wallet canister caller
Caused by: The replica returned an HTTP Error: Http Error: status 400 Bad Request, content type "text/plain", content: Canister bnz7o-iuaaa-aaaaa-qaaaa-cai does not belong to any subnet.
That’s new, I can’t find any reference to create canister changing in the notes. Here’s some dfx.json
"game": {
"type": "custom",
"candid": ".dfx/local/canisters/game/game.did",
"build": "bash -c 'backend/scripts/app/build.sh game'",
"wasm": ".dfx/local/canisters/game/game.wasm",
"shrink": true,
"gzip": true,
"metadata": [
{
"name": "candid:service"
}
]
},
Hi @borovan, are you seeing this when trying to create against your local network?
One of the big changes is that dfx start is now using pocket-ic by default.
Can you try with:
# With the replica
dfx start --replica --clean
And also:
# with pocketic
dfx start --clean
1 Like
Yes, passing --replica fixes it, thanks! I don’t think I would have worked that one out without help because I haven’t used pocketic.
1 Like
Running a local replica is still possible with
--replica, but this option will be removed in the near future.
Can I ask why this option will be removed? What if you want to test on the complete stack of a single node, and want to prioritise that over the mocked consenus and networking layers that PocketIC provides?
1 Like
The error message Canister bnz7o-iuaaa-aaaaa-qaaaa-cai does not belong to any subnet suggests that dfx uses the wallet canister bnz7o-iuaaa-aaaaa-qaaaa-cai from a local (replica) instance that you used in the past, but this wallet canister bnz7o-iuaaa-aaaaa-qaaaa-cai does not exist on your new local (PocketIC) instance (I anticipate this because the canister ID bnz7o-iuaaa-aaaaa-qaaaa-cai belongs to the canister ranges used by the local replica). I’d expect that using the --clean option should fix the issue. In particular, I couldn’t reproduce your issue with dfx canister create --all, the following sequence of commands succeeds for me:
$ dfx --version
dfx 0.26.0-beta.1
$ dfx new test
✔ Select a backend language: · Motoko
✔ Select a frontend framework: · No JS template
✔ Add extra features (space to select, enter to confirm) ·
Created new project "test"
$ cd test
$ dfx start --clean
Running dfx start for version 0.26.0-beta.1
Using shared network 'local' defined in /home/martin/.config/dfx/networks.json
Replica API running on 127.0.0.1:8080. You must open a new terminal to continue developing. If you'd prefer to stop, quit with 'Ctrl-C'.
and in another tab
$ dfx canister create --all
Created a wallet canister on the "local" network for user "default" with ID "uqqxf-5h777-77774-qaaaa-cai"
test_backend canister created with canister id: uxrrr-q7777-77774-qaaaq-cai
test_frontend canister created with canister id: u6s2n-gx777-77774-qaaba-cai
What scenario would you like to test using the local replica anticipating that PocketIC might not be suitable for that? (Please note that the local replica started by dfx start --replica is not a full-blown IC subnet as in production. In particular, it consists of a single node and thus the consensus and networking layers do not behave as in production either.)
Off the top of my head, IC OS election proposals should ideally allow for inspection by running the full stack locally (even if it’s just on one node) in additon to testing using PocketIC (which provides a representation of the consenus and networking layers).
Are you saying that there’s never going to be divergence in behaviour that would make testing on both a proper replica node, and a mocked consenus subnet, valuable?
1 Like
Our goal is that canister developers benefit from testing on PocketIC (multiple subnets, realistic cycles scaling, realistic message size limits, typically better performance etc.) because they don’t care about testing the IC protocol itself. For the latter use case, the dfinity/ic repository uses a collection of tools (unit tests, StateMachine tests, PocketIC tests, and VM-based system tests) with various levels of abstraction. I believe that using a single replica instance doesn’t substitute that collection of tools for testing the protocol itself (in my opinion, it doesn’t actually outperform any single tool from the collection) and thus I’m not sure about the value of keeping the local replica available in dfx.
Yeah, but the IC OS needs to be as testable as possible by third parties (not DFINITY). Thats what I’m trying to get at. By removing the ability to run on the full replica stack via dfx (even if it is just one node), don’t you think this makes it harder for anyone and everyone out there to test and inspect the IC OS?
I for one would like to continue reviewing IC OS election proposals in the future (like I used to when I was a member of CodeGov). If/when I do get that opportunity again (under CO.DELTA) we’d like to be able to run the latest replica releases locally (both using PocketIC, and as a proper replica node).
1 Like
I don’t think so because running a single replica doesn’t really tests the “full replica stack” since that single replica doesn’t really communicate with other nodes etc.
Could you please share more details on the test scenarios you’re executing when running the latest replica releases locally?
But it’s arguably better than PocketIC. IC OS election proposals make changes to the full Guest OS (and Host OS) - not PocketIC. Changes to PocketIC are not managed by NNS consensus. The utility of running PocketIC to inspect an IC OS release is limited (only covering the common ground).
There are plenty of scenarios where changes are made to the Guest OS but which have no effect on the behaviour of PocketIC, and vice versa. (Unless I’m misremembering)
It’s been 6 months or more since I’ve sat down and reviewed an IC OS proposal, and I never attempted to run the release (only build verification and browsing through the commits). To my understanding this is what every other IC OS election proposal reviewer is doing, but this is very limited in terms of levels of due diligence that can be exercised.
I understand that there are many things that simply can’t be practically tested by a member of the community (at the moment). But I don’t think that makes it right to make it even harder to run the code that a reviewer is supposed to be reviewing.
In the future I would very much like to be able to run the lastest release locally if I’m reviewing a proposal for that release. I think it goes without saying why that should be considered important. In fact, what I’d love to be able to do is spin up a bunch of high end VMs and run a fully fledged subnet in the cloud, and then spin them down when not in use. I know that’s a long way off (and probably not even on the current roadmap), but I don’t think the IC will really be adequately decentralised until something like that is possible.
My main concern is that what you’re suggesting sounds like a step in the wrong direction with regards to the promise and principles of web3. At least with respect to reviewing changes to the IC OS (as opposed to just the canisters running on that OS).
2 Likes
Correct, but the same argument applies to running a single replica instead of PocketIC. E.g., if the consensus and xnet layer were scrapped entirely, a single replica might continue to work as before.
That’s what it’d take to meaningfully test the “full replica stack”. Spinning up a single replica the way dfx does it at the moment is of no help towards that ultimate goal.
2 Likes
To be clear, DFX can connect to any address running the IC protocol. If you want to test the replica binaries for some replica-specific implementation detail, you can run ic-starter yourself as dfx has been doing for you. It all still works. We just don’t want to keep adding the extra 100MiB to the dfx distribution and basing the local dev model off of its limitations. dfx has never been an IC-OS verification tool; it is there to facilitate the local development environment and PocketIC is strictly better at being one. Third-party IC-OS verification tools are, of course, welcomed.
2 Likes
Thanks @AdamS. Can I ask how much of this post is still relevant in terms of the necessary steps for running a replica without using dfx? →
Cool. At the moment one could set up two VMs and run a replica on each, but obviously they’re not going to talk to each other. In principle, what would it take to get these two replica instances to form a two node subnet for community IC OS inspection / testing?
The steps have not really changed, though icx-proxy has been deprecated in favor of PocketIC’s gateway. We do not have good documentation on it right now but you can see how dfx initializes it here.
1 Like
If it can be useful, the scripts a compiled/used in Juno Docker. Ultimately I would like to migrate to pocket-ic too but, currently still using the replica so you can have a look there.
3 Likes
I’m not expecting to be able to dig into this any time soon as much as I’d love to (at least, unless the grants for voting neurons committee change their mind and the community votes for CO.DELTA).
Given the fact that dfx acts as the go-to documentation for how the replica should be run locally, I think the fact that there’s a plan to remove this from dfx is concerning.
A year or so from now, I gather it will no longer simply be possible to say →
…and in the meantime, dependencies will be changing, and potentially important steps may change with nothing to document this.
If the replica is no longer being consistently run via dfx, my concern is that there’s no pressure for the replica to remain something that’s easily runnable locally.
If this is going to be removed from dfx, would DFINITY please consider putting together some documentation to capture these sorts of details (documentation that’s kept up-to-date)? I’d love it if it could also tackle this question, even if it’s just an aspirational point.
2 Likes
A possible idea is to look at turning that into a dfx extension like the SNS or NNS extension. That way only people who need it can add it to their installation
Like this:
OR
2 Likes
I understand you want to be able to run the replica locally in the context of proposal verification, which is a laudable goal. Yet, as Martin already wrote above, the replica as used from dfx doesn’t run all of the protocol – so running a replica in this setting does not even give you anything realistic.
On the positive side, in the context of Utopia, we are building additional options for deploying the replica code. (Utopia is supposed to run, e.g., with cloud providers or on premises.) So while that is still under development right now, I think it should help address your concerns (in an actual better way than the replica as part of dfx) in the future.
5 Likes