I’d like to start a discussion with the community and the stakeholders at DFINITY about the use of ic-ref as an IC emulator in dfx.
In the wake of introducing the Interface Specification, which defines the externally visible behaviour of the Internet Computer, we created a reference implementation called
ic-ref. This is a completely independent implementation of the Interface Specification, and helped us to give the Spec more weight and maturity, by showing that it can be implemented. It also helped with testing of the agents and CDKs: By running their test suite against both production and reference implementation of the IC, when things don’t work, the cause can be triangulated more easily. Finally, it was (or was meant to be) a way to experiment with new features, and allow agents and CDKs to work on supporting them even before the replica has them.
As a reference implementation,
ic-ref takes many shortcuts: Not optimized for performance, not networked, not necessarily secure, no disk usage. Some of these properties make it actually a nice way to run canisters locally, so a while ago, we started shipping
dfx, and you can run your program against it instead of the production replica by running
dfx start --emulator.
We also added features to
ic-ref that are needed particularly for its use as a dev emulator, in particular the ability to dump and restore the state, so that you can install your canisters, stop the emulator, start it later, and don’t have to re-install them.
There were bolder visions, of course: A self-contained, simpler implementation of the IC with a Wasm interpreter could be extended with all kinds of cool development features that we may not want in the real replica: Better logging, live detailed debugging, manual control over message scheduling, manual insertion of faults etc.
But resources are limited, some of the poeple excited about this kind of work left or were made to leave the company shortly after I left, so these extra features did not happen.
Also, there were ideas of writing a fully dedicated dev emulator separate from
ic-ref, which made it unclear if we should add them to
ic-ref (but these also have moved on).
So now we are in the unfortunate state that
ic-ref is shipped as the emulator, but doesn’t provide that much added value. At the same time it adds cost and friction:
- The features that were added only for the emulator use case (mostly state serialization) need to be maintained, slow down building etc. Removing unused code is always good.
ic-refin a way that an be distributed in a self-contained, distibution-agnostic way for Linux and MacOS is relatively tricky (and currently prevents a dependency bump in the repo, which is why I am writing this now, to procrastinate from debugging this issue).
dfxrelease process is tied to yet another repository and toolchain, which adds complexity.
- As far as I know there are currently no plans to add more dev-focused features to
ic-ref, which would require more dev resources.
So, is having
ic-ref shipped with
dfx useful to anyone? Has anyone here ever used it for good effect? Or can we just remove this? Is this question worth an NNS motion?
(I am not questioning the use of
ic-ref as the reference implementation, nor its use in the test suite of agents, CDK, or in canister tests like the one for Internet Identity, were the usefulness is quite clear.)
Pinging @kpeacock as the
dfx representative here.