Self-describing Services standard discussion

In Joachim Breitner’s last blog on Candid he ends with this:

Self-describing Services

As you have noticed, Candid was very much designed assuming that all parties always have the service type of services they want to interact with. But the Candid specification does not define how one can obtain the interface of a given service, and there isn’t really a an official way to do that on the Internet Computer.

That is unfortunate, because many interesting features depend on that: Such as writing import C "ic:7e6iv-biaaa-aaaaf-aaada-cai" in your Motoko program, and having it’s type right there. Or tools like ic.rocks, that allow you to interact with any canister right there.

One reason why we don’t really have that feature yet is because of disagreements about how dynamic that feature should be. Should you be able to just ask the canister for its interface (and allow the canister to vary the response, for example if it can change its functionality over time, even without changing the actual wasm code)? Or is the interface a static property of the code , and one should be able to query the system for that data, without the canister’s active involvement. Or, completely different, should interfaces be distributed out of band, maybe together with API documentation, or in some canister registry somewhere else?

I always leaned towards the first of these options, but not convincingly enough. The second options requires system assistance, so more components to change, more teams to be involved that maybe intrinsically don’t care a lot about this feature. And the third might have emerged as the community matures and builds such infrastructure, but that did not happen yet.

In the end I sneaked in an implementation of the first into Motoko, arguing that even if we don’t know yet how this feature will be implemented eventually, we all want the feature to exist somehow, and we really really want to unblock all the interesting applications it enables (e.g. Candid UI). That’s why every Motoko canister, and some rust canisters too, implements a method

__get_candid_interface_tmp_hack : () -> (text)

that one can use to get the Candid interface file.

The name was chosen to signal that this may not be the final interface, but like all good provisional solutions, it may last longer than intended. If that’s the case, I’m not sorry.

I actually thought because of things like candidUI and ICRocks that option 1 was already the case. I am working on an application myself where I want to be able to call
functions on canister A from canister B without knowing in advance what those functions will be.

In general it seems like a really nice thing to have from the perspective of creating composable services on the IC. An example usecase that I can think of is downloading the interface of a service and mocking it for local development even if the code is not open sourced.

I would love to hear the thoughts from the foundation and community around this topic. In the meanwhile I will always put __get_candid_interface_tmp_hack : () -> (text) in my rust canisters and I hope everyone else will do the same.

6 Likes

The third of @nomeata ’s options actually does exist: Canlista ; ) It was a primary goal to aid composability of services like this. Some discussion went on about how rust canisters could expose their interface signatures since some types couldn’t be directly mapped with candid, so we went with allowing them to choose the candid they present on their listing: You can store your Rust canister candid there for others to retrieve (Motoko canister candid will be automatically populated for you), and they’re versioned so changes over time can be compared. The candid can also be queried by other canisters or frontend apps via the getCandid endpoint documented here: Canlista: docs

It will get more features over time, but for now this is a driving use case, so at least while we all figure out the best approach the more developers support the registry by listing their canisters, the more useful in this function it can become!

2 Likes

Thank you for starting this thread. We are planing to include the candid interface as metadata in the Wasm module directly. Users can then query the state tree or call the management canister to get the metadata.

This feature is also very important to ensure safe upgrade, i.e. the interface from the current code is backward compatible with the running canister.

Once the feature is implemented, we will eventually deprecate __get_candid_interface_tmp_hack. The Rust CDK will also include the candid interface by default.

7 Likes

Is this now live in dfx 0.11.0? How do we get the candid file from a Rust canister now?

1 Like

If you build and deploy the canister with 0.11.0, it will be ready as metadata. You can only get the did file via agent, not via inter-canister calls.

For agent-js, see this example: agent-js/index.test.ts at main · dfinity/agent-js · GitHub
For ic-agent, use this function: Agent in ic_agent::agent - Rust. The path is candid:service

3 Likes

Does this mean we can skip providing a path to the Candid file in dfx.json? Does this work for custom canisters as well?

You still need to provide did file in dfx.json. dfx build will write this did file into the Wasm module as part of the post-build process. It works for both Rust and custom canisters.

1 Like

It would be nice to remove this requirement and have a DX like Motoko