Let's have a candid discussion about Candid

[icpp]

When I first looked at the IC, and tried to understand what it was all about, my first experiment was to code up a JSON request/response canister. Once I got that working I was convinced this is worth digging in further and I applied for a developer grant.

I imagine that many developers will come to the IC like that, and maybe we need to focus on that early learning stage first.

As a developer I am now in a spot where I really like Candid. I can read .did files, and understand the interface and value the explicit typing. It does help a lot when trying to understand a new canister interface.

I think there is place for both.

[NathanosDev]

I think this is a great initiative! Prioritizing tooling, languages and frameworks that developers are already familiar with is almost always a good thing in my opinion. The cases where I don’t think it’s a good thing are when we limit progress by pigeonholing ourselves into existing patterns or technologies.

It has already been mentioned, but Candid’s support for subtyping helps move us towards a future where open internet services can safely rely on each other without fear of the interface breaking. There are standards and best practices in web2 to avoid breaking changes in APIs, but they exist at a project level, not at a technology level and so they can differ greatly from project to project. Candid takes this concept and introduces it on a technology level, which is progress that I personally think we should find ways to leverage in developer friendly ways, rather than abandon entirely.

I’m a little skeptical about supporting window.fetch. Achieving this for all possible scenarios with HTTP certification puts an insane amount of work on the HTTP Gateway. The HTTP Gateway is useful for allowing HTTP clients (like a web browser) to talk to the Internet Computer and we have HTTP Certification to facilitate this in a secure and performant way. As already mentioned though, it’s extremely limiting. It works great for static content, but shows it’s limitations as soon as you start adding dynamic content.

A simple GET endpoint returning a list of items that can be filtered, sorted and paginated very quickly generates an infinite number of combinations to certify. Then if you add authentication (the endpoint returns a different list of items depending on who the caller is) the HTTP Gateway also needs to be aware of the authentication mechanism in order to verify this effectively. There is always the option of using the “upgrade to update” feature to handle these cases.

In the future there will be an “upgrade to certified query” feature that would also not require explicit canister-side certification. I don’t know where this sits on the roadmap and I’m also not sure of it’s performance characteristics. We can safely assume it sits somewhere between a normal query call and an update call, but it’s unclear how big the difference will be from a normal query call. So I can imagine some kind of canister framework that would try to certify static data to a point, upgrade to certified queries for dynamic data and then upgrade to update calls for cases where canister state mutation is needed, but I struggle to imagine this framework making all of those decisions automatically. I think developers will still need to learn when to use which flavor of call.

A side note on the authentication mechanism. Secure enclaves will not help with privacy or security here if we’re pursuring window.fetch support because SSL is terminated on the HTTP Gateway and the gateways can then see the entire response. It’s also possible to add support for secure enclaves in the HTTP Gateways, but I’m personally not a fan of this idea because I really like the idea of having serverless HTTP Gateways running for example on Cloudflare or Fastly and leveraging the same huge CDN infrastructure that web2 gets to take advantage of. Secure enclaves would limit us to only using hardware that supports it.

So even if we don’t follow the idea of supporting window.fetch, there may still be some value in using JSON as Candid. If a canister only implements the http_request and http_request_update endpoints, then you can interact with it using a JS client without needing to generate Candid typings. I put together one such JS client as an experiement here. This does lose the benefits of Candid, but simplifies the developer onboarding experience.

You can then combine this with client side certification, for example with this other (also slightly experimental) lib. This allows certification to remain flexible by allowing applications to define their own mechanisms, rather than trying to make a one-size-fits-all solution on the HTTP Gateway. That of course increases complexity again, but there is always the “upgrade to update” and “upgrade to certified query” options I mentioned above, which can be done locally the same way they are done on the HTTP Gateway.

[fredericrous]

Frontend dev here! specialized in react

• candid spec is appreciated to describe the backend but it lacks tooling (that’s why I would prefer openapi)
• json is appreciated for the body of requests and for looking at the network tab without a chrome extension

My first affirmation can seem weird to most devs. It is because I don’t wish candid files generated from code but the other way around. Indeed, the best DevX I experienced within a large team was to adopt API-First development:

• Backend Dev writes the spec
• Frontend Dev reviews
• Backend Dev merges the updates to the spec
• “backend code” gets generated: request data types and code to do inter-microservices calls
• frontend code gets generated: we get react hooks. We don’t care if the frontend and backend communicate with json, xml or 0 and 1, we don’t need to know. It is encapsulated by the code generated by the openapi code generator.

I don’t know if moving away from candid is the right solution, but if it is to support expressjs, go for it I would even be more excited to see Remix support
In my eyes, we need more tooling around candid: like react hook generation. When code generation is not in place, a tremendous amount of time is wasted by react devs building hooks to fetch requests

[b3hr4d]

I’ve developed a tool that might be of interest to you.

  const { call, error, loading } = useUpdateCall({
    functionName: "addTodo"
  })

  return  <button onClick={() => call([todo])}>Save</button>

You can find it on npm at @ic-reactor/react - npm. While the documentation is still a work in progress, also vanilla JavaScript version accessible at @ic-reactor/core - npm. This is part of our broader initiative to provide robust tools for the Internet Computer ecosystem.

Additionally, I’m currently exploring some innovative methods to generate form data based on Candid. This is part of my ongoing effort to enhance the usability and functionality of our tools within the Internet Computer ecosystem.

I am actively seeking help to extend this to Svelte. Additionally, any assistance in enhancing and fixing the documentation would be greatly appreciated. We’re also open to pull requests. Collaborative efforts like these not only improve the tool but also foster a more vibrant and resourceful community. If you have expertise in Svelte or technical writing and are interested in contributing, your input would be invaluable.

Repo - B3Pay/ic-reactor: IC-Reactor: A suite of JavaScript libraries for seamless frontend development on the Internet Computer platform, offering state management, React integration, and core functionalities for efficient blockchain interactions. (github.com)

[rumenov]

@skilesare

• How does the IC handle transaction de-dupe at the ingress layer(or does it even). There is specific code in the ledger canisters trying provide some dedupe code, but I also assumed that there was also something happening at the agent/request level to make sure that a malicious boundary node can’t just keep feeding your signed request back into the IC over and over. (ie, someone grabs your post to taggr and then spams taggr with it over and over until your run out of charge, creating the same post over and over). So does the IC do something to keep this from happening, and if so, what does a rest-based access need to do to get the same protections? Ethereum transactions have a nonce that is always increasing that has to be submitted with each request and the state machine inside the evm actually keeps track of this nonce for every account. Duplicating that kind of architecture across subnets would be untenable. Perhaps the agent is doing something under the hood with the root and/or subnet sig to kee things from being dipped(maybe with a sliding trx window). Or maybe it is all on the canister writer in which case we need to beef up our tutorials on defending against malicious boundary nodes.

The IC uses the combination of expiry_time and nonce to de-dupe at the ingress layer.

The Internet Computer Interface Specification | Internet Computer

When user creates a request an expiry time is be set (that is not more than 5 min in the future). Since the request is signed, the IC will make sure this request is processed only once for the duration of the expiry. If a request is received after it’s expiry time it won’t be processed. The nonce parameter allows user to resend the same twice if desired.

[domwoe]

@infu

let me = await dfx.identity("some").getPrincipal()

await dfx.build("main");

await dfx.canister(GOVERNANCE_CANISTER_ID)
.install("./build/main.wasm")
.candid("./build/main.did")
.initParams({ 
   admin: me,
   some: [1,2,3,4,5],
   logo: await getFile("./mylogo.png")
   });

I like that a lot! @Tbd weren’t you working on something like that? Besides the build step that seems pretty straightforward.

In general, I think this is an important discussion. Thanks, @lastmjs for starting it! I’m not so sure about retiring Candid entirely, but I’m definitely in favor of trying to hide it from new developers as much as possible.

[josephgranata]

@lastmjs

This is our observation over time as we engage in education, and especially at in-person hackathons with relatively young web developers (and completely new to ICP). It is especially difficult for a dev to figure out how to hook up their frontend to their backend, as they must generate bindings, hook up an agent, and engage in RPC calls, as opposed to just using regular HTTP with window.fetch and JSON.

This is one of the many issues about programming for the IC, the difficulty of making sense of the connections between front-end and Canister-IC backend. It is far from trivial, especially when compared with standard web development where it is seamless, especially if one uses JavaScript on both ends, but even if the backend is Rust or Java or Kotlin or whatever: things just work.

I do like Candid, I think it could be modernized somehow to play better with REST, and have a “raw mode” to skip some of the hurdles it currently has.

Question for DFINITY: why not start a Rust Framework project that takes one of the top REST development oriented frameworks and port it to the IC? How about taking Actix Web, or Rocket or any other main web framework and extending it, or porting it to the IC? I would prefer that much more than continuing with the efforts today where we have to rebuild a lot of the backend infrastructure from primitives. Primitives are fine, thanks for providing them, but please focus now on providing a powerful IC Framework, I vote for a Rust fork of Actix Web or Rocket.

Maybe a couple more wishes since we are talking almost year end requests: First a real file system, or a file system inside a canister. And please do not reply that we can “build it” I am talking a working solution that every Web2 cloud already has. And yes my second wish is a database, even a small one that works reliably, does REST, and yes it’s officially supported and performs decently.

Well those are my year end wishes, Merry Christmas!

[kpeacock]

I like this vision a lot! Some of these are bigger problems than even ICP - the whole wasm community would benefit from good filesystem tooling or http bindings