Deduplication of Candid Types in Rust

Developers
1

Hi, our project has a large data model, we’re using Unity, and Rust. The software that we’re using to generate the C# classes for Unity uses the .did files that are derived from the CandidType macro in the rust library.

Anyway, we have so many types that have been changed by the de-duplication code that everything is a mess. Rust doesn’t use structural typing and we do have a lot of newtypes end up having the same structures (albeit with different traits).

We’ve got stuff going on like

set_name ( Description )
update_item_set ( AbilitySet )
create_rarity ( Quality )

and without forking Candid or rewriting the CandidType derive macro (actually I’m not even sure that helps), we’re out of luck.

We can’t just write our own candid because it’s thousands of lines that change daily. I get that this is the way Candid is supposed to work, but the option to replace the .did files exists, but the CandidType macro is great, exactly what we need other than being overly restrictive. Currently we’ve got 744 fields over 180 or so structs (not including metadata, new types, enums, primitives etc…)

ChatGPT tracked down the part of the code that’s causing us so much grief. Anyway thanks!

image
image726×929 45.7 KB

2 Likes
2

image

this is our current solution and it’s horrible and doesn’t even cover all of the cases

image
image735×401 16.2 KB

3

lol, that function ChatGPT displayed doesn’t exist. But it’s right that by changing the implementation of TypeContainer, you can change the deduplication behavior.

Let me understand the problem a bit more. The problem is that the generated did file changes a lot after a commit, which causes the generated C# classes to change? I don’t have a perfect answer, but I can see some speculative workaround:

  • Forking Candid to turn off the deduplication. I think it’s possible to turn off most of the deduplication logic. But you will get a much larger did file, and the non-stability from recursive type still exists.
  • You can write did file manually as the source of truth, similar to protobuf. And use Candid to generate both Rust and C# bindings. Our current Rust bindings generation is still work in progress. You need to manually modify the generated data to add derived traits for example. Hopefully, this can get better soon.
  • Candid is structural typing, so names don’t really matter. Maybe there is a way for the C# binding to normalize the did file first, so that the name changes doesn’t affect the binding generation.
4

Hah, yeah I had used it to understand the candid code, and it was doing a pretty good job… until then I guess.

I want to fix this, and it’s going to be a couple of days of work, I’m just not sure I should

  1. Fork Candid and make it do my bidding.
  • PRO - all of the types (Errors, API calls) have exactly the same name as in the Rust definitions.
  • CON - have to keep it constantly updated
  • CON - in an environment where you’ve got multiple re-exports of candid with different versions you can introduce hard to find bugs.
  1. Write a C# class generator
  • PRO - finite amount of work
  • CON - there’s already a team working on this (but it’s using the .did file)
  • CON - things like Result, Error, LoadQuery are still using the CandidType macro names.
  • CON - the front end still has to send types with incorrect names to the backend. If we add new types it changes. If we’re sending RaritySet to create_item_set, and then we add a AbilitySet, the RaritySet may no longer exist.
  1. Add a unique field to every record and entity
  • CON - doesn’t work with newtypes
  • CON - not even going there

  1. Change our ORM so that the structure and names are separated. I’m not even sure this is possible and it’s a complete ground-up rewrite.

  2. Keep the .did files updated manually

  • CON - this would be about 5 hours of work a day with the rate things are changing
4 Likes