Candid support for generics

Hi, we’re running into a few issues with Dragginz and wanted the community to do a quick sanity check.

We’re building a feature-rich ORM with ACL rules, indexes, tree structures, validation rules, etc. However we can’t get around the fact that even though the library code relies heavily on generics, the front end can’t.

It’s weighing in at about 20,000 lines of motoko code and 148 entities (80 of which have TrieMaps to store data). By the time we’re at MVP it’s probably going to be closer to 300 entities and 40k lines of code. A lot of this code is generated by Go so it’s really easy to make structural changes. (sorry, @Gabriel)

It’s not a huge issue really. Compile time is about 2 minutes 15 seconds, which is annoying. If I get rid of all the Main.mo shared functions then it’s 36 seconds.

Wondered if anybody had any input!

1 Like

as an aside, if anybody’s curious here’s one of the 80 database classes with indexing

and here’s one that has a data transformer that takes a parentID and turns it into a nested set implementation

2 Likes

Related: Feature Idea: Generic data · Issue #245 · dfinity/candid · GitHub

Wow, above my pay grade… but implementing this would be great for our project.

I think you can remove a lot of code by using a variant where you might imagine you’d want a generic type argument.

Does this approach improve things at all?

type EntityType = {
  …
  #zone;
};
type Entity = {
  …
  #zone : E.Zone;
};
public shared query({ caller }) func loadMany(entity : EntityType, ids : [Nat]) : async (Result, [Entity]) {
  switch entityType {
    …
    case (#zone) {
      #zone Query<R.Zone, M.Zone>(caller, db.zone, "root.admin.load").
      loadMany(ids);
    }
  };

Actually, since [Entity] is being returned you’d need to use map to wrap everything in the array with #zone

…or have a version of Entity called Entities where #zone : [E.Zone] and return Entities instead.

This could work actually, I have to look into it because the rest of the code is pretty convoluted… but if each front end function was a massive switch then it may be a lot faster.

I think a lot of the compile time comes from the public actor functions as Main.mo is about 20% of the overall codebase but 80% of the compile time.

Will give it a go and let you know! Thanks

I think this is actually too much of a recode right now.

Maybe it’s worth it to get the compile time down, but the issue is we have index lookup functions that also pass generic fields. Doesn’t seem a clear path to having a cleaner code base long term.

It is what it is I guess, shouldn’t really complain that much!

I’m a bit confused: Is this thread about bad Motoko compiler performance (which could be looked into, I expect a lot of low hanging fruit there) or missing language features in Candid?

Well both really, but the latter is more important for us.

It was a case of “we can’t do this so let’s generate a load of code, and we think that extra code is slowing down our compile time.”

Turns out the compile time was caused by a public TrieMap on our database store class. Once we made it private and restricted access to the methods the compile time halved.

So yes, we’d love candid generics, but only because it simplifies our codebase.