# Candid code generation for nullable types

**URL:** <https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820>\
**Category:** Developers\
**Created:** [August 30, 2021, 7:46pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820 "2021-08-30T19:46:06Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![rem.codes](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/rem.codes/32/40260_2.png) [@rem.codes](https://forum.dfinity.org/u/rem.codes)\
**Post date:** [August 30, 2021, 7:46pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/1 "2021-08-30T19:46:06Z")

</div>

Can somebody explain this to me,

I have a type like so;

```auto
public type MemberWithProfile = {
  member: User.User;
  profile: Profile.Profile;
};

```

in the function i have a empty list;

```auto
var membersWithProfiles: List.List<Response.MemberWithProfile> = List.nil();

```

When iterating over the `Array` with values to fill up the `membersWithProfiles` list;

```auto
var usersList = Trie.toArray<Principal, User.User, ()>(users, func (k, v) {
var profile = ProfileHelper.getProfileByPrincipal(k, profiles);
  switch(profile) {
    case(null) {
      return;
    };
    case(?profile) {
      var memberWithProfile: Response.MemberWithProfile = {
        member = v;
        profile = profile;
    };
    membersWithProfiles := List.push(memberWithProfile, membersWithProfiles) ;
  };
}

```

Everything works fine and i get the candid code as;

```auto
export interface MemberWithProfile { 'member' : User, 'profile' : Profile }
export type MemberWithProfileResponse = { 'ok' : Array<MemberWithProfile> } | { 'err' : Error };

```

But when i change the properties of `MemberWithProfileResponse` to be nullable like so (and change the iterate function accordingly by prefixing the values with a `?`);

```auto
public type MemberWithProfile = {
  member: ?User.User;
  profile: ?Profile.Profile;
};

```

I get this candid code? (the member and profile are both an array)?

```auto
export interface MemberWithProfile {
  'member' : [] | [User],
  'profile' : [] | [Profile],
}
export type MemberWithProfileResponse = { 'ok' : Array<MemberWithProfile> } |
  { 'err' : Error };

```

Is there some explanation to this or is it a bug?

---

<div class="post-metadata">

**Author:** ![rossberg](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/rossberg/32/795_2.png) [@rossberg](https://forum.dfinity.org/u/rossberg)\
**Post date:** [August 30, 2021, 8:09pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/2 "2021-08-30T20:09:51Z")

</div>

A type `?T` in Motoko isn’t a “nullable” type, it is a proper option type. The difference being that it composes properly: for example, the type `?Nat` has values `null`, `?0`, `?1`, and so on. The type `?(?Nat)` on the other hand has `null`, `?null`, `?(?0)`, `?(?1)` and so on. Notably, `null` is different from `?null`, something that “nullable” types confuse. That is relevant e.g. when you have generic abstractions. For example, consider a table class:

```auto
class Table<A> {
  public func lookup(key : Text) : ?A
  ...
};

```

When the table is used to store options itself, i.e., you have a `Table<?Nat>`, then it is quite relevant whether `lookup` returns `null` or `?null`.

This distinction needs to be expressible on the JS side as well, which is why it cannot simply encode options as `null | T`. Instead, it uses an array of 0 or 1 element (though that’s just a choice we made, e.g., `null | [T]` would have worked as well).

---

<div class="post-metadata">

**Author:** ![peterparker](https://avatars.discourse-cdn.com/v4/letter/p/b9bd4f/32.png) [@peterparker](https://forum.dfinity.org/u/peterparker)\
**Post date:** [August 30, 2021, 8:51pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/3 "2021-08-30T20:51:42Z")

</div>

> [@rossberg](#):
>
> This distinction needs to be expressible on the JS side as well, which is why it cannot simply encode options as `null | T` . Instead, it uses an array of 0 or 1 element (though that’s just a choice we made, e.g., `null | [T]` would have worked as well).

Thanks for the explanation! In that case, wouldn’t had it worked out using `undefined` in addition to `null` and the proper value?

```auto
export interface MemberWithProfile {
  member? : null | User;
}

```

P.S.: I’ve got a similar [question / feed](https://forum.dfinity.org/t/incorrect-optional-typescript-declaration-generation/6045/5) about the same topic.

---

<div class="post-metadata">

**Author:** ![rossberg](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/rossberg/32/795_2.png) [@rossberg](https://forum.dfinity.org/u/rossberg)\
**Post date:** [August 31, 2021, 6:59am UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/4 "2021-08-31T06:59:30Z")

</div>

Well, my explanation applies to Candid as well: in Candid, `null ≠ opt null`. The JS binding needs to define a mapping that can faithfully represent this distinction. And it can only define this mapping uniformly, without knowing any context.

Throwing in `undefined` as a secondary nullish value, like JS did, does not solve the basic problem anyway. It still does not compose and just kicks the can down the road. For example, someone might happen to need type `?(?(?T))` somewhere.

(FWIW, these observations apply to union types in general: unions are fundamentally non-compositional and thus a poor basis for composing data structures. This is in contrast to variants, of which options are a special case.)

---

<div class="post-metadata">

**Author:** ![peterparker](https://avatars.discourse-cdn.com/v4/letter/p/b9bd4f/32.png) [@peterparker](https://forum.dfinity.org/u/peterparker)\
**Post date:** [August 31, 2021, 6:30pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/5 "2021-08-31T18:30:14Z")

</div>

I see your point, thanks for the feedback. As I may have spend too much time developing frontend apps, it was kind “weird” and felt uncommon to use the JS binding `[] | [T]` (in term of developer experience). I’ll probably get use to it after a while I guess 😉.

---

<div class="post-metadata">

**Author:** ![Ori](https://avatars.discourse-cdn.com/v4/letter/o/ecae2f/32.png) [@Ori](https://forum.dfinity.org/u/Ori)\
**Post date:** [September 1, 2021, 10:26am UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/6 "2021-09-01T10:26:19Z")

</div>

You might find it useful to create small helper functions to wrap/unwrap these optional values in your frontend Javascript, so you don’t handle this array format everywhere in your code. It’d also then be simple to swap out if changes are made to the js library in the future.

---

<div class="post-metadata">

**Author:** ![rem.codes](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/rem.codes/32/40260_2.png) [@rem.codes](https://forum.dfinity.org/u/rem.codes)\
**Post date:** [September 1, 2021, 12:20pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/7 "2021-09-01T12:20:15Z")

</div>

Yeah this is the way i do it for everything, i just thought that it’s kind of weird to assume that i was getting a `Profile | undefined` and not a `[Profile]`, but the explanation from @rossberg is pretty clear

---

<div class="post-metadata">

**Author:** ![peterparker](https://avatars.discourse-cdn.com/v4/letter/p/b9bd4f/32.png) [@peterparker](https://forum.dfinity.org/u/peterparker)\
**Post date:** [September 1, 2021, 3:44pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/8 "2021-09-01T15:44:39Z")

</div>

> [@Ori](#):
>
> create small helper functions to wrap/unwrap

Of course I did (shared [here](https://forum.dfinity.org/t/incorrect-optional-typescript-declaration-generation/6045/7)). Nevertheless, the less code I wrote because the platform supports it, the better 😉.

---

<div class="post-metadata">

**Author:** ![nomeata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/nomeata/32/733_2.png) [@nomeata](https://forum.dfinity.org/u/nomeata)\
**Post date:** [September 1, 2021, 4:54pm UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/9 "2021-09-01T16:54:53Z")

</div>

Given that these TS bindings are generated from the Candid description, and that that tool sees the type `t` inside the `opt t`, I think it wouldn’t be unreasonable to make different mapping choices for, say, `opt record {…}` and `opt opt …`: as long as the `t` cannot be `undefined`, use `… | undefined`, if that’s the idiomatic way to handle that in TypeScript. I suggest to open an issue in the Candid repo.

Similarly (maybe a bit more contentious) the tool could map records with fields that have an optional type to TS objects where the field may be omitted. Again, if that’s idiomatic; I don’t know TS well enough.

It’s a trade off between idiomatic types and a uniform translation. (And it may require changes to the JS Candid library in case it currently assumes that it can recover the Candid type from just the JS type.)

---

<div class="post-metadata">

**Author:** ![rossberg](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/rossberg/32/795_2.png) [@rossberg](https://forum.dfinity.org/u/rossberg)\
**Post date:** [September 2, 2021, 8:58am UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/10 "2021-09-02T08:58:05Z")

</div>

@nomeata, I’m not sure such a non-compositional conversion would be wise on the binding level. Then possible generic code consuming this would have to make a case distinction somehow – which might be difficult in general due to local loss of information. Likewise, the client would have to interconvert wherever it wanted to compose the value with something else and feed it back into another IC call.

---

<div class="post-metadata">

**Author:** ![nomeata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/nomeata/32/733_2.png) [@nomeata](https://forum.dfinity.org/u/nomeata)\
**Post date:** [September 3, 2021, 10:27am UTC](https://forum.dfinity.org/t/candid-code-generation-for-nullable-types/6820/11 "2021-09-03T10:27:30Z")

</div>

We have such non-composibility all over the place already, e.g. to map records-that-look-like-tuples to real tuples in those host languages that have tuples. So I assume it would work reasonably well.

But yes, it probably works better with binding generators (that produce encoders/decoders for the given interface) than with purely host-type-driven generic converters (like in Motoko or Haskell).