Variants and records are exactly dual to each other with respect to backwards-compatible extensibility. Consider two canister functions:
f : () -> (T)
g : (T) -> ()
If T contains a record, then adding a field to it is fine for f, because an existing caller will just ignore the new field. It is not fine for g, because an existing caller will fail to pass the extra field. Candid hence only allows adding a new field there when it is optional, and transforms a missing field to a field whose value is null.
If T contains a variant, then adding a case to it is fine for g, because an existing caller will simply never supply the new case. It is not fine for f, because an existing caller that switches on the result will have no handler for the new case. Candid hence only allows adding a case there when the entire variant is optional, and transforms an unknown case into null.
In other words, if you want a variant to be extensible, you need to make it an option whenever it is used in a return position:
f : () -> (opt variant {a; b})
This way, all clients are forced to implement a fallback for null, and that will automatically handle any new case c they do not (yet) understand.
It does appear that extending TransferError would be a breaking change for icrc1_transfer because it uses Err : TransferError instead of Err : opt TransferError
I think it’s unfortunate that to support extensibility we need to do something unintuitive. More importantly, I don’t like that this introduces otherwise invalid states.
i.e. Err : opt TransferError would allow the error to be omitted.
I think it’s unfortunate that to support extensibility we need to do something unintuitive.
I don’t disagree, but unfortunately, there is no obviously better alternative. One could imagine a way to support variant extension by subcasing, i.e., new cases can only be introduced as subcases of previously existing ones (so that a preexisting consumer sees it as the preexisting case). But what if there is no suitable case to refine? In general, you’d need to plan ahead by introducing a fallback default supercase upfront that’s initially unused, but then that’s no different from wrapping an opt around.
More importantly, I don’t like that this introduces otherwise invalid states.
Well, the same is true when adding record fields which are forced to be optional. That’s the price for backwards-compatible extensibility.
Sure, you could do that (I’d call it 60s style ), but that’s much worse wrt what @paulyoung is concerned about, because it introduces not just one but an infinite number of invalid states. Moreover, it will enable zero help from languages regarding case coverage.
Technical specifications are first and foremost the pursuit of API invariance. If this is happening in the official base functionality and the topic you are focusing on is happening more in the business layer, please give the business layer back to the developers themselves. The more you do, the more you ostensibly make it easier for developers, but actually hinder many innovative possibilities and deprive developers of choice.
If it is an API for an end product, it is appropriate. But with a specification like ICRC1, it is difficult to extend using variant in the interface.
This one is scalable. As for use cases, developers will form a consensus and publishers can make " conventionality " to guide them. “icrc1:symbol”, “icrc1:name”, “icrc1:decimals”, They are good " conventions ".
Yes, it solves the problem. You know, what I said above is a matter of general applicability, because the demonstration of examples leads developers into a development paradigm.
The point I’m trying to make is that it’s not a good habit to expose variant to the public API.
I think having a typed interface through candid is an innovation and a great advancement. We should not give it up. Personally, I am fine with taking the precautions mentioned above around the proper usage of opt to deal with upgradeability. There are so many advantages in terms of safety that compiler checks bring here. dfx even warns you if you upgrade a canister and the new interface is breaking the old interface. That isn’t possible if we circumvent the check with a Nat error code and a Text value.
But I do acknowledge that there sometimes can be differences between canister to canister calls and agent calls from outside. You could provide an additional API specifically for agents who do not care about candid and do not care about compile-time type checking on their end.
I think it’s important to distinguish different use cases.
For errors, you don’t usually care as much about knowing the complete list of possible errors, because in most cases they are just diagnostics that you do not use programmatically but just forward elsewhere, e.g, to the user.
Other variants are used much more programmatically, meaning that client code typically needs to perform an exhaustive switch on the possible values. And they are often fixed. Say, day-of-week, order, or bool (which is nothing but a built-in 2-case variant). In those cases, proper variants are preferable, because the types then tell exactly which cases exist and need to be handled, and a modern language can check that.
If you want, variants essentially are the “typeful” alternative to an unchecked and uncheckable nat/text representation. And like with all choices in the typed-vs-untyped space, that involves (a) a tradeoff, and (b) a learning curve to get most out of the type system.
That issue basically enables the opt variant type to upgrade with new tags, as we discussed here. But to implement this with serde, we have to use unsafe features. We are still trying to decide whether that solution is production-ready or not.
The section on actor subtyping claims that " Actor V2 is technically NOT a subtype of actor V1 (like in the case of object subtyping)". That’s incorrect. In Motoko, subtyping on actors follows exactly the same rules as on objects.
In the section on function subtyping there is a note saying: “There is an exception for variant return types in new versions of Candid”. This note (which is repeated twice) is misleading in several ways. Candid has more permissive subtyping rules on variants than Motoko does, but they apply everywhere; there is no exception specific to function types. It also has similarly more relaxed rules for object types. (Also, technically, these more liberal rules always existed in Candid, but for variants they were broken for a while due to other changes.)
Specifically, in Candid, object supertypes can have more fields as well, as long as those are optional. Dually, variant supertypes can have fewer fields as well, as long as the whole variant is optional. That greatly extends the set of changes that are legal and safe for upgrades. Since Candid is what matters fur upgradability, maybe this section needs to distinguish between Motoko and Candid, and explain the difference. Subtyping in Motoko has a different role (mainly, supporting OO-style patterns), technically, it is neither necessary nor sufficient for upgradability.
On the second point: I will remove the NOTE and think about making a clear distinction for Motoko and Candid subtyping after experimenting a bit with both.
Aah ok! I’m learning as I go! I will have to cover this separately.
Thanks a lot! (and sorry for spamming this thread )
It actually is, though easy to miss: actor is one of the possible sorts that occur in the subtyping rule for object types. Other sorts are object and module; all of them are different forms of object type.
), but that’s much worse wrt what 
)