The prosperity of the ic ecosystem depends on the threshold for programming smart contracts. motoko is one of the canister programming languages and lacks some advanced programming features, which further raises the barrier for developers to enter the ic ecosystem.
candid encoding/decoding support in Motoko.
As motoko does not support polymorphism (Parametric polymorphism and async - #5 by paulyoung), this can be solved by encoding and decoding the parameters in motoko.
reflection support in Motoko.
Reflection is necessary when developing some tools or packages.
We are developing some public packages in Motoko to provide development tools, but have encountered the above obstacles, which we hope the Motoko team will improve soon.
Middleware, framework, and toolkit developers, will use the advanced features of a language, as is the case in other languages.
A development language that wants to be widely used needs to lower the developer threshold with the help of rich frameworks and toolkits. And developing these requires advanced features like reflection.
Example.
A tool package needs to wrap an external canister interface.
(1) If you use reflection, you only need to write a few functions to do this. Otherwise, the wrapper needs to be written function by function.
(2) If the external canister interface is upgraded, the tool package written using reflection can adapt well to the changes (Just need to update the interface file); otherwise, the tool package must be upgraded simultaneously.
The candid encoding/decoding feature is also a necessity. Reference example: solidity prioritizes the release of these features, and it opens the assembly interface.
Exposing Candid to/from Blob serialization is not difficult and largely already supported by the compiler, but not exposed. It would be easy to add as a dedicated construct (similar to debug_show) but harder to surface as a library.
I would be very reluctant to add reflection to the language as it breaks all sorts of properties of the language itself and introduces much overhead.
It sounds like you are mostly interested in reflecting on candid interfaces, not all Motoko features. That seems like a more reasonable ask and I can see the applications for that. However, even though canisters currently can expose their textual interfaces using a (hidden) text based query (thatās how icrocks obtains them) weād probably want something with more structure than just text for a programmatic interface.
I can imagine raw shared methods public raw func endpoint(bulk : Blob) and a dynamic analysis of the bulk ingress message. To do it in a nicely type-safe way (pattern-matching fashion), one would need GADTs, I supposeā¦ (to emulate dependent pairs)
Iām thinking the msg in public shared (msg) func .... is already representing the senderās message. We now exposes msg.caller, but we could also expose msg.args for example. This would fit in with inspect_message for example.
Something like:
type WithdrawArgs = { amount: Nat; to_account: AccountId }
public shared (msg) query func inspect_message() {
switch (try_decode<WithdrawArgs>(msg.args)) {
case (#ok(args)) { /* do something with args */ };
case (#err(err)) { throw(err) };
}
}
So try_decode would be the magic system function and it must always take a type argument. What do you think?
This application scenario is very widely used because there are times when you cannot rely on the off-chain for encoding and decoding.
For example the wallet_call function of Cycles Canister. If there is no encoding and decoding in Motoko, it can only implement proxies and cannot handle business logic (do something (using a) and do something (using return))
I agree that this is the solution to support polymorphism without upgrading the interface. Because, if the smart contract changes the interface, it affects composability and immutability.
Multiple use cases show that candid encoding and decoding is necessary.
Iāve felt like in the past that I wish I had some reflection available. Probably mostly when trying to write some generic helper functions that might operate over some limited set of types that I want to do specific things for.
This is all likely based on my bad practices from javascript and some old .net habits. I know they are bad habits, but it is still frustrating to have a language that claims to be general-purpose but doesnāt let me do some things.
I wrote the candy library(which really bastardized the language - candy_library/lib.mo at main Ā· aramakme/candy_library Ā· GitHub ) because I needed a way to store JSON style dynamic data structures. I know that I shouldnāt use them in general, but I also need them in specific circumstances, especially when trying to plan for future extensibility without accidentally blowing away my data store on an upgrade because I added a variant in the wrong order. With the library Iām able to reflect on the data coming in and out of my functions. Maybe a refactoring of the library by a better programmer could make it useful for addressing the situations where you need reflection without needing to change the language.
This is mostly a collection of thoughts while Iām in a place where Iām having a bit of trouble concentrating, but I thought Iād throw the thoughts out there for discussion.
I have found that I need to write custom serialization/deserialization functions for every entity type if I want a generic storage solution for backups. Coming from .net, this extra code is burdensome. If itās all in the name of security and/or the necessary optimization of the IC, then I am very content with the challenge, as itās tiny compared to the challenges faced by the Dfinity team. However, if the inconvenience is a matter of language maturity, then I wait patiently for future versions.
Adding the serialization primitive isnāt hard, but not easily done as a function because it needs to be variadic and have special typing rules (all arguments must be shared) (so better just a new language construct with dedicated typing rules).
Indeed @nomeata proposed and implemented something similar with a trapping deserialization (not opt returning one) here:
What we currently have is a pair of overloaded intrinsics (prims) that are hard (but not impossible) to access unless you are a compiler writer.
Regardless, I donāt think your approach would quite work because Motoko would still attempt to deserialize the blob at type () (for the fuction argument) and fail, before you even enter the function. Maybe if we you typed the arguments as type Any (not ()) but then the message payload would need to be a single argument too, IIRC. Of course we could hack it to work but Iād like to avoid hacks as much as possilbe.
For the particular application of inspect_message, Iāve actually got some strawman proposals sketched here:
I appreciate what you are saying, but, for the record, one can implement wallet_call with out extending Motoko with serialization primitives, all you need is the call_raw functionality that is coming with dfx 0.9.1.
ācandid serialization/deserialization + call_rawā can help programmers solve a lot of problems, it can basically achieve āwhat rust can do, motoko can also doā.
I think it is worth it if it is needed to introduce additional language rules.
I think we may just need a cbor encoder/decoder with call raw. But then again you may need the schema to decide. I donāt know much about cbor. Maybe we need a candid parser and a cbor parser?
