Nice post! Indeed, regulation will come one way or another. The best we can do is to prepare for it, exploring a method to do e.g. KYC the Web3 way while meeting regulatory requirements.
One possible solution we are building using Legit ID is to combine W3C DID/VC and ZKP to prove identity attributes while preserving user privacy, e.g. proving one is a qualified adult investor from EU without disclosing one’s PII.
We will explore how to integrate this function into ICP using the II in early 2024.
Motoko wishlist, some practical things that can save time:
Errors “expression of type … cannot produce expected type …” could pinpoint the problem. In some cases, they just output both objects and let you wonder what’s wrong.
public functions being able to return objects that have ‘var’ inside - some kind of auto-conversion.
Luc’s stable memory. Luc ftw
Other:
NNS Dapp - able to sign IC calls, perhaps with the new Identity WG standards.
NNS & SNS - sign 3rd party messages with neurons, so they can send signals not only to proposals but to anyone who wants their signal.
I had been in contact with someone who was using this heavily, I believe there were still a number of limitations, though I’m not sure on the current status.
3 - Don’t really understand the benefit of this…sounds cool though.
4 - Is this basically reflection? If I’ll be able to do typeof(x) == MyType then I’m all for it.
5 - Is there something about the current certification library that is lacking? Is this basically optimization on @nomeata 's library? Or something more aligned with v2 certification?
6 - I have some concerns about pushing too much into base. This could be seen as…not necessarily coopting work, but we do want to support 3rd party library development. If everything has to go into base it could dissuade that small part of ego that devs sometimes have and/or put out an external facing view that looks like ‘just wait for dfinity to do it’. I think that can be overcome! Perhaps with reward bounties for things that make it into base…we could even do some fun NFT thing for devs that get code into base.
7 - Why not IC based debugging (other than it is harder). This is mostly targeted at base library devs right? The second you start doing async you’d lose the ability to debug?
8 - I don’t understand this one at all. Would these be type annotations to my types? (so could I use #ok in my motoko class but it looks like #Ok to external programs?(And why didn’t the motoko team sync the Result object with how System Canisters were doing #Ok?
9 - A nice to have, but any serious work likely needs an IDE of some kind? Maybe? Probably? Can’t someone like github or VSCode Online draw on more resources to do this kind of online editing?
10 - I’d encourage Motoko devs to write tests in Motoko and not ICRepl. It makes you a better motoko dev!
11 - Is this just like type MyType = “mo:myclass/lib/MyType”? What would this look like?
12 - I’d actually love to get a broader opinion on this thought: I know it is best practice to split things up, but as I begin to rely more and more on bootstrapping things with AI, it is actually really beneficial to have as much context in One file as possible. I’ve never started keeping my types together some times. Don’t throw rocks!
13 - I thought this was already in? What would be the “more comprehensive” part?
14 - Now you’ve peaked my interest. Is this a bitfinity EVM type thing? In what way would solidity be able to call motoko? Or is this a bootstrapping mechanism that could take a solidity contract and try to shell out a motoko contract from it?
15 - What makes something ‘unreachable’? My use of memory in the past has been write and forget and only read when the program looks it up. Maybe this is related to the more advanced stable evolution…in that case, yes I want GC
17 - What are you doing here reading this post…go work on that immediately…
18 - I’m not sure what extra this gets you…maybe it is wonderful, but I’d encourage all motoko devs to build their integration tests in motoko to get better at motoko and to better understand how the IC canister to canister interaction works. I’m sure there are use cases and I’d love to hear them! Maybe if you have to do main net testing?
19 - This would be awesome especially if we could download our canister state.
20 - I’m not sure I understand this one? Would this be like secure enclave access? or encryption at rest? Something like that?
21 - I’d prefer to keep this at ingress and have things handled internally as candid…maybe the boundary node could take a translation file or something? I think newer devs would love to just send json to their canisters. I do think some way to bootstrap from a swagger file to a motoko template would be very cool.
23 - Huge for adoption!!!
I probably need to resubmit my responses after better understanding these.
Here is some quick clarification for a few of the survey items:
13 - The concrete goal here is to add a file system API to the WASI runtime.
14 - I was thinking that we could create a command-line tool which generates Solidity smart contract bindings as a Motoko module (converting ABI to Candid and generating code for calling the EVM RPC canister). For example, if you wanted to call a contract in Motoko, you could run a command to generate the bindings and then call the contract from Motoko as though it’s any regular async function. This would also give us momentum for doing something similar the other way around (i.e. calling IC canisters from Solidity).
18 - If you’ve come across the ic-state-machine-tests crate for the Rust canister ecosystem (example usage), this would be an equivalent for Motoko. It would essentially be a way to deploy canisters, mock HTTP outcalls, etc. in a simulated replica with all tests written purely in Motoko. We’re currently gauging interest for this, since it’s more powerful but has a lot of overlap with ic-repl and other existing integration testing tools.
Hope it’s okay to co-opt this thread a bit @lastmjs
I’d like to also share the SDK Roadmap Community Survey and get your opinions on what the SDK team should focus on this year. We want to know what you’d like to see in your tooling! You can find the survey here: SDK 2024 Roadmap Community Survey
3 - Don’t really understand the benefit of this…sounds cool though. Graph Copy
Related to the orthogonal persistence, offer a mechanism for flexibly changing the memory layout in the future. For such rare cases, implement a scalable heap serialization that also supports 64-bit representations.
This is basically fixing the long standing issue with serialization and deserialization of stable variables that:
doesn’t handle value graphs properly, exploding them to trees and duplicating large value (text/blobs).
can’t deal with large amounts of data
can stack overflow on moderately deep data-structures (e.g. linked lists).
These limitations are artifacts of using an inappropriate Candid-derived serialization format that can and should be avoided by using better serialization techniques (when needed). Candid is great as a wire format for small messages, but not great as a data storage format for large volumes of data.
4). Is this basically reflection? If I’ll be able to do typeof(x) == MyType then I’m all for it.
Although it could be used for this, we really don’t want to go there: reflection lets you break all sorts of abstraction properties and is a bad idea in general (even though it can be convenient). Really, it’s main application is to enable the graph-copy based stabilization/destabilization algorithm to accommodate (in-memory) data representation changes when necessary. But it also helps data recovery and debugging tools by letting them look at the representation of a value to determine its type rather than consulting out-of-band type information (which we may not even have).
Why not IC based debugging (other than it is harder). This is mostly targeted at base library devs right? The second you start doing async you’d lose the ability to debug?
It’s just because it’s easier than full IC debugging, tbh. Don’t let the best be the enemy of the good and all that.
Is this just like type MyType = “mo:myclass/lib/MyType”? What would this look like?
import { type List; pop } = "mo:base/List";
i.e. allow destructuring/pattern-matching imports to bind type components as well as values.
15 - What makes something ‘unreachable’? My use of memory in the past has been write and forget and only read when the program looks it up.
It’s the job of the GC to recover memory that is no longer referenced (reachable) from the live data of your program. The GC does this for ordinary Motoko values but not the special region objects. Currently, if you drop a reference to a region object the stable-memory dedicated to that the region will never be reclaimed nor reused but could (and really should) be in future.
I love that you wrote all these out Jordan, and I believe they touched on many of the issues/roadblocks that developers on ICP experience.
I’m going to focus on a feature you mentioned that has been a massive blocker for us, and is a major limitation for any serious dapp on ICP.
I’m referring to Full Inter-Canister Query Calls.
Or another way to phrase it would be “Being able to do a Composite Query during an Update Call”
To build a scalable database on ICP, there is a big limitation in that inter-canister query calls take as long as update calls. Of course, this has improved with composite queries, but they only work if the originating call is a query call. If you’re trying to query another canister during an update call, then the query call is treated like an update call, and adds 2-3 seconds.
In order to build massively scalable databases on ICP, you need to scale horizontally by spreading data in thousands of canisters. However, the blocker I mentioned above leads to it taking 4-6 seconds to make modifications to data in databases. An example flow is a user calls a dapp canister, which queries a database canister. Then the dapp canister does modifications to the data, then returns the response back to the user. This takes 4-6 seconds minimum because it’s the equivalent of two update calls. If the dapp canister could make a composite query to the database canister during the update call, and the composite query takes 100-200ms, this would be a huge milestone. It would allow for complex and scalable databases to be built without the dapp being super slow.
It’s a major blocker for us and any serious dapp on ICP. I hope it gets prioritized. We want to show how fast ICP can be!
Also, another feature we would love is being able to download canister state as a backup file and restore it to the canister in case data becomes corrupted during an upgrade. Would make upgrades way less stressful.
That is not exactly accurate. A canister-to-canister call can be “instant” (i.e. it may complete in the same round) if the callee is on the same subnet; or it will take 5+ seconds if the callee is on a different subnet. Of course it all depends on subnet load (no canister is guaranteed to get scheduled every single round unless it pays for 100 CPU allocation; and there isn’t all that much CPU to allocate).
In your particular case, I’m not sure whether the 4-6 seconds you’re observing are simply ingress latency plus one second here and there due to subnet load, preventing everything from being scheduled in one round (and your canisters are both on the same subnet); or it’s because you’re actually making cross-subnet calls and that’s just how long a cross-subnet call takes.
Again, that is not exactly accurate. On the same subnet, it is likely more (or just as) efficient to make a replicated query call as it would be for every replica to independently execute a non-replicated query. And cross-subnet, it would still not be advisable to implement it the way you describe it, because would basically block replicated execution (and stall the caller’s subnet) for however long it takes to execute the query.
But it is an interesting idea for reducing cross-subnet latency nonetheless. Currently, as said above, a XNet call has a latency of 5-6 seconds (essentially 2x ingress latency, because first the request then the response have to be included into into a block, block needs to be finalized; message executed; resulting state certified). It would be technically possible to introduce a “read-only XNet call” (or “XNet query”) that would work more or less like an HTTP outcall, except as opposed to requiring every replica on the caller subnet making the call and then achieving consensus, it could rely on a single call and either certified variables or a certified query (where f+1 replicas are queried and each of them certifies the response). Something like this may theoretically cut latency in half.
It’s not something that is likely to get picked up any time soon, but it’s definitely worth keeping in mind. Thanks a lot for the suggestion.
(other than it is harder). This is mostly targeted at base library devs right? The second you start doing async you’d lose the ability to debug?