The Languages Team is currently working on major changes to the Motoko base library. Our goal is to improve the consistency and readability of Motoko canister code for both humans and AI.
We want to involve everyone in the community as much as possible in this process. Below is a link to the GitHub repository where you can view the new API, comment on proposals, or even submit your own improvements:
The best place to provide feedback is the Discussions page, where you can vote and comment on changes to the base library. Your opinions are highly valuable, especially right now while we are most flexible with the API design.
If you have any questions about this base library initiative, please feel free to reply here or open a new discussion on the GitHub repository.
Here is a quick overview of the improvements which we have talked about internally:
Defining a uniform API for all data structures that can be used in stable memory.
Providing mutable and immutable variations of each data structure.
Simplifying usage of the Random module and including built-in pseudorandom algorithms.
Fixing inconsistencies in the names of functions (e.g. replacing “ArrayMut” with “VarArray”).
Replacing the Iter.range() function with Nat.range(), Int.range(), etc. with an exclusive upper bound.
Since this will involve many small breaking changes, the plan is to do everything in one major update. We will provide a migration guide for the new version of the base library, and the original version will still be available for a gradual transition.
For everyone reading this, we want to make sure that these changes will benefit you, so definitely let us know if you have any feedback!
Why do we need breaking changes? Can’t the existing methods stay there?
What are the advantages of rewriting things to stable memory with the new orthogonal persistence memory model upcoming? (Because it is more expensive I typically avoid it if possible). I understand it has become the goto in rust but that seems like a rust problem.
Data structures that are viable in production-real-world applications are unlikely to be using the data structures in base anyway. They typically need more nuance than they’ve typically provided. Perhaps we get the objects we need, but there are already a number of community produced libraries that DFINITY could audit and enhance that might be a better starting point.
The overall goal of these changes is to reduce the learning curve for new developers and improve the ability for AI language models to generate high-quality Motoko code.
Why do we need breaking changes? Can’t the existing methods stay there?
Essentially, we want to rework the function names from the ground up so that the base library is more intuitive when coming from mainstream programming languages such as JavaScript and Python.
What are the advantages of rewriting things to stable memory with the new orthogonal persistence memory model upcoming?
Since the base library currently has a mix of different data structure APIs (mutable / immutable, class-based / module-based, etc.), we want to establish a consistent pattern so that data structures are more or less interchangeable with each other for common use cases. We are also planning ahead for the stable classes proposal, although it’s worth mentioning that we don’t have an official timeline for this feature as of writing this post.
Data structures that are viable in production-real-world applications are unlikely to be using the data structures in base anyway.
Right; agreed. The opinion within the Languages team is that we want the base library to focus on simple, readable “building-block” data structures and defer to third-party packages for specialized or optimized implementations with usability tradeoffs.
Just to clarify, we don’t intend to implement new low-level data structures in stable memory directly (e.g. the equivalent of Rust’s “stable structures”). Instead, we just want to ensure that the Motoko native heap data structures are stable where possible and can easily be stored in stable variables without requiring awkward and dangerous pre-upgrade hooks.
Ok…so stable but not stable storage(this has always been awkward). I typically refer to these as sharable, because they can be passed by actors, but I don’t think that is quite right either since some functions currently can be passed but aren’t “stable”.
I’d love someone from the team to take a look at the class plus component I just put out. Maybe there is something there that could be further abstracted into the language(and maybe I made some bad assumptions), but I think that it solves a bunch of this from a software viewpoint. When paired with the migration pattern pre and post upgrade goes away. Devs can already stress less about what is stable or not if the lib devs implement it correctly.
Wow, this looks awesome and I can’t wait for it and it’s obviously needed, but it invokes a sense of existential dread as I realize that days and days of my past life spent in development time are going to be abstracted away into the language.
This is great in the future. But how do other developers deal with this psychologically? I guess it’s healthy for any new language, but it “feels” bad.
Thank you for sharing your perspective. I understand where you’re coming from with this and want to make sure that we keep the value of the immense amount of work that’s built on the current version of the language. My hope is that stable classes would give more expressiveness to Motoko and complement rather than replace the existing design patterns.
I’m always excited when it comes to introducing changes, especially ones that bring consistency and usability improvements. I really hope the proposed updates introduce more uniformity with module-based data structures and pave the way for shareable data structures. Making it easier to serialize types into stable memory, especially custom ones, and simplifying conversions between types would be a welcome change! Stable classes would be awesome.
On my personal wishlist, I’d love to see a Cycles.measure(func : () -> ()) method to measure the cycles cost of specific operations similar to the countInstructions available in the ExperimentalInternetComputer library. That kind of feature could provide real insights into performance and help developers optimize their code. Another idea would be something like a fabricate transaction feature, similar to dfx ledger fabricate-cycles. Expanding this concept to support fabricating the receipt or sending of ICP (e.g., dfx ledger fabricate-icp) could be great for local testing. Though I know this might fall more under the SDK team than the Motoko team, it’s something I wanted to throw out there.
I also think it should be much easier to interact with management canisters like cycles minting and the IC ledger. We shouldn’t have to explicitly tell Motoko what the management canister, the ICP ledger, or the cycles management is—this is Motoko’s platform, it should know by design! It could also abstract away the underlying inter-canister calls to making it easier for developers to use. For example, we could have methods like IC.create_canister(settings : Args) -> CanisterId, Ledger.icrc1_transfer(x : TransferArgs) -> TransferResult, or CMC.notify_top_up(y : NotifyTopUpArg) -> NotifyTopUpResult.
These kinds of abstractions would make development far more intuitive and would be a worthy addition to the base library. I’m excited to see what other changes can be made. How do you plan to manage migration? How long will the original version of the base library remain available? Will there be support for older canisters that rely on the existing base library methods, or will developers need to refactor and redeploy their canisters? Will the base library adopt versioning to allow canisters to specify which version of the library they depend on?
Thanks for the suggestions! I like the idea of adding wrapper functions for the management canister, ledger, etc. and will run this past the rest of the team.
How do you plan to manage migration?
The current plan is to nest the original base library within a subdirectory, e.g. mo:base/deprecated/Array. We will also create a migration guide which gives an overview of how to transition to the new modules and functions where relevant.
Will there be support for older canisters that rely on the existing base library methods, or will developers need to refactor and redeploy their canisters?
It will be possible to use the previous base library version during a transitional phase, and afterwards you could downgrade the compiler version (currently 0.13.5).
Will the base library adopt versioning to allow canisters to specify which version of the library they depend on?
We will continue to ship the base library separately from the compiler as a Mops package for flexibility with versioning.
