Let's solve these crucial protocol weaknesses

Developers
111

Note: I am posting this not as Diego, the person, but on behalf of DFINITY R&D Team.

Thank you everyone for both posting and reading, a lot of folks have joined in on this thread to ask for clarification, respond to queries, propose ideas, but they mostly have been doing it as individuals. We thought it would be helpful to post what we (the org) is currently thinking.

TLDR:

There are various points brought up. Two obvious themes from reading this thread are:

  1. We (at DFINITY) need to publish our latest thinking on the R&D projects DFINITY is planning so we can explain our thinking, get feedback and align expectations. We include a high-level view of this roadmap in this post.
  2. There are a lot of ICP community members who deeply care about the R&D and wish to make it better. We recognize and appreciate it!

What we saw

Here are some of the points that folks have brought up as issues with the current state of the protocol:

  1. Instruction limits
  2. Memory limits
  3. High latencies
  4. Message size limits
  5. Storage limits
  6. High costs
  7. Rigid network architecture
  8. Centralizing DAO governance

First, I realize it can come off as nitpicky, but because forums are a written medium, it is critical to get every word right. That is why I would argue with some like “high costs” since costs are fairly low relative to Web3… but we can understand if people want to lower them!

Similarly, we can understand if people want to raise the instruction and memory limits. This goes with the course of Computer Science of course: make it faster, bigger, cheaper. No one in the ICP community thinks the IC is perfect “as is”… that is why the NNS exists.

The only question is really about priorities and trade-offs. I will use Jordan (@lastmjs) as an example for a second since he is the OP of this thread: I know he hits the instruction limit often. I know he works around it. The hard part (and one which I think requires many voices for input) is to know how many people this impacts, how urgent it is, what is not happening because of current limits, or how that compares with some unforeseen security issue X, etc… And of course the process is messy and qualitative.

So without getting into every point by point of this large thread, we thought we’d present a relevant part of an updated R&D roadmap that is to be published in a few weeks.

R&D Roadmap

First of all, we would like to point out that the current R&D roadmap on the Internet Computer Web page is somewhat outdated and incomplete and has not been maintained well recently.

Also, its horizon is mid term and it does neither contain our visionary long-term roadmap items, nor many of the community-requested items.

We are currently in the process of defining a new Internet Computer Technical Roadmap that is much more comprehensive and long term than the current roadmap and also much more inclusive of community-requested features. This new roadmap addresses (at least good parts of) the criticism voiced in this thread. We cannot yet provide the whole new roadmap proposal now as it is still work in progress, but we would like to give some examples of items that are on the new roadmap and can help resolve some (many) of the issues raised in this forum topic.

Past Achievements

Before we present the relevant parts of the new roadmap, let us briefly revisit some of the ICP protocol improvements that have been deployed already and also address some of the issues raised in this forum topic.

  • Increase of Stable Memory Limit to 32GiB (the initial value was 8GB)
  • Increase of Stable Memory Limit to 48GiB
  • Increase of Stable Memory Limit to 96GiB
  • Increase of Stable Memory Limit to 400GiB
  • Support of 450GB replicated storage
  • Support of 750GB replicated storage
  • New HTTPS outcalls pricing (with lower cost per call and byte)
  • Deterministic Time Slicing (DTS) (increases the instruction limit by an order of magnitude)
  • Network Scalability: State Sync, Certification, and XNet
  • Optimizations throughout the protocol stack
  • …

These improvements show well that we all are dedicated to improving the protocol and reaching the ultimate goal of a “crypto cloud.” Also, some of those improvements are hard to “see” or “feel” by users / developers, unless they have hit a limit earlier and could benefit from those changes. As another example, substantial efforts have been put into an update of the P2P layer, resulting in a further 20% latency decrease, thereby increasing the block rate. This looks like something small, but is a huge gain in an already performant system where we are slowly approaching inherent limits (e.g., global message propagation time).

The Story Behind the New Technical Roadmap

Let us also give some background on why we have been working on a new roadmap. The current roadmap only contained items that were already in the pipeline of the R&D teams and discussed technically to a certain level of detail and decided that they will be implemented as one of the next items. This roadmap does not contain the long-term visionary items that are not yet well defined in terms of their scope and technically checked to be implementable. It has been clear to leadership in the Foundation that this needs to change and that the public technology roadmap needs to reflect the long-term vision for the Internet Computer Protocol and the needs of our (technical) community. Also, it should not be constrained to items that have been analyzed already in detail, but rather be visionary and also contain items that we (both the community and the Foundation) want, even without a detailed analysis having been done, with the risk that we find out later that a roadmap item may not be realizable for some (technical) reason.

People within DFINITY have listened a lot to the community on various channels and included many of the asks of the community in the new roadmap (this forum thread is an example of such a channel and listening). This is reflected on the one hand by the overall composition of the roadmap that strives to build the crypto cloud of the future, and on the other hand very concrete items from the community that have made it into the roadmap. Once the full roadmap will be published in a few weeks, you will recognize many of your own inputs in one form or the other there. We have listened. We are listening.

A Selection From the New Technical Roadmap

Despite all the achievements so far, we are all well aware that a lot still remains to be done in order to get where we want to be. Besides realizing new features, the new roadmap contains many items that target the improvement of performance of ICP, both in terms of better throughput and scalability as well as reduced latency. Developer experience (DX) is another crucial topic, where we have deployed a feedback board, including voting, for actively listening to the community. We may want to have built-in voting capabilities for the new roadmap to allow people to express their preferences on priorities. That’s an idea for later, though.

Let us next present some relevant items from the new roadmap that will help achieve the goals. We present them in the context of the respective points (in bold) of criticism raised in this forum topic.

1. Instruction limits

  • Increasing instruction limits for query and update calls (incl. increasing the number of rounds DTS can span)

2. Memory limits, storage limits

  • Increase Stable Memory Limit to 400GiB (we’re almost there!)
  • Add support 1TB replicated storage (in progress)
  • Add support 3TB replicated storage
  • Add a Motoko Incremental Garbage Collector (required to manage large heaps, in progress)
  • Add orthogonal persistence for Motoko (to provide the DX for using large memories in Motoko, in progress, demoed already in GR&D)
  • Implement higher-level stable memory libraries (to provide the DX for using large memories in other languages)
  • Deploy a blob storage protocol extension (increasing storage capacity per subnet using erasure coding instead of n-fold replication)
  • Implement fast blob streaming (download)
  • Implement fast blob streaming (upload)

3. Latencies

  • Reduce P2P latency (this immediately reduces latency of consensus and, by implication, of XNet calls which involve consensus twice, once for the request and once for the response)
  • Reduce call latencies (reduce call latencies for the most-used call patterns that negatively affect dapp responsiveness)
  • Improve chain-key ECDSA throughput and latency
  • Implement low-latency threshold ECDSA signing

4. Message size limits

  • Increase block size and throughput
  • Implement chunked upload of large Wasm files in dfx (in progress)
  • Allow for small guaranteed-response messages (this is not about increasing limits, but requiring less reservation of space for the response, therefore allowing for considerably scaling messaging as a whole)
  • Implement libraries for large data transfers (abstract away some of the message limits)

5. Lowering Cost

  • Implement low-replication subnets for LLMs

Note: Cost is very low compared to other networks already and there are initiatives to further lower cost, also as part of other features, e.g., lowering storage cost through blob storage or having low-replication subnets for LLMs, improved throughput for chain-key / t-ECDSA signatures with lower cost etc.

6. Softening limitations of the network architecture

  • Add canister migration
  • Add subnet splitting V2 (simple proposal-driven interface)
  • Add autonomous capacity management
  • Allow for same-subnet placement for canisters
  • Implement messaging model enhancements
  • Implement Improved inter-subnet message routing
  • Implement XNet composite queries

Potentially, lower replication subnets or even single-node subnets (already discussed on the forum elsewhere in the context of “gaming subnets”) would be something that may be of interest for some in the future to give developers more choice than just the two replication factors (app subnets and fiduciary subnets) developers can currently choose from.

7. DAO governance

The voting power distribution has become more decentralized over time; DFINITY’s direct voting power has decreased from just under 40% around launch to just under 20% now. Of course, a large number of NNS participants have currently configured their neurons to follow DFINITY, which adds a large pool of indirect voting contribution. One main requirement for this to change is the availability of reliable alternative options for following. CodeGov is a great initiative that has reliably (and transparently) voted on the proposal topic on replica code upgrades; more such initiatives will be needed before the total (i.e., direct and indirect) voting contribution will significantly change.

8. more relevant items from the community to improve developer experience

  • Implement HTTPS outcalls V2 (various optimizations and new modi operandi for HTTPS outcalls: IPv4 support, single-node outcalls, fire-and-forget outcalls)
  • Implement HTTPS outcalls V3 (HTTPS outcalls for queries)
  • Support REST- & JSON-centric interfaces
  • Standardize canister response codes
  • Build a file system on ICP
  • Build a simple file upload library
  • Enable simple asset creation (e.g., token ledgers created via proposal)
  • Allow for Motoko-written interactive Web UIs running in Wasm
  • Build more Motoko libraries, or support the community in doing so
  • Implement language-interoperability support in Motoko so that Rust (and other) code can be used in Motoko-based canisters

The above is only a small part of the updated Internet Computer Technical Roadmap that is to be published in the coming weeks. As you can see, these items address already important parts of the criticism in this forum topic and it is unfortunate that we have not been able to publish it yet as this could have avoided (some of) the current discussions. Priorities of the items are yet to be determined together with the community. So, clearly, DFINITY agrees to the importance of some of the points raised and has already started taking action.

A Reflection of the Forum Discussion

In most parts, the forum discussion is a productive and technically-driven one, and it also shows that people feel that there is still lots of work to be done for improving the protocol. We fully acknowledge this and would like to make clear that we have the same vision for ICP being the decentralized crypto cloud of the future, but we have not reached all the goals yet. We have started the journey and have made tremendous success on this journey so far, just mentioning some key points below:

  • ICP is a world computer that you can program in Rust, Motoko, TypeScript and Python (many thanks here to Jordan for his great work!), C, C++, anything that compiles to Wasm) and every instruction is executed and every bit of memory is deterministically replicated throughout a globally decentralized network, providing a high degree of data integrity. Thus, a lot has been achieved already in terms of abstracting away the decentralized nature of the system.
  • ICP currently supports 750GB of replicated state per subnet that behaves like random access memory, with almost 40 subnets being deployed.
  • A subnet can execute around 8 billion instructions for update calls per second (assuming currently 4 cores executing update calls on a replica) and a much larger number of query instructions per second.
  • Subnets have been largely abstracted away in the programming model, except for the extra latency incurred in XNet calls compared to calls on the same subnet. Those are caused by 2 rounds of consensus required to secure the request and response to achieve the desired properties of XNet communication being certified through threshold signatures.
  • ICP integrates trustlessly with Web2 and other blockchain ecosystems, using advanced chain-key cryptography. This is a feature not many can claim to support in today’s Web3 landscape.

The journey to realizing the full vision of the world computer is ongoing and requires continued hard work and relentless execution of a dense R&D agenda, for example:

  • Further improving throughput and latency of the different call patterns on ICP.
  • Adding further network management capabilities, such as canister migration, decentralized capacity management etc.
  • Improving the DX.
  • Increasing limits, and where we hit boundaries, offering libraries to abstract away the limits, as far as possible.

We rely on you, our community, to challenge the state of the art of ICP, provide us with inputs of what you want to see built in the future so that we can together realize the vision of the world computer. We acknowledge that publishing the updated roadmap earlier might have made many of the current discussions unnecessary, but there have been some delays in bringing the new roadmap to completion.

I very much liked the depth of a post by Austin (@skilesare). Let me quote one essential and broadly applicable thought from it:

As long as we try to make the IC about replacing existing things we don’t like with something on the IC with THE SAME ARCHITECTURE, we’re doomed to fail.

This is something absolutely crucial to keep in mind when thinking about limitations of ICP – any Web3 system works vastly differently than Web2 and this does need to be considered when implementing Web3 applications and when thinking about performance.

Conclusions

We hope that this post and the preview on the new technical roadmap it presents help establish confidence that the criticism raised by the community is being addressed already by the Foundation and that much of the community input for potential protocol improvements that has been voiced on the forum, Twitter, and other channels has been considered already in the upcoming new technical roadmap. The Foundation is listening, has always been, and benefits a lot from the community inputs.

Quoting from Jordan’s (@lastmjs) original post here:

Wisdom is of course required to weigh these concerns with the many other concerns.

This could not be expressed any better! It is really crucial for everyone to keep in mind that the roadmap items to address the most burning issues can not all be implemented at once because of finite engineering resources, but that this is a longer-term process that will require a continued discourse between the Foundation and the community to find the right priorities and balance between the items and relentless execution by implementing the most valuable improvements to the protocol.