Technical Working Group: Scalability & Performance

Hello everyone,

we’d like to announce a new technical working group:

Topic: Scalability & Performance
DFINITY lead: @johan
Coordinator: @domwoe
Community Coordinator: @icme
Discord: Scalability
Meeting Notes: Link
Cadence: Monthly on the third Thursday of every month at 5:30pm CET
First meeting: July 21, 2022 at 5:30pm CET
The event is part of the regular calendar for technical working groups.

The Internet Computer provides a unique blockchain platform for decentralized applications. The actor model allows parallel execution of code in a given subnet and the IC can scale horizontally by adding more subnets. Already today, this allows the building of decentralized applications using smart contracts that aren’t possible on other blockchains. However, these are still early days for the Internet Computer, and design patterns for building decentralized applications on a global scale are only starting to emerge. The Internet Computer itself is also continuously maturing with performance improvements, new features, and better tooling. The goal of this working group is to foster a regular exchange between R&D at DFINITY and application developers with the following focus:

  • Updates on the relevant R&D roadmap and presentation of ideas
  • Collecting and discussing pain points of application developers
  • Identification and sharing of best practices and design patterns for building applications on the IC

Thanks a lot to @icme who provided the spark to initiate this working group. We hope that there’s interest in the community to participate in this working group. If you think we’re missing something important in the scope or you have ideas already for special topics you want to discuss or present please reply in this thread or the Discord channel.

For the first session of this new working group we’d like to start with an AMA on scalability to get a feeling about the questions and topics that you’re pondering with. Please give us a heads up by filling out the following Typeform.

14 Likes

Want to bring this to your attention @lastmjs @paulyoung @Maxfinity @senior.joinu @spnrapp @emilbroman @hassen.saidi @GLdev @C-B-Elite @skilesare

Probably forgot quite a few (and can’t tag more than 10 people in a post as it turns out) so please tag others that might be interested or can contribute :slight_smile:

5 Likes

@quint @Gabriel @jzxchiang

Also would be great to get some people from the DFINITY Asia community & AstroX @zire @neeboo

Fee free to tag some people from the gaming community to understand their requirements going forward, as well as @rckprtr @hpeebles to understand their solutions & needs given that they are the top 2 apps on the IC (DSCVR/OpenChat)

5 Likes

First meeting is happening tomorrow. Please fill out the Typeform if you haven’t already.

2 Likes

Hey Distrikt here :vulcan_salute:
I’m definitely interested and I will gladly join !
@rckprtr @hpeebles @Maxfinity will you join too ?

3 Likes

I’ll be joining! I may be a few minutes late though… (but hopefully I won’t be :crossed_fingers:)

1 Like

Thank you all for participating in the first session!

Here are the notes and the recording.

2 Likes

Unfortunately, that recording link has expired.

1 Like

Johan is on vacation this week. Hence, we need to postpone the discussion of the roadmap wrt. scalability and performance of the IC. In the survey, the most requested topic was “safe canister upgrades” and two 3/4 of the survey participants are using Rust for canister development.

One issue that devs encounter in canister upgrades is that the instruction limit gets reached while serializing or deserializing state to/from stable memory. One way to circumvent this is to use stable memory for storing state directly. Additionally, this allows also to use up to 8 GB for canister state currently.

There have been recent developments by Dfinity and the community to increase the usability of stable memory in Rust, and we’ll have two presentations on these:

  1. @ielashi will present StableBTreeMap which was developed for the Bitcoin Integration
  2. @senior.joinu will present ic-stable-memory

I’m looking forward to these presentations and the discussions wrt. the usage of stable memory on the IC.

Please join the session this Thursday (8/18) at 5:30 pm CEST.

5 Likes

Hey everyone,

I’ve created this Living Feedback Document for the members of the Scalability and Performance working group and those who are interested.

It contains viewable resource links and content with respect to what we’ve covered in previous meetings, and members who show up to the meeting will be permitted to comment/suggest questions or feedback to discuss in future meetings.

If you’d like to suggest a topic to be discussed in future meetings, you can do so by:

  • Commenting in this topic or the #scalability channel in the Developer discord
  • Pinging me to add your topic/feedback to the Living Feedback Document
  • Attending the meeting :slight_smile:

See you at this Thursdays meeting for some exciting presentations! (See @domwoe’s post above)

3 Likes

Notes and recording for the second session (2022/08/18)

3 Likes

Thanks for sharing the notes.

I noticed that the notes mention that Johann from DFINITY will soon work on Wasm multiple memories support. Has that been standardized by Wasm yet? (I know it’s available in wasmtime with a command-line flag.)

If that is shipped, won’t that deprecate most of the libraries that facilitate stable memory access, including ExperimentalStableMemory in Motoko? What is the ETA on this? Thanks.

2 Likes

Hey thanks for sharing !
I was late for the meeting so I would like to watch what I missed but I get this message when I try to access the recording : You cannot view this recording. No permission.

Would you happen to know why @domwoe ?

1 Like

Oh, thanks for the heads up. The permissions were indeed mixed up. Should be working now.

2 Likes

Hi @jzxchiang!

Multiple memories is still in Phase 3, i.e., not fully standardised. But it is already implemented in several engines, including Wasmtime, which the IC uses. Moreover, according to Andreas Rossberg, the proposal’s champion, and former principal engineer at DFINITY, the proposal has been unchanged for a long time and is not expected to change. Thus, we are rather comfortable experimenting with using it on the IC. New libraries for Rust and Motoko will be needed to take advantage of multiple memories. We haven’t started working on it in earnest, so it is too early to give an ETA. We’ll continuously update the Scalability & Performance WG with our progress. Best, Johan.

5 Likes

We moved this week’s (9/15) Scalability & Performance WG to next week 9/22 same time as usual. If you subscribed to the Technical Working Groups calendar the event should’ve been updated.

1 Like

Didn’t know Andreas left Dfinity. Was there a specific reason for this?

2 Likes

Andreas Rossberg Still working at Dfinity?

1 Like

No, Andreas left Dfinity a few months ago (maybe the end of June?) but he has been active here in the forum since then.

1 Like

This week is our next Scalability & Performance WG with the following agenda:

  • @johan will give a quick update on the plan to increase stable memory on the IC
  • @RMCS (Catalyze) will talk about their multi-canister scaling approach and their open issues

I’m looking forward to the discussions!

Link to Recording

We know that there’s a lot of interest in discussing Inter Canister Query Calls (ICQCs) as well, and the plan is to have a discussion on this topic led by @ulan and @dsarlis in the next WG call in October.

5 Likes