LAMENT: A tale of constant struggle of what it's like trying to scale on ICP

Hi @saikatdas0790!

I understand the challenges you’re facing. I fully agree that there is a lot of progress to be made in terms of balancing load on ICP, handling busy subnets more gracefully, perhaps allowing easier canister migration between subnets, and more. As I mentioned in another thread, we are very much looking into this and hoping to make significant improvements soon. The compute load on ICP has skyrocketed lately, and the canister count has doubled in the past year (from ~300k to 600k), so naturally some growing pains show up and DFINITY will prioritize addressing them. First improvements are coming in the replica version that DFINITY will propose today.

That being said, the two main issues you bring up (subnets currently not handling huge numbers of canisters well, and the cycle cost of a fleet of canisters) boil down to the architecture that Yral chose to follow, where it uses a new canister for every user. DFINITY R&D has repeatedly warned Yral that this is not a scalable architecture, urged the Yral team to revisit this choice, and offered help to make that change. Yral chose to stick to the one-canister-per-user approach, and now run into scalability challenges and the per-canister costs. My advice remains the same: don’t use a new canister per user, for projects that aim to onboard a big amount of users.

fwiw this is not true, i wrote exactly what you see now, nothing is “redacted”, what you’re seeing is another user’s comment modifying a quote of my message.

21 Likes