Thanks for the ping @diegop .
@jzxchiang : after increasing to 8GiB, as expected, we uncovered some performance bottlenecks that the team is currently working on addressing. One of the key bottlenecks that we discovered is discussed in more detail in Quick Fix for Smart Contract Memory Bottleneck | E007 - YouTube. The work here is not completely done yet.
In general, the problem is that when canisters produce too large a heap delta, they put a lot of load on the node’s memory system. Previously this was slowing down the entire subnet. @abk then worked on an optimisation where we track the heap delta per canister. This allowed us to mitigate the problem so that we only slow down the offending canister. Next @ulan and Adam are working on a file backed PageMap allocator so that we can further reduce the load on the memory system.
Note that in tandem they are also working on the canister sandboxing feature so progress here is a bit slow. Once, their PageMap allocator work is done, barring any other critical performance bottlenecks are discovered, we should be in a good place to further increase the canister capacity.