The actual reasoning is the whole story, so here it goes.
This is all because of ic-stable-memory
. In this library there is a stable memory allocator and a bunch of custom collections, like Vec
or BTreeMap
, which store data completely in stable memory.
The goal is to somehow have both:
-
transactional execution - if there is not enough stable memory to complete message execution (we want to allocate two memory blocks during the call, we allocated one, but there is no memory for another), the state of the canister should reset to what it was at the beginning of this execution;
-
horizontal scaling opportunity - when your canister is close to being out of memory, you should somehow be able to react to this situation and run some code (for example, when your canister sees, that there is only 10MBs left in the subnet, it may want to spin up a copy of itself on another subnet).
It turns out, that you can have any one of these easily, but not both. Transactional execution can be achieved by simply trapping when there is no more memory. Horizontal scaling can also be performed at the same exact moment. But you can’t both trap and run some code afterwards.
Initially I was thinking like: Ok, I'll just grow() stable memory on-demand and make all collections' methods transaction-like, so they would manually restore the state back to how it was before the failure. And also all methods would return an Error in that case, so developers could just react to that error and do something in order to automatically scale their app
.
But, unfortunately, this solution only works for some simple use-cases and very much makes your code unreadable.
For example, it won’t work for BTreeMap
, because it basically means, that you have (for each insert operation) allocate additional logN + 1
of nodes in advance (in order to see if there is enough memory) and if there is, you should somehow pass these newly allocated nodes inside your insertion code (which is very much recursive) in order to fill them with correct values and attach them to the tree. It is both complex and slow.
User code also becomes a mess, since you have to react to every Error
returned by every collection, in order to reset all the previous operations you did during this transaction (and yes, everyone would have to manually do that).
Then I was thinking: Yea, this idea is bad, but what about if I will grow() some amount of stable memory in advance, to always keep it above some level and if I can't, I will execute some user-specified canister method like "on_low_stable_memory()"? I won't do any transaction-specific stuff inside collections - just trap and the state is safe.
This solution sounds good and pretty simple to implement (and I assume, this is what you propose), but it doesn’t work in practice.
For it to work in practice we have to make sure, that the level of grown stable memory that we keep is always bigger or equal than our maximum possible allocation during a single call. For example, if our canister has a single method that allocates exactly 1MB each time it is called, than we only need to make sure, that we have 1MB of stable memory in advance. In this scenario, after each such call we will allocate 1MB of grown stable memory and then grow()
1MB more. If we can’t grow()
more - we just call on_low_stable_memory()
hook and everything is good.
But real collections do not work like that. For example, let’s imagine a canister that stores a history of transactions (a ledger) in a vector. Vectors work in such a way, so when they reach their maximum capacity they try to allocate twice as much of memory to continue growing.
For example, we had a vector that had capacity of 10 elements; once we inserted the 11th element, this vector will reallocate into a new memory block that can now hold 20 elements.
This means, that in order for such a vector to work properly in our “grow-in-advance” setup, we always have to grow twice the size of this vector in advance. This means, that if our transaction history occupies 2GBs of stable memory, than we have to have 4GBs more of grown in advance memory (which is, by the way, completely unused, until you can’t grow more).
Ofc, you can imagine special collections that won’t reallocate that way and will work maybe a little slower, but only consuming a small portions of new memory in order to continue growing (and that is what I initially was going towards). But the main question stays the same: how much stable memory exactly do you have to allocate in advance in order to keep it cheap (to not pay much for unused storage) and fail-proof?
I’m building this library for almost half a year now (reinventing for myself all the uni’s CS program), and I don’t know how to answer this question.
So, my final though (and this is what I propose here) - let's completely decouple both these processes. Let's make transactions trap, when they reach memory limit and let's give a user some way of understanding, how likely it is for their canister to fail.
This concept of on_low_stable_memory()
taught me one more interesting thing: If your canister can’t grow now, it doesn’t mean that it won’t be able to grow after a couple of minutes!. Memory is very flexible on IC. Some canisters decrease the total amount of available memory in a subnet, but some - increase (when destroyed).
So, providing a user with some kind of on_low_stable_memory()
system hook is actually a bad idea, because you won’t be able to answer the question: “if the subnet again has enough memory to allocate (after some canisters died), should this hook be called once again, when there is no more memory again?”. Everyone would have a different answer to it.
So it is better to just not do that, but instead give everyone a tool to track the available memory and to react how they like. For example, if we had a stable64_pages_left()
method, we could use heartbeat
in order to achieve the same result as with on_low_stable_memory()
, but with less effort and more freedom.
This is it.
P.S. Actually, as far as I understand, all of these points are also valid for common heap memory. You can easily run out of heap without even reaching 4GB’s, because of how the system works. Maybe, there should also be a method for that? Or maybe stable64_pages_left()
will automatically resolve this issue also, because when serialized the heap and stable memory are the same kind of memory, so it doesn’t matter and if stable64_pages_left()
shows 0, then you’re probably won’t be able to store more data on the heap also.