Message memory is currently not appropriately charged for. We propose a charging model that fixes this while only minimally impacting typical workloads.
Whenever a canister produces a message for another canister, i.e., a canister-to-canister message, it has to be stored in the replicated state until it is delivered and picked up for processing by the recipient. Message memory is a shared resource among all canisters on a subnet. It is different to other canister related types of memory – like heap or stable memory – because messages are ephemeral by their nature, and, in the current implementation of the protocol, messages are held in RAM at all times and serialized/deserialized upon every checkpoint. Also messages in subnet-to-subnet streams have to be certified every round. Naturally, this means that message memory is more resource intensive than other canister related types of memory. The pricing for message memory, however, is currently the same as for other canister related types of memory.
As described above, the reasoning behind the current pricing for message memory has a major flaw: message memory should be considered much more valuable than other types of subnet memory because it is scarcer and additionally has a bigger relative impact on replica and subnet performance.
Similarly to the previous case of fixing the incorrect compute allocation fee, we would like to address the flaw in message memory pricing.
We target a model that should have minimal impact for typical workloads, while still leading to a reasonable fee for utilizing big chunks of message memory for prolonged periods of time.
Concretely, we propose to charge 0.125 cycles per byte per second (or approximately 250K cycles per message per second).
For subnet-local inter-canister calls, which complete within the same or the next execution round, the impact would be minimal. For actual cross-subnet calls that complete within 10 seconds (the median cross-subnet roundtrip time is 5 seconds), the additional cost would be around 2.5M cycles, which is the same as would be charged for an additional 2.5 KB of payload.
If the NNS proposal to elect a (to be implemented) replica binary revision which uses the new price for message memory and charges accordingly is accepted, the change will be rolled out to all subnets via subsequent upgrade proposals.