@haida Based on your reasoning for scaling being “data growth”, you may not actually need a dynamic canister autoscaling solution - in which case you wouldn’t need to build multi-level cycles recharge tooling. (See my recommendation at the bottom).
However, if you’re interested in how you might do this regardless, there isn’t exactly standard, but from what I’ve seen most applications with dynamic canister generation use some sort of a “cycles battery” or “cycles manager” to facilitate child canister topup when they get low. It’s up to the developer to determine what “low is”, but you can build your own cycles accounting system, or use a tool like CycleOps to profile the burn rate of different child canisters deployed to mainnet. That should give you a sense of what the right cycles topup threshold and amount would be. Additionally, then you need to instrument your canister with automated topups to ensure the battery canister stays topped up with cycles, or you can use a monitoring tool like CycleOps or CanisterGeek.
There are two ways that you can implement the dynamic child canister monitoring:
- The cycles manager pattern, where satellite canisters are aware of their own lower bound limits and when they get low, call back into the cycles manager to request more cycles. You can attach checks to update calls, timers, or, when Canister Lifecycle Hooks get released the
canister_on_low_cycles hook.
- On a periodic basis, have the cycles battery check the balances on each of the child canister if necessary. This requires inter-canister calls for every cycles balance check, so in many cases option #1 may be preferred.
For multi-level canister apps, it depends. You can still just hook everything up to the same cycles manager/battery canister, which might be simpler.
Battery canister -> distributes cycles to all canisters
Some applications have lower level canister factories (factories spin up new canisters), and have chosen to distribute cycles from those factory canisters to the child canisters spun up by that factory
Battery canister -> distributes cycles to factory canisters -> distributes cycles to child canisters underneath each factory canister.
This all being said, I’d recommend keeping things simple if possible to start with.
- Use a single cycles manager/battery canister to distribute cycles to all child canisters
- Don’t dynamically spin up more canisters than you need (easier to split canister later than to combine canisters after they’ve split).
If data growth is the reason for scaling, you may not need to spin up new canisters for this.
- Rust has stable-structures, which allow you to store data in canister stable memory (up to 400 GB available)
- Motoko recently merged it’s orthogonal persistence feature, which once 64-bit heap memory is available to canisters (coming soon) should eventually allow Motoko developers to store up to the full canister memory in heap memory (~400GB).
Therefore, if data growth is your concern I’d potentially recommend starting out with a single canister or few canister architecture. For example, then instead of multiple account contracts start with one, and the same for your transaction ledger canister, credits canister, etc.
As an added benefit, then you don’t need to write a dynamic cycles management system and can focus your time in other areas!
