As BigMap library is late to the party I decided to build a mini BigMap library which is an auto-scalable canister solution.
Note this is an MVP and there’s still work to do. eg: inter-canister query calls etc.
The core architecture relies on actor classes that allows canister creation on the fly. The container actor stores an array of all the canisters created on the fly and the size of that canisters.
In order to get a canister memory size I’m using
rts_memory_size : the rts_memory_size is the current size of the Wasm memory array (mostly the same as the canister memory size). This can only grow, never shrink in Wasm, even if most of it is unused. However, unused/unmodified canister memory ought to have no cost on the IC.
You can use the IC management canister interface to get the memory of a canister as well ie:
IC.status but rts_memory_size and IC.status numbers are not the same - the implementation will do some more work before and after after you call rts_memory_size (deserializing arguments and serializing results, which may affect memory size after you’ve read it. So you can use either but from my tests
rts_memory_size is closer to reality.
Due to the way garbage collector is built in motoko (takes about half of the canister memory so a bit less than 2GB) and from my tests I decided to set a threshold of 2GB
private let threshold = 2147483648; // ~2GB so once the threshold is reached the container will spawn a new canister for storage.
Note : you only pay for what you consume.
Using the IC management canister I can update the new canister settings
compute_allocation = ?5; memory_allocation = ?4294967296; and the controllers to the wallet canister and the container canister.
Frontend: you can upload any type of file from this category:
jpeg, gif, jpg, png, svg, avi, aac, mp4, wav, mp3 but you can update the front-end
getFileExtension to allow/remove types.
Files are split in chunks of 500Kb and uploaded into an available bucket.
Any improvements/comments are welcome.
Many thanks to @claudio