We have an issue with a canister containing NFT’s it is eating through cycles like no tomorrow. 2T an hour. We notice that even though we loaded around 1.2 GB of assets the canister size is hitting 4 BG. Canister is t2mog-myaaa-aaaal-aas7q-cai. What are we doing wrong?

[image] domwoe: There is no non-linear increase in (storage) costs when you get close to the 4 GB limit. I must have misread this the first time, i thought you were saying there is a non-linear storage cost. That is unfortunate, i was hoping it would be something obvious like that …

[image] LightningLad91: I must have misread this the first time, i thought you were saying there is a non-linear storage cost. Sorry, forgot the “no” and edited it back in afterwards :grimacing: [image] LightningLad91: Do you have an idea for why the canister is so large when the assets …

Another point to consider is that HashMap should be strongly discouraged to manage large set of data. If it has to go through rehashing of keys, it will eat through A LOT OF cycles.

@domwoe @PaulLiu thank you both. You’ve given me a lot to consider. I will work towards a solution that attempts to address these points.

[image] domwoe: Is the memory usage essentially constant or does it fluctuate/increase ? The “memory” as reported by the CanisterGeek interface is constant at 4GB. The “heap memory” starts out at 1GB after an upgrade, fluctuates between 1GB & 2GB for a bit, and then it holds steady at 2GB. (…

[image] LightningLad91: The “memory” as reported by the CanisterGeek interface is constant at 4GB. The “heap memory” starts out at 1GB after an upgrade, fluctuates between 1GB & 2GB for a bit, and then it holds steady at 2GB. (quotes are intended to convey that I’m not familiar enough with the …

Are you storing any large binary assets as [Nat8] rather than Blob? That would take 4x space and stress the GC.

Also, which version of dfx or motoko are you using. We have optimized the memory requirements of upgrades. These used to take up to 2x space during upgrade (to serialize stable variables into ordinary memory before copying them stable memory). The new releases serialise directly to stable memory in …

I believe I am using 0.10.0. Can’t remember if I pulled the latest update yet.

This could be it! I am using the ic-py python agent to call the ‘addAsset’ method. The ‘asset’ type defined in the canister expects a ‘blob’ entry but the ic-py agent doesn’t provide a ‘blob’ type so i had to structure it as ‘vec vec nat8’.

Canister burning through cycles

Developers

domwoe June 15, 2022, 8:18pm 14

There’s a thread that provides some explanation: Motoko, Array, Memory

This brings up also the issue with Array.append() which you’re also using extensively

1 Like

Topic		Replies	Views
What happens when a canister hits the memory heap limit? Developers BigMap	24	1993	July 5, 2022
NFTAnvil IC network tests report Developers	13	1587	February 2, 2023
Heap out of bounds ， use motoko Developers Functional-Programming	45	3316	June 22, 2023
Prim.rts_heap_size() and Prim.rts_memory_size() Developers	27	1627	March 7, 2023
Two questions about canister storage Developers	30	3555	March 9, 2022

Canister burning through cycles

Related topics