Failed to approach canister's heap limit

Developers
1

Hi everyone

So I want to fill my canister’s with natural numbers, until the heap size reached its limit (4 GiB).
But somehow it always got killed, before even going beyond 2 GiB. I got the following errors:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on 
an `Err` value: ENOMEM', rs/memory_tracker/src/lib.rs:424:14

Jun 01 00:56:01.624 ERRO 
s:s2kp6-hlfbh-5gyuw-ynveb-qs7or-kerhk-rhwmj-f2l2l-z2vox-2n6hp-3ae
/n:gidfg-6ox3w-nxm25-3muzj-mmr47-tkj6g-k6fso-kbvet-guebs-2rw6e-bae
/ic_canister_sandbox_replica_controller
/sandboxed_execution_controller 
History for canister rwlgt-iiaaa-aaaaa-aaaaa-cai with pid 28929: 
CloseMemory(memory_id=memory-id-983)

here’s my test source code:

let { nhash } = Utils;
stable let set = Set.new<Nat>(nhash);
public func add(len : Nat) : async Nat {
	for (i in Iter.range(0, len - 1)) {
		Set.add(set, nhash, Set.size(set));
	};

	Set.size(set);
};

public query func mem() : async {

} {
	let heap = Prim.rts_heap_size();
	let memo = Prim.rts_memory_size();
	let max = 4294967296; // 4 GiB
	return {
		heap = heap; 
		h = (heap * 100) / max; 
		memo = memo;
		m = (memo * 100) / max; 
		h_m = (heap * 100) / memo;
	};
};

how i test:
I insert 1 million natural numbers repeatedly, but after heap is 12%, it will throw the above error.
then I insert 100k nat nums repeatedly, same thing.
then i insert 10k nat nums repeatedly, the heap can go until 18% before it throws.
then i insert 1k nat nums repeatedly, the heap goes to 21% before it throws.
what’s the easiest way to fill a canister and avoiding the above error? because inserting 100 natnums repeatedly will take a very long time.

I think i read somewhere on the forum that it’s the canister’s ddos protection mechanism? is there any way i can turn this off for local canisters?

note: the Set was made by ZhenyaUsenko.

previous thread

1 Like
2

Found something interesting.
Looks like the “maximum size” of @ZhenyaUsenko 's Set (using the nhash function) is 15734651.
cc: @skilesare

I spawned multiple processes to insert A natural number into the Set, repeatedly. Now all the processes are throwing this error: The Replica returned an error: code 5, message: "Canister rwlgt-iiaaa-aaaaa-aaaaa-cai exceeded the instruction limit for single message execution.". Maybe it reached the limit by checking the uniqueness of each item.

image
image907×1110 69.7 KB

3

What garbage collector were you testing with?
You wont be able to go above 2GB with the default one. As there should be some space left for the garbage collector to operate. With the recent addition of Incremental GC, though, it should be possible to get close to 4GB. However, I don’t know what the safe limit is (until the canister becomes unresponsive). @icme did some testing Incremental testing results - from the Incremental GC :)

2 Likes