Possible memory leak with `VarArray.repeat` in Motoko

Developers Motoko
1

I am building an identity provider and noticed, thanks to cycleops, that memory usage was increasing by a few MB on every sign-in.

To debug this, I reduced the issue to a minimal example which points to VarArray.repeat as the likely source of the leak.

Here is the code:
https://icp.ninja/i?s=z8MqX

import Nat "mo:core/Nat";
import VarArray "mo:core/VarArray";
import Text "mo:core/Text";
import Prim "mo:⛔"; // only for reporting memory size

persistent actor {
  var testCallCount = 0;

  public shared func test() : async () {
    testCallCount += 1;

    for (i in Nat.range(0, 1000)) {
      ignore VarArray.repeat(0 : Nat8, 1000)
    }
  };

  public shared query func getStats() : async Text {
    debug_show ({
      testCallCount;
      rts_memory_size = Prim.rts_memory_size();
      rts_heap_size = Prim.rts_heap_size();
      rts_stable_memory_size = Prim.rts_stable_memory_size()
    })
  };
}

Steps to reproduce:

  1. Call getStats → note the heap size.
  2. Call test once, then getStats → heap grows by about 8 MB.
    1_000_000 elements × 8 bytes (64-bit) = 8 MB.
  3. Call test a few more times → heap grows linearly with the number of calls.

The allocations in test are ignored and not stored anywhere, so they should be eligible for garbage collection. But the heap size keeps increasing and is not reclaimed.

2

Thanks for the report and example code! I created a GitHub issue to track the bug. It’s worth mentioning that this would likely also affect other usages of Prim.Array_init, which is used behind the scenes to implement VarArray.repeat.

3

Thank you for reporting this. We will investigate but it might not be a bug, actually.

Even though the arrays are eligible for garbage collection, this doesn’t mean that GC happens immediately. GC usually kicks in after several update messages being scheduled (using a heuristic to determine whether it should a GC phase or not).

For example, if you rewrite your for-loop as:

    for (i in Nat.range(0, 10)) {
      ignore VarArray.repeat(0 : Nat8, 1000 * 1000);
      await async {};
    }

you will see how the GC kicks in more frequently.

4

I tried the suggestion with await async {} inside the loop.

Now I see that rts_memory_size grows until a ceiling of a bit over 200 MB, and then stops. At the same time rts_heap_size shrinks when GC kicks in.

Interestingly, I get similar results with the original example without the async call, so the await async {} does not seem to be the deciding factor here.

This is more than I would have expected, but at least it shows the growth is not unbounded.

Thanks for the clarification and for looking into this.