# Suggestion: make the maximum single memory operation an explicit limit

**URL:** <https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959>\
**Category:** Developers\
**Created:** [August 6, 2026, 1:51pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959 "2026-08-06T13:51:03Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\
**Post date:** [August 6, 2026, 1:51pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/1 "2026-08-06T13:51:03Z")

</div>

We run LLM inference on-chain ([llama\_cpp\_canister](https://github.com/onicai/llama_cpp_canister)), so our canisters allocate multi-GiB heaps. After the `-deterministic-tracker` GuestOS variant rolled out, loading our Qwen3-1.7B model with a 16K context stopped working. We load the model from stable memory (a virtual file in C++ canisters) into working memory:

```auto
IC0522: Canister attempted to perform a large memory operation that used
4_057_019_087 instructions and exceeded the slice limit 2_000_000_000.

```

An existing, deployed canister stopped working.

We found that using a smaller context still works fine, so we have a workaround, though not ideal.

After some digging, and with help from @Lorimer, we found the root cause: `b7225383e` unified page charging at 5 000 instructions/OS page (heap writes were 1 000, heap reads 0). A single bulk memory operation can’t be sliced, so its whole cost lands in one slice, and DTS rejects it once it exceeds `slice_limit + max_slice_limit` = 4 B instructions. Expressed as a size:

```auto
max single memory op = 2 × MAX_INSTRUCTIONS_PER_SLICE / dirty_page_overhead × 4 KiB

   before: 2 × 2B / 1000 × 4KiB = 15.26 GiB (heap reads were free → effectively unbounded)
   after: 2 × 2B / 5000 × 4KiB = 3.05 GiB
   ours: 4_057_019_087 / 5000 × 4KiB = 3.10 GiB → just over

```

We’re not arguing against the new cost model, but would like to point out that this particular size limit isn’t declared anywhere. It’s an emergent result of `MAX_INSTRUCTIONS_PER_SLICE` (a latency knob) and `dirty_page_overhead` (a cost knob), so it moves whenever either changes for unrelated reasons, and it isn’t obvious from either PR that a size limit is being affected at all.

Two small things that would help canisters like ours:

1. An explicit, documented constant for it — ideally with a test pinning it — so that a change in either input surfaces the effect on maximum operation size at review time.
2. An error that states the size: _“attempted a 3.10 GiB memory operation, limit is 3.05 GiB”_. The current message gives instruction counts, and the conversion to bytes isn’t obvious unless you already know the page cost.

And a related question, if the limit does become explicit: **would it be possible to raise it a little?** At 3.05 GiB it now sits just below the ~6 GiB heap the platform already supports, so a canister can’t initialise its whole heap in a single operation. We don’t have a strong view on the mechanism — we appreciate `MAX_INSTRUCTIONS_PER_SLICE` is deliberately tied to block time — but a limit at least as large as the maximum supported heap would feel natural, and would let us go back to the 16K-context configuration.

We’ll try to chunk our allocations on our side regardless although it is not going to be easy to implement.

---

<div class="post-metadata">

**Author:** ![ZackDS](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/zackds/32/40368_2.png) [@ZackDS](https://forum.dfinity.org/u/ZackDS)\
**Post date:** [August 6, 2026, 2:21pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/2 "2026-08-06T14:21:42Z")

</div>

Second interesting one, after the BTC cannister [issue](https://forum.dfinity.org/t/proposal-143242-to-upgrade-the-bitcoin-canister/74861/2) .  
[https://forum.dfinity.org/t/proposal-to-elect-new-release-rc-2026-07-17-04-19/74609/7?u=zackds](https://forum.dfinity.org/t/proposal-to-elect-new-release-rc-2026-07-17-04-19/74609/7)

---

<div class="post-metadata">

**Author:** ![stefan.schneider](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@stefan.schneider](https://forum.dfinity.org/u/stefan.schneider)\
**Post date:** [August 6, 2026, 3:04pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/3 "2026-08-06T15:04:15Z")

</div>

Hmm interesting. It’s unfortunate that the change had a negative impact on your workload.

Increasing the slice limit (how much of a multi-round message fits into a round) is not really an option, because that would directly affect the round duration.

In your case, would it be possible to just do multiple, smaller allocations? Because that would allow the protocol to split it up over several rounds. With the 6GB heap size limit soon getting lifted, it won’t be possible to allocate everything all at once either way.

---

<div class="post-metadata">

**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\
**Post date:** [August 6, 2026, 3:18pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/4 "2026-08-06T15:18:30Z")

</div>

Thanks for the explanation.

> [@stefan.schneider](#):
>
> would it be possible to just do multiple, smaller allocations

Not easily done. We’re vendoring regular llama.cpp code inside our llama\_cpp\_canister code, and this will require updating that vendored code.

Not impossible though and we will go that route.

---

<div class="post-metadata">

**Author:** ![michael-weigelt](https://avatars.discourse-cdn.com/v4/letter/m/b487fb/32.png) [@michael-weigelt](https://forum.dfinity.org/u/michael-weigelt)\
**Post date:** [August 6, 2026, 3:24pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/5 "2026-08-06T15:24:35Z")

</div>

Hi icpp,

> [@icpp](#):
>
> An existing, deployed canister stopped working.

This is unfortunate and I am sorry for your inconvenience. Prior to the change, we rolled out metrics on all subnets which estimated the new costs so that we could rule out cases such as yours. However, we focussed on message limits, not on slice limits. It is rather unusual (or so we thought) that a single instruction would go close to the slice limit. It is obviously a miss on our part and we should have announced this change ahead of time.

> An explicit, documented constant for it

That does not make sense to me, as you have explained in the paragraph above: It’s the result of two existing constants.

> we appreciate `MAX_INSTRUCTIONS_PER_SLICE` is deliberately tied to block time — but a limit at least as large as the maximum supported heap would feel natural,

This is again a contradiction. These read a bit like an LLM is talking ☹

But the error message improvement is noted.

> **would it be possible to raise it a little**

We are still on the low end of a fair price for a page, so this is unlikely. The price may increase further in the future. But why do you think it’s difficult to fix it on your end? Why not just go from

```auto
bulk_instr(2N);

```

to

```auto
bulk_instr(N);
bulk_instr(N);

```

? That can be DTS’d between the two system API calls.

Edit: I have now seen your answer to this question.

Semi-related: This change unblocks a future change to make the heap limit much larger, which might be good news for your canister. But it also shows that the total heap size cannot ever be completely accessible in a slice or even message.

---

<div class="post-metadata">

**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\
**Post date:** [August 6, 2026, 3:55pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/6 "2026-08-06T15:55:51Z")

</div>

> [@External Image](https://avatars.discourse-cdn.com/v4/letter/m/b487fb/48.png):
>
> These read a bit like an LLM is talking ☹

Sorry for that. It was indeed a lot of LLM speak there. Thanks for unwrapping that for me.

---

<div class="post-metadata">

**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\
**Post date:** [August 6, 2026, 4:05pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/7 "2026-08-06T16:05:06Z")

</div>

> [@External Image](https://avatars.discourse-cdn.com/v4/letter/m/b487fb/48.png):
>
> ? That can be DTS’d between the two system API calls.

Actually, this is the solution. The update appears to be quite straight forward. I will report back if it indeed works out.

Thanks again for your detailed response.

---

<div class="post-metadata">

**Author:** ![Lorimer](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/lorimer/32/17601_2.png) [@Lorimer](https://forum.dfinity.org/u/Lorimer)\
**Post date:** [August 6, 2026, 5:43pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/8 "2026-08-06T17:43:55Z")

</div>

Also kudos to @Zane for his review on the relevant commit. That made it much easier to track the issue down when I searched the forum.

> [@On another topic](https://forum.dfinity.org/t/74732/10):
>
> [b7225383e](https://github.com/dfinity/ic/commit/b7225383e) (Only on replica version b7225383ebc592fa5f4e1eac1c8e403e1e098ef1) Switched Wasm page handling to also use the deterministic memory tracker for instruction charging, rather than charging dirty heap and stable memory pages through separate paths. Previously `WasmtimeEmbedder` passed a nominal page overhead of 1 to the DMT and `list_memory_infos` always set DirtyPageTracking::Ignore for stable memory. `WasmtimeEmbedder` now passes self.config.dirty\_page\_overhead to `sigsegv_memory_tracker`, with the default increased from 1000 to 5000, and makes stable memory use the `dirty_page_tracking` value derived from `ModificationTracking`. DMT can therefore charge heap and stable memory page accesses and writes during execution, allowing DTS to pause memory intensive messages correctly. Removed the previous post execution heap charge from `WasmtimeInstance::run` and the dirty page decrements from `StableWrite` and `Stable64Write` to avoid charging newly dirty stable memory pages twice.

---

<div class="post-metadata">

**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\
**Post date:** [August 6, 2026, 5:48pm UTC](https://forum.dfinity.org/t/suggestion-make-the-maximum-single-memory-operation-an-explicit-limit/74959/9 "2026-08-06T17:48:30Z")

</div>

Good news. It turns out llama.cpp can already split a large allocation into several smaller buffers, which are then zeroed one at a time instead of in a single operation — the CPU backend just doesn’t switch it on by default.

Enabling it in our wasm build was enough and now llama\_cpp\_canister can again load Qwen3-1.7B at the larger context window. Verified on mainnet and available in the next release (0.16.4).

Thanks again for the pointers.