# ICRC-1 Ledger Upgrade Compatibility Issue: Cannot Upgrade Past 2024-09-13

Developers
1

Issue Summary

We’re unable to upgrade our ICRC-1 ledger canister to any version released after 2024-09-13 due to stable memory compatibility issues.

Technical Details

Current Working Version: 2024-09-13 (commit: 5ba1412f)
Canister ID: irorr-5aaaa-aaaak-qddsq-cai
Network: IC Mainnet

Failed Upgrade Attempts

Attempted to upgrade to the following ledger-suite-icrc releases, all failing with the same error:

  • :cross_mark: 2024-10-17
  • :cross_mark: 2024-11-28
  • :cross_mark: 2025-01-07
  • :cross_mark: 2025-01-21
  • :cross_mark: 2025-02-27
  • :cross_mark: 2025-04-14
  • :cross_mark: 2025-05-22

Error Message

Error from Canister irorr-5aaaa-aaaak-qddsq-cai: Canister called `ic0.trap` with message: 
'Panicked at 'Cannot upgrade from scratch stable memory, please upgrade to memory manager first.', 
rs/ledger_suite/icrc1/ledger/src/main.rs:192:9'

Upgrade Command Used

dfx deploy dusd_ledger --argument '(variant { 
  Upgrade = opt record { 
    token_symbol = opt "DUSD"; 
    token_name = opt "Doxa USD"; 
  } 
})' --ic --identity doxa

Root Cause Analysis

The error suggests that ICRC-1 ledgers deployed before 2024-10-17 use “scratch stable memory” format, while newer versions expect “memory manager” format. This appears to be a breaking change in the stable memory layout.

Questions for the Community

  1. Is there an intermediate upgrade path between 2024-09-13 and 2024-10-17 that handles the memory format migration?

  2. Are there any migration tools or specific upgrade procedures for ledgers stuck on pre-2024-10-17 versions?

  3. What’s the recommended approach for production ICRC-1 ledgers that need to stay on older versions for compatibility?

  4. Will this be addressed in future releases with backward compatibility support?

Related Discussion

Similar issue discussed here: Upgrading old ICRC1 token to the ICRC ledger suite

Any guidance on safe upgrade paths or migration strategies would be greatly appreciated!

2

Are you aware of this migration guide? (cc @mathiasb)

3

no but thankyou for sharing
i am reading it

1 Like
4

Hi @Vishnu, thanks for the details, and sorry to hear you’re having issues upgrading your ledger canister!

Looking at the canister info and metadata, I’m a bit confused:

$ dfx canister info --network ic irorr-5aaaa-aaaak-qddsq-cai
Controllers: 2bfxp-uzezm-gf5ny-ztks2-ybgzc-4dfjc-7gdhs-he2ek-vdjfk-w4yph-fqe 6cfa5-kyaaa-aaaag-qdk4a-cai
Module hash: 0xa170bfdce5d66e751a3cc03747cb0f06b450af500e75e15976ec08a3f5691f4c

The module has suggests that this canister is running ledger-suite-icrc-2024-09-13.

$ dfx canister --ic metadata irorr-5aaaa-aaaak-qddsq-cai git_commit_id
e54d3fa34ded227c885d04e64505fa4b5d564743

However, the git_commit_id metadata suggests it is running a commit from October 17, 2024.

Thanks for pointing to the migration post @marc0olo!
@Vishnu if you are able to upgrade after referring to the instructions in the post, feel free to disregard my follow-up questions.

Just to clarify, you are running the Rust ICRC ledger from the ic repository? Did you build the canister yourself, or download the WASM from a CDN? The error suggests that you are running a pre-2024-09-13 release of the ledger.

Regarding your questions for the community:

  1. Is there an intermediate upgrade path between 2024-09-13 and 2024-10-17 that handles the memory format migration?

    No, the only (tested) path is the one documented in the post @marc0olo pointed to.

  2. Are there any migration tools or specific upgrade procedures for ledgers stuck on pre-2024-10-17 versions?

    Again, I’d refer to the other post.

  3. What’s the recommended approach for production ICRC-1 ledgers that need to stay on older versions for compatibility?

    DFINITY won’t maintain older versions with e.g., bugfixes or backporting of features. The fact that the older ledger versions store their data on the heap rather than in stable structures means that they are limited by the amount of data they can store. The recommended approach is to upgrade to the newer versions, and if you’re encountering issues, we’ll do our best to figure them out and help you upgrade!

  4. Will this be addressed in future releases with backward compatibility support?

    I’m not sure I fully understand the question, but backward compatibility in the sense of being to roll back a ledger running a newer WASM version that stores its data in stable structures, to an older version storing its data on the heap, will not be supported. This was considered as part of the migration work, but decided against, with one of the primary reasons being that ledgers using stable structures may end up storing large amounts of data that would no longer fit on the heap (and/or (de)serializing the data to/from stable memory during canister upgrades would run into the instruction limit).

1 Like
5

@Vishnu The error message you posted:

Error from Canister irorr-5aaaa-aaaak-qddsq-cai: Canister called `ic0.trap` with message: 
'Panicked at 'Cannot upgrade from scratch stable memory, please upgrade to memory manager first.', 
rs/ledger_suite/icrc1/ledger/src/main.rs:192:9'

seems to be from the 2024-11-28 release. The earlier releases do not fail with that error, so if you could let us know what the error is when trying to upgrade to the 2024-10-17 release, that would be helpful!