ICP Ledger Canister Suite Upgrades 2024-11-04

[mathiasb]

The Financial Integrations team has submitted the following proposals today, 2024-11-04 for upgrading the ICP Ledger canister suite. DFINITY plans to vote on these proposals on Thursday.

ICP Index (proposal 133945):

Proposal to upgrade the ICP index canister

Repository: https://github.com/dfinity/ic.git

Git hash: dac2f36f96d7549d82fa8e3c714979255ce57afd

New compressed Wasm hash: 3cc807d6c602be5041635c03522b99049d91c2a2ed18cb87251d5611ef779c98

Upgrade arguments hash: 0fee102bd16b053022b69f2c65fd5e2f41d150ce9c214ac8731cfaf496ebda4e

Target canister: qhbym-qaaaa-aaaaa-aaafq-cai

Previous ICP index proposal: https://dashboard.internetcomputer.org/proposal/133374

---

Motivation

Update ic-cdk to 0.16.0, implement stable structures storable interface for Allowance.

Upgrade args

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
cd rs/ledger_suite/icp/index
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' 6dcfafb491092704d374317d9a72a7ad2475d7c9..dac2f36f96d7549d82fa8e3c714979255ce57afd -- packages/icrc-ledger_types rs/ledger_suite/icp/index rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core/src rs/ledger_suite/common/ledger_core rs/rust_canisters/http_types rs/rust_canisters/canister_log
61a229ece test(ICP_ledger): FI-1506: Thread limit in flaky ICP ledger and index tests (#2208)
944b8d0c3 feat(ledger-core): FI-1437: Implement stable structures storable interface for Allowance (#2000)
03dd6ee6d fix(Ledger-Suite): renamed state machine tests (#2014)
d70b9eb6f feat(ICP-Ledger): FI-1508 add test icp allowance getter endpoint (#1934)
1a484503e chore: StateMachine tests only bump time if strictly necessary (#1892)
fcbc91f0a chore: update `ic-cdk` to 0.16.0 (#1868)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
"./ci/container/build-ic.sh" "--canisters"
sha256sum ./artifacts/canisters/ic-icp-index-canister.wasm.gz

ICP Ledger (proposal 133946):

Proposal to upgrade the ICP ledger canister

Repository: https://github.com/dfinity/ic.git

Git hash: dac2f36f96d7549d82fa8e3c714979255ce57afd

New compressed Wasm hash: 50c05fd687883fe788c0bb91996de358d8f856ba56088c6ff47767ea853001d7

Upgrade arguments hash: 0fee102bd16b053022b69f2c65fd5e2f41d150ce9c214ac8731cfaf496ebda4e

Target canister: ryjl3-tyaaa-aaaaa-aaaba-cai

Previous ICP ledger proposal: https://dashboard.internetcomputer.org/proposal/133375

---

Motivation

Use MemoryManager during upgrade, implement stable structures storable interface for Allowance.

Upgrade args

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' 6dcfafb491092704d374317d9a72a7ad2475d7c9..dac2f36f96d7549d82fa8e3c714979255ce57afd -- packages/icrc-ledger_types rs/ledger_suite/icp/ledger/src rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core/src rs/ledger_suite/common/ledger_core/src
b811de98a feat(ICP-Ledger): FI-1436: Implement V2 for ICP ledger - use memory manager during upgrade (#1969)
944b8d0c3 feat(ledger-core): FI-1437: Implement stable structures storable interface for Allowance (#2000)
d70b9eb6f feat(ICP-Ledger): FI-1508 add test icp allowance getter endpoint (#1934)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
"./ci/container/build-ic.sh" "--canisters"
sha256sum ./artifacts/canisters/ledger-canister_notify-method.wasm.gz

ICP Archive 1 (proposal 133947):

Proposal to upgrade the ICP archive1 canister

Repository: https://github.com/dfinity/ic.git

Git hash: dac2f36f96d7549d82fa8e3c714979255ce57afd

New compressed Wasm hash: 9c3ef43b1c68223d98bae6a5664d33d490c6abdcec356077022212a1de2b13a4

Upgrade arguments hash: 0fee102bd16b053022b69f2c65fd5e2f41d150ce9c214ac8731cfaf496ebda4e

Target canister: qjdve-lqaaa-aaaaa-aaaeq-cai

Previous ICP archive1 proposal: https://dashboard.internetcomputer.org/proposal/133376

---

Motivation

Implement stable structures storable interface for Allowance.

Upgrade args

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' 6dcfafb491092704d374317d9a72a7ad2475d7c9..dac2f36f96d7549d82fa8e3c714979255ce57afd -- packages/icrc-ledger_types rs/ledger_suite/icp/archive rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core rs/ledger_suite/common/ledger_core
944b8d0c3 feat(ledger-core): FI-1437: Implement stable structures storable interface for Allowance (#2000)
d70b9eb6f feat(ICP-Ledger): FI-1508 add test icp allowance getter endpoint (#1934)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
"./ci/container/build-ic.sh" "--canisters"
sha256sum ./artifacts/canisters/ledger-archive-node-canister.wasm.gz

ICP Archive 2 (proposal 133948):

Proposal to upgrade the ICP archive2 canister

Repository: https://github.com/dfinity/ic.git

Git hash: dac2f36f96d7549d82fa8e3c714979255ce57afd

New compressed Wasm hash: 9c3ef43b1c68223d98bae6a5664d33d490c6abdcec356077022212a1de2b13a4

Upgrade arguments hash: 0fee102bd16b053022b69f2c65fd5e2f41d150ce9c214ac8731cfaf496ebda4e

Target canister: qsgjb-riaaa-aaaaa-aaaga-cai

Previous ICP archive2 proposal: https://dashboard.internetcomputer.org/proposal/133377

---

Motivation

Implement stable structures storable interface for Allowance.

Upgrade args

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' 6dcfafb491092704d374317d9a72a7ad2475d7c9..dac2f36f96d7549d82fa8e3c714979255ce57afd -- packages/icrc-ledger_types rs/ledger_suite/icp/archive rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core rs/ledger_suite/common/ledger_core
944b8d0c3 feat(ledger-core): FI-1437: Implement stable structures storable interface for Allowance (#2000)
d70b9eb6f feat(ICP-Ledger): FI-1508 add test icp allowance getter endpoint (#1934)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
"./ci/container/build-ic.sh" "--canisters"
sha256sum ./artifacts/canisters/ledger-archive-node-canister.wasm.gz

ICP Archive 3 (proposal 133949):

Proposal to upgrade the ICP archive3 canister

Repository: https://github.com/dfinity/ic.git

Git hash: dac2f36f96d7549d82fa8e3c714979255ce57afd

New compressed Wasm hash: 9c3ef43b1c68223d98bae6a5664d33d490c6abdcec356077022212a1de2b13a4

Upgrade arguments hash: 0fee102bd16b053022b69f2c65fd5e2f41d150ce9c214ac8731cfaf496ebda4e

Target canister: q4eej-kyaaa-aaaaa-aaaha-cai

Previous ICP archive3 proposal: https://dashboard.internetcomputer.org/proposal/133378

---

Motivation

Implement stable structures storable interface for Allowance.

Upgrade args

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' 6dcfafb491092704d374317d9a72a7ad2475d7c9..dac2f36f96d7549d82fa8e3c714979255ce57afd -- packages/icrc-ledger_types rs/ledger_suite/icp/archive rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core rs/ledger_suite/common/ledger_core
944b8d0c3 feat(ledger-core): FI-1437: Implement stable structures storable interface for Allowance (#2000)
d70b9eb6f feat(ICP-Ledger): FI-1508 add test icp allowance getter endpoint (#1934)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout dac2f36f96d7549d82fa8e3c714979255ce57afd
"./ci/container/build-ic.sh" "--canisters"
sha256sum ./artifacts/canisters/ledger-archive-node-canister.wasm.gz

[cyberowl]

proposal - 133945

Vote: ADOPT

Reason:

The hashes for both the Wasm and the arguments match, along with all the specified commits.

Hash Match: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits

b579d94bef
Similar to proposal 133905

The main changes include allowing opaque callers for the get_changes_since function in the registry canister, which removes previous authorization restrictions.

The get_changes_since function is a read-only operation, designed to retrieve registry changes from a specified version onward, without performing any mutations or modifications.

[61a229ece]
Add env RUST_TEST_THREADS to fix tests

[944b8d0c3]
Adds ic-stable-structures to ledger core.
Storable trait for the Allowance struct parameterized with a generic Tokens type that can be converted to and from Nat. It defines methods for serializing (to_bytes) and deserializing (from_bytes) the Allowance instance, storing only the amount and expires_at fields, while excluding arrived_at as it’s not required after a migration.

[03dd6ee6d]
Renaming the test suite from ic-icrc1-ledger-sm-tests to ic-ledger-suite-state-machine-tests across various files in the codebase, including Cargo.toml, BUILD.bazel, and Rust test files.

[d70b9eb6f]
icrc2_allowance, which retrieves the allowance of a specified spender for a given account if ICRC-2 features are enabled on the ledger, and exposes it as a Candid method for external calls, with conditional compilation to allow a similar allowance query under specific features (icp-allowance-getter).

[1a484503e]
Incrementing the time only by 1 nanosecond when needed to ensure it progresses between test rounds to avoid unnecessary time bumps and maintain sequential timing consistency in StateMachine tests.

[fcbc91f0a]
ic-cdk and ic-cdk-macros dependencies from version 0.13.5 to 0.16.0 across multiple files. Standardize the use of stable_size, stable_grow, stable_read, and stable_write methods instead of their previous stable64_ counterparts.

proposal - 133946

Vote: ADOPT

Reason:

The hashes for both the Wasm and the arguments match, along with all the specified commits.

Hash Match: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits

[b811de98a]
Support for building multiple versions of the ledger library by dynamically setting name_suffix and crate_features for each version in a list comprehension, including a new ledger_next_version with the next-ledger-version feature.

[944b8d0c3]
Duplicate from proposal 133945

[d70b9eb6f]
Duplicate from proposal 133945

proposal - 133947

Vote: ADOPT

Reason:

The hashes for both the Wasm and the arguments match, along with all the specified commits.

Hash Match: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits

[944b8d0c3]
Duplicate from proposal 133945

[d70b9eb6f]
Duplicate from proposal 133945

proposal - 133948

Vote: ADOPT

Reason:

The hashes for both the Wasm and the arguments match, along with all the specified commits.

Hash Match: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits

[944b8d0c3]
Duplicate from proposal 133945

[d70b9eb6f]
Duplicate from proposal 133945

proposal - 133949

Vote: ADOPT

Reason:

The hashes for both the Wasm and the arguments match, along with all the specified commits.

Hash Match: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits

[944b8d0c3]
Duplicate from proposal 133945

[d70b9eb6f]
Duplicate from proposal 133945

[ZackDS]

Proposal #133945

Vote: Adopted
Reason: Both of the hashes match for the wasm and arguments, as do all the listed commits.

61a229ece Sets a limit of 4 on the number of threads used flaky ICP ledger and index tests.
944b8d0c3 Implement ic_stable_structures storable interface for Allowance, and adds the allowance_serialization test. Was reviewed before.
03dd6ee6d Simple rename of ic-icrc1-ledger-sm-tests to ic-ledger-suite-state-machine-tests.
d70b9eb6f Adds allowance endpoint to the ICP ledger but only for testing.
1a484503e Matches description.
fcbc91f0a Updates ic-cdk from 0.15.0 to 0.16.0.

Proposal #133946

Vote: Adopted
Reason: Both of the hashes match for the wasm and arguments, as do all the listed commits.

b811de98a Adds LEDGER_VERSION used for only forward upgrade compatibility.
Rest are the same as for the Index upgrade proposal.

Proposal #133947

Vote: Adopted
Reason: Both of the hashes match for the wasm and arguments, as do all the listed commits.

Same as for Index.

Proposal #133948

Vote: Adopted
Reason: Both of the hashes match for the wasm and arguments, as do all the listed commits.

Same as Archive1.

Proposal #133949

Vote: Adopted
Reason: Both of the hashes match for the wasm and arguments, as do all the listed commits.

Same as Archive1.

[0rions]

Proposal 133945

It’s not mentioned how the args were computed, but they correspond to an empty parameter:

didc encode '()' | xxd -r -p | sha256sum
0fee102bd16b053022b69f2c65fd5e2f41d150ce9c214ac8731cfaf496ebda4e  -

Args hash, wasm hash and install mode are reproducible for this proposal.

This update adds dependencies on ic-stable-structures and proptest for stable serialization and testing in ledger_core, implementing Storable for Allowance<Tokens> preparing to move allowance map to stablestructure. A rename aligns ic-icrc1-ledger-sm-tests to ic-ledger-suite-state-machine-tests. No issues noted.

Proposal 133946

Args hash, wasm hash and install mode are reproducible for this proposal.

This update focuses on serialization for Allowance<Tokens> in approvals.rs, adding an implementation of Storable and enhancing tests to ensure deserialization consistency. A new default_ledger_version function initializes the ledger version, ensuring version compatibility during upgrades and avoiding downgrades.

An icp_allowance getter was also introduced, gated by the icp-allowance-getter feature, allowing users to query allowances with IcpAllowanceArgs without knowing the principal which is more consistent with the ICP ledger implementation.

Proposal 133947 - Proposal 133948 - Proposal 133948

Args hash, wasm hash and install mode are reproducible for this proposal.

This update adds stable storage support and tests for allowances in the ledger core. New dependencies (ic-stable-structures and proptest) were added. The Allowance<Tokens> struct now implements Storable for serialization, excluding arrived_at. IcpAllowanceArgs was introduced for allowance-related API structuring. Changes appear straightforward, with no issues detected.

[LaCosta]

Proposal 133945

Vote: ADOPT

Reason:
Build successful and hashes match, commits look great and found no issues.

Review:

[61a229ece]: Adds a new env variable RUST_TEST_THREADS = 4 that sets a limit of 4 threads for running flaky ICP ledger and index tests.

[944b8d0c3]: Implements a new interface for serializing and deserializing Allowance, enabling storage within stable structures. To accomplish this a new trait Storage is implemented for the Allowance which includes methods to_bytes which converts the Allowance amount and expires_at fields into a byte array and from_bytes that does the opposite.

[03dd6ee6d]: Renames the ircr1 tate machine tests from ic_icrc1_ledger_sm_tests to ic_ledger_suite_state_machine_tests.

[d70b9eb6f]: Introduces a new allowance endpoint to the ICP ledger canister that retrives the allowance set between two AccountIdentifier's. The function get_allowance handles the retrieval of the allowance while icrc2_allowance and icp_allowance expose this functionality within a test environment guarded by feature flags.

Refactors the Bazel configuration by moving shared canister setup elements to a new file, ledger_canisters.bzl.

Adds tests to ensure that the new allowance endpoint works with two scenarios, endpoint retrieves the expected allowance successfully and endpoints fails when the endpoint is accessed in a production environment.

[1a484503e]: Instead of automatically bumping the time by 1ns before each round, the StateMachine now includes the field time_of_last_round that is used to check if the current time is equal to the time of the last round and only increments the time if that is true. This improves efficiency and consistency of the StateMachine.

[fcbc91f0a]: Updates the version of ic-cdk from 0.13.5 to 0.16.0.

The 32-bit stable memory API has been deprecated with changes and ic-cdk only proivides Rust API that deals with 64-bit stable memory so there was a change in the API usage where before it was named like this: stable64_size() and now stable_size(), since there is no support for 32-bit stable memory there is no longer the need to have a different name. This was done across multiple functions and all pair of 32/64 bit functions were turned into one.

Adds log_visibility and wasm_memory_limit to the type DefiniteCanisterSettings of the cketh_minter, ckbtc_minter and the ledger_suite_orchestrator.

Proposal 133946

Vote: ADOPT

Reason:
Build successful and hashes match, commits look great and found no issues.

[b811de98a]: Adds a ledger_version field in the Ledger struct to track the version of the ledger and allows the post_upgrade method to prevent downgrades to lower ledger versions since this is not supported. The BUILD.bazel file now supports both ledger versions by setting or not a name_suffix and features

[944b8d0c3]: Duplicate commit from proposal 133945

[d70b9eb6f]: Duplicate commit from proposal 133945

Proposal 133947

Vote: ADOPT

Reason:
Build successful and hashes match, commits look great and found no issues.

[944b8d0c3]: Duplicate commit from proposal 133945

[d70b9eb6f]: Duplicate commit from proposal 133945

Proposal 133948

Vote: ADOPT

Reason:
Build successful and hashes match, commits look great and found no issues.

[944b8d0c3]: Duplicate commit from proposal 133945

[d70b9eb6f]: Duplicate commit from proposal 133945

Proposal 133949

Vote: ADOPT

Reason:
Build successful and hashes match, commits look great and found no issues.

[944b8d0c3]: Duplicate commit from proposal 133945

[d70b9eb6f]: Duplicate commit from proposal 133945

[Zane]

Proposal 133945

Vote: ADOPT

Reviews:

61a229ece Added RUST_TEST_THREADS environment variable to set a limit to the number of threads to be used for some icp ledger and index tests.

944b8d0c3 Already validated as part of a previous release. Implemented Storable trait for Allowance<Tokens> so it can be persisted in stable memory.

03dd6ee6d Already validated as part of a previous release. Renamed ic-icrc1-ledger-sm-tests to ic-ledger-suite-state-machine-tests.

d70b9eb6f Added allowance endpoint to the ICP ledger to fetch available allowance and expiration date provided with an account and spender AccountIdentifiers. The endpoint is only available when building the canister with icp-allowance-getter feature enabled, which is only the case for a test ledger. Tests have been added to ensure the endpoint works only with the proper wasm. Sligh refactor of rs/ledger_suite/icp/ledger/BUILD.bazel to reduce code duplication.

1a484503e Change how time is bumped in state machine tests, it is no longer incremented at the start of an execution round but at the end of it and only if the time hasn’t been bumped from the previous round.

fcbc91f0a Already validated as part of a previous release. Bump ic-cdk to 0.16.0 and addressed breaking changes.

Proposal 133946

Vote: ADOPT

Reviews:

b811de98a Added LEDGER_VERSION const to the icp ledger and ledger_version field to Ledger struct. These are used to determine whether a ledger upgrade can go through, a ledger should always be upgradable to a newer version but not the other way around, if a downgrade is detected the process will panic to prevent data loss.

Proposal 133947

Vote: ADOPT

Proposal 133948

Vote: ADOPT

Proposal 133949

Vote: ADOPT