ICP Ledger Canister Suite Upgrades 2024-10-11

[mathiasb]

The Financial Integrations team has submitted the following proposals today, 2024-10-11 for upgrading the ICP Ledger canister suite. DFINITY plans to vote on these proposals on Monday.

ICP index (proposal 133374):

Proposal to upgrade the ICP index canister

Git hash: 6dcfafb491092704d374317d9a72a7ad2475d7c9

New compressed Wasm hash: 8d02ebb47b413ebeecaf54e5cfbc6bd9000ebf57d164a07d67e1c56473c53aed

Target canister: qhbym-qaaaa-aaaaa-aaafq-cai

Previous ICP index proposal: https://dashboard.internetcomputer.org/proposal/132137

---

Motivation

Add decoding quota to http_request and fix candid comment in get_account_identifier_transactions.

Upgrade args

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
cd rs/ledger_suite/icp/index
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' b0ade55f7e8999e2842fe3f49df163ba224b71a2..6dcfafb491092704d374317d9a72a7ad2475d7c9 -- packages/icrc-ledger_types rs/ledger_suite/icp/index rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core/src rs/ledger_suite/common/ledger_core rs/rust_canisters/http_types rs/rust_canisters/canister_log rs/rosetta-api/icp_ledger/index rs/rosetta-api/icp_ledger/src rs/rosetta-api/ledger_canister_core/src rs/rosetta-api/ledger_core
af9e0c860 test(ICP_ledger): FI-1428: Add ledger, archive, and index block parity checks for ICP golden state test (#1794)
4eca90d6e chore(Rosetta): FI-1512 move rosetta dependencies (#1801)
3bbabefb7 chore(Ledger-Suite): FI-1502 move icp and icrc ledger suites (#1682)
f7791372e chore: remove old hyper and bump prost and tonic versions (#1597)
133a41512 chore(FI): FI-1505: Mark some FI tests as flaky (#1716)
d66fdcb4c chore: bump rust version to 1.81 (#1645)
bdca94716 chore(ICP-Index): fix candid comment for get_account_identifier_transactions  (#1678)
70bc126e8 test(icp_index): FI-1470: Avoid unnecessary calls to the archive in check_block_endpoint_limits (#1370)
644de3a7b refactor: merge the constants from ic_prep and ic_constants (#1329)
4d09678d2 chore: sort rust derive traits (#1241)
4e5d6322b chore: add decoding quota to http_request in NNS canisters (#1060)
f4242cbcf chore: add decoding quota to http_request in NNS root canister (#1031)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
./ci/container/build-ic.sh -c
sha256sum ./artifacts/canisters/ic-icp-index-canister.wasm.gz

ICP ledger (proposal 133375):

Proposal to upgrade the ICP ledger canister

Git hash: 6dcfafb491092704d374317d9a72a7ad2475d7c9

New compressed Wasm hash: 4fe38a91a3130e9d8b39e3413ae3b3f46c40d3fbd507df1b6092f962d970a7ea

Target canister: ryjl3-tyaaa-aaaaa-aaaba-cai

Previous ICP ledger proposal: https://dashboard.internetcomputer.org/proposal/132138

---

Motivation

Add ability to read from memory manager in post_upgrade and add decoding quota to http_request.

Upgrade args

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' b0ade55f7e8999e2842fe3f49df163ba224b71a2..6dcfafb491092704d374317d9a72a7ad2475d7c9 -- packages/icrc-ledger_types rs/ledger_suite/icp/ledger/src rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core/src rs/ledger_suite/common/ledger_core/src rs/rosetta-api/icp_ledger/ledger rs/rosetta-api/icp_ledger/src rs/rosetta-api/ledger_canister_core/src rs/rosetta-api/ledger_core
6dcfafb49 feat(ICP-Ledger): FI-1433: Implement V1 for ICP ledger - add ability to read from memory manager in post_upgrade (#1020)
3bbabefb7 chore(Ledger-Suite): FI-1502 move icp and icrc ledger suites (#1682)
f7791372e chore: remove old hyper and bump prost and tonic versions (#1597)
133a41512 chore(FI): FI-1505: Mark some FI tests as flaky (#1716)
d66fdcb4c chore: bump rust version to 1.81 (#1645)
81ea9eb06 feat(icp-ledger): add bazel target for local replica testing (#1487)
644de3a7b refactor: merge the constants from ic_prep and ic_constants (#1329)
d323465e0 feat(ICRC-Ledger): add ability to read from memory manager in post_upgrade (#746)
4d09678d2 chore: sort rust derive traits (#1241)
4e5d6322b chore: add decoding quota to http_request in NNS canisters (#1060)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
./ci/container/build-ic.sh -c
sha256sum ./artifacts/canisters/ledger-canister_notify-method.wasm.gz

ICP archive1 (proposal 133376):

Proposal to upgrade the ICP archive1 canister

Git hash: 6dcfafb491092704d374317d9a72a7ad2475d7c9

New compressed Wasm hash: 4cc58df55310e28e20ed1a66d14ec9459b3b6a50491d436a938baee1c125e77b

Target canister: qjdve-lqaaa-aaaaa-aaaeq-cai

Previous ICP archive1 proposal: https://dashboard.internetcomputer.org/proposal/132139

---

Motivation

Add decoding quota to http_request.

Upgrade args

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' b0ade55f7e8999e2842fe3f49df163ba224b71a2..6dcfafb491092704d374317d9a72a7ad2475d7c9 -- packages/icrc-ledger_types rs/ledger_suite/icp/archive rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core rs/ledger_suite/common/ledger_core rs/rosetta-api/icp_ledger/archive rs/rosetta-api/icp_ledger/src rs/rosetta-api/ledger_canister_core rs/rosetta-api/ledger_core
3bbabefb7 chore(Ledger-Suite): FI-1502 move icp and icrc ledger suites (#1682)
f7791372e chore: remove old hyper and bump prost and tonic versions (#1597)
d66fdcb4c chore: bump rust version to 1.81 (#1645)
644de3a7b refactor: merge the constants from ic_prep and ic_constants (#1329)
4d09678d2 chore: sort rust derive traits (#1241)
4e5d6322b chore: add decoding quota to http_request in NNS canisters (#1060)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
./ci/container/build-ic.sh -c
sha256sum ./artifacts/canisters/ledger-archive-node-canister.wasm.gz

ICP archive2 (proposal 133377):

Proposal to upgrade the ICP archive2 canister

Git hash: 6dcfafb491092704d374317d9a72a7ad2475d7c9

New compressed Wasm hash: 4cc58df55310e28e20ed1a66d14ec9459b3b6a50491d436a938baee1c125e77b

Target canister: qsgjb-riaaa-aaaaa-aaaga-cai

Previous ICP archive2 proposal: https://dashboard.internetcomputer.org/proposal/132140

---

Motivation

Add decoding quota to http_request.

Upgrade args

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' b0ade55f7e8999e2842fe3f49df163ba224b71a2..6dcfafb491092704d374317d9a72a7ad2475d7c9 -- packages/icrc-ledger_types rs/ledger_suite/icp/archive rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core rs/ledger_suite/common/ledger_core rs/rosetta-api/icp_ledger/archive rs/rosetta-api/icp_ledger/src rs/rosetta-api/ledger_canister_core rs/rosetta-api/ledger_core
3bbabefb7 chore(Ledger-Suite): FI-1502 move icp and icrc ledger suites (#1682)
f7791372e chore: remove old hyper and bump prost and tonic versions (#1597)
d66fdcb4c chore: bump rust version to 1.81 (#1645)
644de3a7b refactor: merge the constants from ic_prep and ic_constants (#1329)
4d09678d2 chore: sort rust derive traits (#1241)
4e5d6322b chore: add decoding quota to http_request in NNS canisters (#1060)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
./ci/container/build-ic.sh -c
sha256sum ./artifacts/canisters/ledger-archive-node-canister.wasm.gz

ICP archive3 (proposal 133378):

Proposal to upgrade the ICP archive3 canister

Git hash: 6dcfafb491092704d374317d9a72a7ad2475d7c9

New compressed Wasm hash: 4cc58df55310e28e20ed1a66d14ec9459b3b6a50491d436a938baee1c125e77b

Target canister: q4eej-kyaaa-aaaaa-aaaha-cai

Previous ICP archive3 proposal: None

---

Motivation

Add decoding quota to http_request.

Upgrade args

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
cd rs/ledger_suite/icp
didc encode '()' | xxd -r -p | sha256sum

Release Notes

git log --format='%C(auto) %h %s' b0ade55f7e8999e2842fe3f49df163ba224b71a2..6dcfafb491092704d374317d9a72a7ad2475d7c9 -- packages/icrc-ledger_types rs/ledger_suite/icp/archive rs/ledger_suite/icp/src rs/ledger_suite/common/ledger_canister_core rs/ledger_suite/common/ledger_core rs/rosetta-api/icp_ledger/archive rs/rosetta-api/icp_ledger/src rs/rosetta-api/ledger_canister_core rs/rosetta-api/ledger_core
3bbabefb7 chore(Ledger-Suite): FI-1502 move icp and icrc ledger suites (#1682)
f7791372e chore: remove old hyper and bump prost and tonic versions (#1597)
d66fdcb4c chore: bump rust version to 1.81 (#1645)
644de3a7b refactor: merge the constants from ic_prep and ic_constants (#1329)
4d09678d2 chore: sort rust derive traits (#1241)
4e5d6322b chore: add decoding quota to http_request in NNS canisters (#1060)

Wasm Verification

Verify that the hash of the gzipped WASM matches the proposed hash.

git fetch
git checkout 6dcfafb491092704d374317d9a72a7ad2475d7c9
./ci/container/build-ic.sh -c
sha256sum ./artifacts/canisters/ledger-archive-node-canister.wasm.gz

[0rions]

Here’s my review: Upgrade Ledger Suite 11/10/2024

[cyberowl]

proposal - 133374

Vote: ADOPT

Hash: MATCH

[af9e0c860]
Tests are verifying the synchronization and block consistency between the ledger, archive, and index canisters by simulating canister upgrades and checking that all blocks are properly synced across them.

[4eca90d6e]
Restructuring the rosetta-api by moving its components under a new icp directory, with corresponding updates to file paths, Bazel build rules, and dependencies to reflect this reorganization. Test utilities and configurations have been adjusted to align with this new directory structure

[3bbabefb7]
Moves components related to ledger and icrc1 functionalities from the rosetta-api directory to a new ledger_suite directory. Build scripts and paths have been updated accordingly to reflect this restructuring, ensuring compatibility with the new organization.

[f7791372e]
Upgrade the prost and tonic libraries to their latest versions (prost from 0.12.2 to 0.13.2 and tonic from 0.11.0 to 0.12.2), alongside corresponding updates to build files and dependencies

[133a41512]
Mark some tests as flaky

[d66fdcb4c]
Update the container image across various CI workflow files to a newer version (from sha256:d883e7f6e5a355d63f6d8a294cfbeb47161bc27055e34730c21c6eeae0617acb to sha256:115daa5ad5149182bb0416cbe5730f305be3bb2f48df576bc2c23067eefce84b)

Replace map functions with inspect for improved error handling in the codebase

[bdca94716]
Fix comments

[70bc126e8]
Fix test check_block_endpoint_limits by limiting calls to archive

[644de3a7b]
Replace the usage of the ic-constants crate across the codebase with a new ic-limits crate

[4d09678d2]
reorganizing the ordering of derived traits such as Debug, PartialEq, Eq, and Clone.

[4e5d6322b]
tests for HTTP request handling, specifically to ensure that canisters reject overly large HTTP requests that exceed decoding limits.
Add ic-canisters-http-types and serde_bytes

[f4242cbcf]
decoding quota for HttpRequest types, ensuring large requests exceeding 10KB are rejected.
test cases were added to validate this new behavior, and the dfn_candid crate was updated to support configurable decoding quotas in various canister endpoints

proposal - 133375

Vote: ADOPT

Hash: MATCH

[6dcfafb49]
upgrade-to-memory-manager feature along with the associated dependencies such as ic-stable-structures and ic-cdk.

[81ea9eb06]
dfx_deploy.sh script to automate the deployment of the ICP Ledger canister to a local replica and updates the Bazel build system to support this process by including relevant files

[d323465e0]
addition of the ic-agent, ic-stable-structures, and ic-icrc1-test-utils as dependencies across multiple files, and the introduction of new tests such as test_upgrade_serialization.

[644de3a7b]
DUPLICATE
[3bbabefb7]
DUPLICATE
[f7791372e]
DUPLICATE
[133a41512]
DUPLICATE
[d66fdcb4c]
DUPLICATE
[4d09678d2]
DUPLICATE
[4e5d6322b]
DUPLICATE

proposal - 133376

Vote: ADOPT

Hash: MATCH

All DUPLICATE

proposal - 133377

Vote: ADOPT

Hash: MATCH

All DUPLICATE

proposal - 133378

Vote: ADOPT

Hash: MATCH

All DUPLICATE

[timk11]

Hashes and args all match.

Proposal 133374

Vote: Adopt

[af9e0c860] Adds a test to rs/ledger_suite/icp/tests/golden_nns_state.rs to check that index blocks match ledger and archive blocks, + helper functions.

[4eca90d6e] Creates new subdirectories within rs/rosetta/ and moves code accordingly.

[3bbabefb7] Adds a new directory rs/ledger_suite/ and moves ICP and ICRC1 ledger suites from rs/rosetta-api/ into their respective subdirectories within the new directory.

[f7791372e] Version change for prost, tonic, hyper and related packages plus associated code changes.

[133a41512] Adds flaky = True flag to some tests.

[d66fdcb4c] Rust version change and associated code changes.

[bdca94716] Comment change in Candid file.

[70bc126e8] Modifies check_block_endpoint_limits test by adding trigger_threshold and num_blocks_to_archive in order to reduce the occurrence of timeouts.

[644de3a7b] Renames ‎rs/constants/ to rs/limits/ and changes dependencies and moves some code accordingly.

[4d09678d2] Re-orders traits within #[derive(…)] attribute lines within numerous files.

[4e5d6322b] [f4242cbcf] Specifies #[query(decoding_quota = 10000)] for fn http_request in rs/rosetta-api/icp_ledger/index/src/main.rs in order to reject excessively large HTTP requests early. + background infrastructure and associated tests.

Proposal 133375

Vote: Adopt

[6dcfafb49] [d323465e0] Enables ICP ledger and ICRC-1 ledger respectively to read from memory manager (using ..._nextmigrationversionmemorymanager.wasm) during post-upgrade.

[81ea9eb06] Adds a bazel target to simplify building and testing a local replica of the ICP ledger.

Other commits have already been reviewed in 133374.

Proposal 133376

Vote: Adopt

Proposal 133377

Vote: Adopt

Proposal 133378

Vote: Adopt

These proposals upgrade the ICP archive1, archive2 and archive3 canisters respectively. The proposal details and commits are identical otherwise. The commits have all been reviewed in the previous two proposals.

[ZackDS]

Proposal 133374 - ICP Index

Hash and arguments match as do the commits, voted to adopt.
…

Proposal 133375 - ICP Ledger

Hash and arguments match as do the commits, voted to adopt.
…

Proposal 133376 - ICP Archive1

Proposal 133377 - ICP Archive2

Proposal 133378 - ICP Archive3

All three are the same, voted to adopt .

[Zane]

Proposal 133374

Hashes and args match. Some of the commits have already been validated in previous releases, no major changes are introduced by newer ones.
Adopted

Proposal 133375

Hashes and args match. Adopted.

[6dcfafb49], [d323465e0] Added a memory manager to both ICP and ICRC1 ledgers and allocate a memory ID to be used to persist data during canister upgrades.

Proposal 133376

Hashes and args match. Adopted.

Proposal 133377

Hashes and args match. Adopted.

Proposal 133378

Hashes and args match. Adopted.

[LaCosta]

Proposal 133374

Vote: ADOPT

Build successful and hashes match.

[af9e0c860b]: Adds a test to rs/ledger_suite/icp/tests/golden_nns_state.rs to verify that the blocksretrieved from the ledger and archives match the blocks retrieved from the index.

[4eca90d6ec]: Moves code in the rs/rosetta-api directory

[3bbabefb70]: Moves icp and icrc1 ledger suites from the rosetta-api/ subdirectory into the ledger_suite/ subdirectory as part of a restructure of the rosetta-api/.

[f7791372e9]: Removes hyper and bumps prost and tonic versions

[133a415129]: Adds the flag flaky = True to the following tests: icp_rosetta_system_tests, ic_icp_index_test and ledger_canister_test

[d66fdcb4c7]: Bumps Rust version to 1.81 and changes code accordingly

[bdca94716f]: Changes a comment for the get_account_identifier_transactions

[70bc126e83]: Adds a trigger_threshold and num_blocks_to_archive to the ArchivingOptions of the icp_ledger to reduce timeouts on tests

[644de3a7bd]: Re-order dependencies and renames ic_constants to ic_limits

[4d09678d23]: Sorts Rust #[derive(...)] traits to maintain a consistent style

[f4242cbcf8], [4e5d6322bb]: Adds a decoding quota for the http_request endpoint of the NNS root canister and a new test to assert the behaviour. The new trait HasCandidDecoderConfig allows to set own quotas.

Proposal 133375

Vote: ADOPT

[3bbabefb70]: Moves icp and icrc1 ledger suites from the rosetta-api/ subdirectory into the ledger_suite/ subdirectory as part of a restructure of the rosetta-api/.

[f7791372e9]: Removes hyper and bumps prost and tonic versions

[133a415129]: Adds the flag flaky = True to the following tests: icp_rosetta_system_tests, ic_icp_index_test and ledger_canister_test

[d66fdcb4c7]: Bumps Rust version to 1.81 and changes code accordingly

[81ea9eb06]: Introduces a Bazel target for deploying the ICP ledger canister to a local ic replica with a single command: bazel run //rs/rosetta-api/icp_ledger/ledger:dfx_deploy

[644de3a7bd]: Re-order dependencies and renames ic_constants to ic_limits

[d323465e0], [6dcfafb49]: Enables the icp and icrc1 ledgers to read from both the plain stable memory and the memory managed by the memory manager during the post_upgrade process

[4d09678d23]: Sorts Rust #[derive(...)] traits to maintain a consistent style

Build successful and hashes match.

Proposal 133376

Vote: ADOPT

Build successful and hashes match.

All commits reviewed previously.

Proposal 133377

Vote: ADOPT

Build successful and hashes match.

All commits reviewed previously.

Proposal 133378

Vote: ADOPT

Build successful and hashes match.

All commits reviewed previously.