Rejecting Proposals 130707 and 130708: Upgrading ICP archives

General
1

Background:
On Friday, 2024-06-24, the following upgrade proposals were created to upgrade the ICP ledger canister suite:

Issue:
The ICP index proposal was successfully executed and the canister upgraded at 2024-06-24, 13:52:15 UTC. The ICP ledger proposal was executed at 2024-06-24, 14:01:23 UTC. The ledger was then unavailable for 5 minutes, after which it again became available. However, the module hash of the canister did not change, indicating that the upgrade did not succeed, so the ledger is running with the previous code. DFINITY is investigating the issue.

To mitigate the potential risk of upgrading the archives without having first upgraded the ledger, we recommend rejecting the outstanding proposals (130707 and 130708) for upgrading the two ICP archives.

2

Investigation:
During our investigations we saw that the ICP index canister was trying to reach the ledger during the 5 minute time window when the ledger canister was unavailable, but the index received error responses of “canister is stopping”. This was likely due to an open call context from the Cycles Minting Canister to the management canister of the lhg73-sax6z-2zank-6oer2-575lz-zgbxx-ptudx-5korm-fy7we-kh4hl-pqe subnet, which at the time was stalled due to an unrelated incident. This open call context prevented the ICP ledger from stopping, and therefore from upgrading. After a 5 minute timeout, the ledger canister was started again by the NNS root canister, still running the old ledger canister wasm. Once the lhg73 subnet was recovered and started making progress again, it started pulling XNet streams from the NNS subnet to complete the backlog of outstanding calls. Another canister upgrade proposal for the ICP ledger should execute successfully now that the lhg73 subnet is operational again.

The following proposals have now been submitted in place of the unsuccessfully executed ICP ledger upgrade proposal (130706), and the rejected upgrade proposals for the two ICP archives (130707 and 130708):

3 Likes