Proposal to elect new release rc--2024-06-05_23-01

Governance
,
1

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 130315.

Here is a summary of the changes since the last release:

Release Notes for release-2024-06-05_23-01-base (d19fa446ab35780b2c6d8b82ea32d808cca558d5)

Changelog since git revision b9a0f18dd5d6019e3241f205de797bca0d9cc3f8

Features:

  • 49bbd8205 Consensus(schnorr): Introduce pool artifact for tSchnorr signature shares
  • 0b9e0985f Consensus(schnorr): Inspect both initial IDkg dealing sets in registry CUP
  • 4242146b8 Consensus(schnorr): Make MasterPublicKeyId in EcdsaKeyTranscript mandatory
  • e6607925e Execution,Message Routing: Implement persistence for CanisterQueue
  • f3adcebc2 Execution,Message Routing: MessagePool persistence
  • b75f397d4 Execution,Runtime: Implement Ic00Method::SchnorrPublicKey behind a flag
  • 5e8cf5fb9 Execution,Runtime: Implement ic00_compute_initial_i_dkg_dealings behind a flag
  • 5017b75f6 Execution,Runtime: Implement sign_with_schnorr management canister API behind a flag
  • 8076976cb Interface(registry): Introduce ChainKeyInitializations to registry CUP
  • 50c5d7567 Node(node-1251): run chrony on setupos
  • 92805e25c Node: add chrony to setup os base image
  • aa96f7321 Runtime: Instrumentation for wasm64 bulk memory ops
  • 8ec0c976a Execution,Runtime: (feat): Implement snapshot loading
  • 149cdfc2c Execution,Runtime: (feat) Delete snapshot when canister is deleted

Bugfixes:

  • 9e7cef791 Execution,Runtime: Fix preserving canister logs after inter-canister call
  • 380809728 Execution,Runtime: Orthogonal persistence: Do not shadow execution state error by persistence check
  • 29125ba9f Networking: gracefully shutdown the ongoing state sync tasks
  • 0b73c9a0f Networking: optimize jaeger settings
  • 42e5cd23b Networking,Message Routing: make Chunk bytes hard to clone

Chores:

  • 568bac66f Consensus: Migrate replica to read chain key config from registry
  • 7fc691209 Consensus(ecdsa): deprecate KeyTranscriptsLayout
  • 60177900f Consensus(schnorr): Cleanup payload fields for non-generalized pre-signatures
  • 088a2f98d Consensus(schnorr): Map valid_keys to type MasterPublicKeyId
  • 6cda1d971 Crypto: fix comment in hash to point fuzzer
  • 74a74a066 Crypto: Update IDKG domain separators relating to internal seeds
  • caed5de4a Crypto: Update the tECDSA/tSchnorr domain separators
  • 47fb6213a Crypto: upgrade some crypto crates and use workspace version
  • f6951cf1a Crypto: upgrade external crates and use workspace version
  • 134a2f1da Execution,Runtime: Update comment about seed used for raw_rand
  • 9ef6d3586 Execution,Runtime: Add speed label for subnet messages
  • 426e9cf2f Interface: Don’t unnecessarily derive Serialize/Deserialize for all types protos
  • 40e72d025 Networking: Use spawn_blocking instead of single threaded executors in http endpoint
  • 4338c8d5d Networking,Boundary Nodes: strip the suffix for the newer versions of http and http-body
  • c8dd8956f Node: Move bare_metal_deploy to dev-tools/
  • 7a591aa27 Node: Organize guestos/etc under misc/
  • ff4f7e3e8 Node: Clean up and organize dockerfiles

Refactoring:

Tests:

  • 2ec7399c0 Consensus(schnorr): Allow creation of generalized pre-signatures in unit tests
  • d7f5f2ca1 Node: Remove old vsock unit test

Other changes:

  • 92f38e41d Consensus,Execution,Runtime,Message Routing(schnorr): Deliver tSchnorr public keys and pre-signatures in batches
  • 84b81de2b IDX,Node: Organize referenced rootfs components
  • 36a81eacc Message Routing,Runtime,Execution: Wasm64: Add support for 64-bit closures
  • 2d9604948 Node: Updating container base images refs [2024-05-31-2319]
  • 1aa116217 Node: Updating container base images refs [2024-05-30-0817]
  • a3bc7d692 Node,Boundary Nodes: Update misc and docs following rootfs rename
  • 26dc1f332 Utopia,Message Routing,NNS,Financial Integrations: upgrade serde crates and use workspace version

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/d19fa446ab35780b2c6d8b82ea32d808cca558d5/gitlab-ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c d19fa446ab35780b2c6d8b82ea32d808cca558d5

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

3 Likes
2

I really appreciate the work process that DFINITY (e.g. @DRE-Team) uses to introduce IC-OS Version Election proposal each week. It helps a lot to have a forum post for each proposal where the community knows when the proposals will occur and has a place to have discussion, especially with DFINITY. The CodeGov team will commonly interact on these forum threads when we have questions and suggestions.

It would be really helpful if there were a similar work process for System Canister Management proposals. I know those are different teams within DFINITY. Who would be a good contact at DFINITY to make a request that the System Canister Management proposal work process align in a similar way to the IC-OS Version Election work process? The two key elements in my opinion are knowing when to expect the proposals (e.g. routine days of the week and/or 1-2 days advance notice) and having a place for community conversation within a searchable forum thread (e.g. tags and/or common forum post titles). The CodeGov team is now reviewing System Canister Management proposals, so it would be very helpful to have a bit more consistency on the System Canister Management proposal work process.

4 Likes
3

Can I ask how DFINITY go about deciding when to unelect replica versions and which versions to unelect? I noticed that this proposal unelects 5 replica versions, none of which are currently running on any subnets (:+1:), but at least one subnet will have no earlier version that it can rollback to (if it encountered a latent bug/incompatibility issue in the version it’s currently running) once this proposal is executed.

Please expand for details

Based on IC-OS election proposal history, there currently appear to be 7 blessed replica versions registered, 5 of which would be unelected by this proposal. I’ve listed these below, ordered by elected date, and crossed out the versions that would be unelected.

  • 9866a6f, elected 2024-05-13 (proposal 129697), UNELECTION PROPOSED, running on 0 subnets
  • 2c4566b, elected 2024-05-13 (proposal 129696), UNELECTION PROPOSED, running on 0 subnets
  • 30bf45e, elected 2024-05-16 (proposal 129706), UNELECTION PROPOSED, running on 0 subnets
  • b6b2ef4, elected 2024-05-20 (proposal 129747), UNELECTION PROPOSED, running on 0 subnets
  • 5ba1412, elected 2024-05-20 (proposal 129746), UNELECTION PROPOSED, running on 0 subnets
  • ec35ebd, elected 2024-05-27 (proposal 130083), running on 1 subnets
  • b9a0f18, elected 2024-06-03 (proposal 130134), running on 36 subnets
Relevant Subnet Version History

I’ve focused on the subnet IC-OS version history of a few of the most important subnets below. The current replica version is in bold, on the left of which are prior deployed versions (crossed out if due to be unelected), and on the right of which are versions that have not yet been deployed to that subnet and are not due to be unelected.

  • tdb26 (system), has been running ec35ebd since 2024-06-03 (4 days):
    • 2c4566b,5ba1412,ec35ebd,b9a0f18
  • uzr34 (system), has been running b9a0f18 since 2024-06-04 (3 days):
    • 2c4566b,30bf45e,5ba1412,ec35ebd,b9a0f18
  • w4rem (system), has been running b9a0f18 since 2024-06-06 (2 days):
    • 9866a6f,b6b2ef4,ec35ebd,b9a0f18
  • x33ed (application), has been running b9a0f18 since 2024-06-06 (1 days):
    • 2c4566b,5ba1412,ec35ebd,b9a0f18
  • pzp6e (fiduciary), has been running b9a0f18 since 2024-06-06 (1 days):
    • 30bf45e,5ba1412,ec35ebd,b9a0f18

In case there’s an unexpected need to rollback to the prior deployed version, it seems sensible to always leave at least one prior deployed version for each subnet remaining in the registry - which isn’t the case for the tdb26 system subnet (the only option would be to roll forward, or await a new IC-OS release if necessary, which seems suboptimal or potentially dangerous).

While this is an unlikely occurrence, I don’t think it’s inconceivable, particularly given that subnets can have different configuration and some may encounter an incompatibility with the IC-OS version (while other subnets don’t). I’m not saying that I expect there to be any issues in this particular case, but I’m explaining why I’m interested in finding out if there is a policy that’s being adhered to when deciding which replica versions to keep in the registry (and what that policy is).

I asked this same sort of question a few weeks ago.

I’m also wondering why unelections are batched. Why not maintain a sliding window of some constant size (number of versions), and always unelect the oldest replica with every new replica election?

Can I also confirm that when ec35ebd gets unelected in a future proposal, we don’t need to worry about this retiring the host OS version that’s also identifiable by this same version hash?

As a side note, it looks like the dashboard is failing to load proposals - e.g. 130315 - if that link doesn’t fail, try refreshing the page - I’m frequently getting An error occurred while loading the proposal.

Thanks in advance :slightly_smiling_face:

1 Like
4

Can I ask how changes to SetupOs can actually take effect via a GuestOS elected binary?

My understanding is that GuestOS versions are deployed subnet-wise via ‘IC OS Version Deployment’ proposals, and HostOS versions are elected separately and deployed node-wise via the same topic (though it used to be the ‘Node Admin’ topic). Given that SetupOS is responsible for booting a replica and installing HostOS and GuestOS, I’m unclear how a change to the SetupOS via an elected GuestOS binary would take effect?

I’m asking on this thread because this proposal includes commits to add a time sync package to SetupOS. Apologies if I’ve misunderstood. Thanks in advance :pray:

2 Likes
5

Hi @Lorimer,

Given that SetupOS is responsible for booting a replica and installing HostOS and GuestOS, I’m unclear how a change to the SetupOS via an elected GuestOS binary would take effect?

You’re right about this: SetupOS installs HostOS which runs a GuestOS VM image.

What might be confusing here is that these release notes are effectively the release notes for all the IC-OS changes and will include changes to setupOS, hostOS and guestOS. What is voted on in the proposal is the version of guestOS (the replica) that can be deployed.

2 Likes
6

Thanks @raymondk, I really appreciate your response. Can I ask under which proposal topic changes to SetupOS would take effect (both in terms of binary election, and deployment)? I’m aware of the process for GuestOS and HostOS, but not SetupOS. Any information or reference material you’re able to point me to would be much appreicated. Thanks again, it’s much appreciated.

1 Like
7

There isn’t a proposal for SetupOS because it is not managed by the NNS. The current SetupOS image is linked to from the dashboard with its hash and is reproducible from the code at the same commit sha that the guestOS was built from.

If a node comes up with an outdated replica version, the NNS will upgrade the replica to the correct version when the node joins the network.

2 Likes
8

Ah, that makes sense, thanks @raymondk! Every day’s a school day :blush:

1 Like
9

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 130409.

Here is a summary of the changes since the last release:

Release Notes for release-2024-06-05_23-01-storage-layer-disabled (08f32722df2f56f1e5c1e603fee0c87c40b77cba)

Changelog since git revision d19fa446ab35780b2c6d8b82ea32d808cca558d5

Bugfixes:

  • 08f32722d Interface: Disable new storage layer

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/08f32722df2f56f1e5c1e603fee0c87c40b77cba/gitlab-ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 08f32722df2f56f1e5c1e603fee0c87c40b77cba

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.