Proposal to elect new release rc--2024-11-28_03-15

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 134336.

Here is a summary of the changes since the last release:

Release Notes for release-2024-11-28_03-15-base (2d8611eb4efa8e69c4dd567546c1c353a545e0a6)

This release is based on changes since release-2024-11-21_03-11-base (a3478d571670053ebd09b9771bec489ef69425fa).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

This release diverges from the latest release. Merge base is 0dc55e08ab70c0f561e3617a3d6f41c58d7b2b38. Changes were removed from this release.

Features:

• 925acaec1 Consensus,Interface(orchestrator): Call sync and fstrim during node unassignment (#2720)
• e3540ad62 Consensus,Interface: Introduce new context type for VetKD requests (#2629)
• fd4d35f4a Consensus,Interface,Node: SOCKS proxy on the API BNs (#2191)
• fd6561a04 Crypto,Interface(crypto): Add sig support to secp256k1 utility crate (#2756)
• 3a9f843de Execution,Interface: Implement Ic00Method::VetKdPublicKey (#2693)
• 8bf9965c2 Execution,Interface: Store compiled Wasms on disk (#2673)
• 6a725785d Execution,Interface: Evict sandbox cache of all idle candidates (#2744)
• 003c70ac1 Execution,Interface: Evict sandboxes based on the available memory (#2651)
• 13006ee4d Execution,Interface,Message Routing: Time out messages in subnet queues (#2708)
• 7d8134765 Interface(ckbtc): re-evaluate discarded UTXOs (#2674)
• f68da752b Interface(ICRC-Rosetta): updated rosetta to support icrc3 standard (#2607)
• e3ee0072c Interface,Message Routing: Unify checkpointed_state and state in commit_and_certify (#2827)
• 90c56a39f Interface,Networking: enable the hashes-in-blocks feature everywhere (#2854)
• 260f1cc09 Interface,Node: Upgrade build container to 24.04 (#1946)
• 245e13ebd Interface,Node: Replace mac_address with deterministic_ips (#2757)
• e9f61b877 Interface,Node(IDX): don’t rely on bazel cache for large test deps (#2752)
• a0be7baf6 Node: Allow local network to access metrics through firewall (#2703)

Bugfixes:

• 052b85232 Consensus,Interface(node): onboarding log typo (#2869)
• a55ac1812 Consensus,Interface(recovery): Clear recovery directory instead of deleting it (#2823)
• c1d7c5ec6 Execution,Interface,Message Routing: Drop duplicate responses in induct_messages_to_self (#2725)
• e7ce87bf2 Interface,Networking: remove the app handshake (#2790)
• e9a0f1894 Interface,Networking(quic-transport): Correct the error code sent for reset stream frames. (#2761)
• 8869fff65 Interface,Node: Revert "feat: Replace mac_address with deterministic_ips (#2785)
• 0210728ba Node: Depend on //:bazel-timestamp instead of //:.git/index when calculating commit_timestamp_txt (#2765)

Chores:

• cdf85cbaf Consensus,Interface(node): update onboarding log (#2830)
• 32cf9b0f8 Consensus,Interface(consensus): remove unnecessary variable in ingress selector (#2683)
• bef38d077 Crypto,Interface: upgrade thiserror and anyhow (#2732)
• 5f4e13e98 Execution,Interface(EXC): Remove IPC calls for compilation in canister sandbox (#2853)
• 9d768e5d8 Execution,Interface: Make request metadata non optional (#2622)
• e636d3f34 Execution,Interface: Drop redundant CanisterQueuesTesting::output_message_count() (#2737)
• ac6341599 Interface: upgrade crates (#2788)
• 385cf07ae Interface,Message Routing(fuzzing): Allow no_op_logger in StateMachine (#2736)
• 0cdfbb49f Interface,Networking: add a logging layer for tracing (#2650)
• bbae0dd15 Interface,Networking: upgrade crates (#2742)
• 647fec3b1 Node: bump ic-bn-lib & ic-gateway (#2767)
• 27e7499a7 Node: Add docs on nightly benchmarking (#2755)
• acdbe1f72 Node: Update Base Image Refs [2024-11-21-0808] (#2731)
• 2202c3f93 Node(IDX): clarify launch-remote-vm script (#2745)

Refactoring:

• ccd5e2921 Consensus,Interface: Deprecate pb::TaggedNiDkgTranscript and implement new deserialization (#2838)
• cf2727ce9 Consensus,Interface(CON): Rename dkg::Dealings to dkg::DataPayload (#2793)
• 7d5f65756 Crypto,Interface(crypto): rename VetKdArgs::encryption_key to encryption_public_key (#2789)
• 2f4c1fca1 Interface,Node: Replace mac_address with deterministic_ips (#2810)

Tests:

• e6f76957a Execution,Interface: Precompile universal canister (#2816)
• ac64a1d95 Execution,Interface: Move long execution tests (#2814)
• 627fae2f7 Execution,Interface,Message Routing: Nicer SystemState tests (#2748)
• e16e7f255 Interface,Networking(quic-transport): Add assertion for reset reason in drop handle test (#2764)

Documentation:

• 7fb898bdf Execution,Interface,Message Routing: Improve documentation around message and callback expiry (#2763)
• ef0c2e44c Interface,Networking: improve error names and track other errors via metrics (#2797)
• b2a094f7e Interface,Networking: improve some comments and remove infallible condition from metrics (#2773)

---

Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/2d8611eb4efa8e69c4dd567546c1c353a545e0a6/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 2d8611eb4efa8e69c4dd567546c1c353a545e0a6 --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 134337.

Here is a summary of the changes since the last release:

Release Notes for release-2024-11-28_03-15-revert-hashes-in-blocks (f9085dbe8d86dcca77b05adaee93d608498dae0b)

This release is based on changes since release-2024-11-28_03-15-base (2d8611eb4efa8e69c4dd567546c1c353a545e0a6).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

• f9085dbe8 Interface,Networking: Revert “feat: enable the hashes-in-blocks feature everywhere (#2854)”

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/f9085dbe8d86dcca77b05adaee93d608498dae0b/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c f9085dbe8d86dcca77b05adaee93d608498dae0b --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[sat]

This is intended to be used only as a fallback, if we need to disable the feature for some reason.

[hpeebles]

Proposal 134336: Adopt

I have successfully run the build verification script and in my opinion all of the commits look fine.

Features:

• 925acaec1 Consensus,Interface(orchestrator): Call sync and fstrim during node unassignment (#2720)
  Review: Looks fine + matches description
  Notes: Runs a clean-up script whenever a node is unassigned to free up the memory that was previously being used plus records failures to new metric counters.

• e3540ad62 Consensus,Interface: Introduce new context type for VetKD requests (#2629)
  Review: Looks fine + matches description
  Notes: Adds VetKd to the ThresholdArguments enum, then updates a few places which were using SignWithThresholdContext to instead use the new IDkgSignWithThresholdContext which doesn’t include VetKd.

• fd4d35f4a Consensus,Interface,Node: SOCKS proxy on the API BNs (#2191)
  Review: Looks fine + matches description
  Notes: Sets up Dante and uses it to run a SOCKs proxy on the API boundary nodes, also opens up the firewall to allow only system subnets to make IPv4 requests via the proxy.

• fd6561a04 Crypto,Interface(crypto): Add sig support to secp256k1 utility crate (#2756)
  Review: Looks fine + matches description
  Notes: Adds sign_message_with_bip341 to the secp256k1 package.

• 3a9f843de Execution,Interface: Implement Ic00Method::VetKdPublicKey (#2693)
  Review: Looks fine + matches description
  Notes: Implements Ic00Method::VetKdPublicKey to calculate the VetKd public key from the master public key, the canisterId and the derivation path.

• 8bf9965c2 Execution,Interface: Store compiled Wasms on disk (#2673)
  Review: Looks fine + matches description
  Notes: Modifies the CompilationCache to support either storing in memory or on disk, this will allow the memory usage to be reduced which still having fast access to compiled wasms. For now the compilation cache is only constructed using the in memory version, the on disk version will follow in later commits.

• 6a725785d Execution,Interface: Evict sandbox cache of all idle candidates (#2744)
  Review: Looks fine + matches description
  Notes: Evicts the sandbox processes of all idle canisters regardless of the total memory usage or how many sandboxes there are in total. Evicting them early reduces overhead to monitor the cache state.

• 003c70ac1 Execution,Interface: Evict sandboxes based on the available memory (#2651)
  Review: Looks fine + matches description
  Notes: Avoids evicting canister sandbox processes based on the total sandbox RSS unless the available subnet memory is below a threshold (currently 250GB).

• 13006ee4d Execution,Interface,Message Routing: Time out messages in subnet queues (#2708)
  Review: Looks fine + matches description
  Notes: When timing out messages, additionally times out messages in the subnet_queues (the management canister queues), since this can contain output best-effort responses which should be dropped if expired.

• 7d8134765 Interface(ckbtc): re-evaluate discarded UTXOs (#2674)
  Review: Looks fine + matches description
  Notes: Modifies the ckBTC minter allowing it to re-evaluate UTXOs that were previously dropped due to being too small or due to being tainted, this is because the rules have changed so UTXOs which may have been ignored before may now be valid.

• f68da752b Interface(ICRC-Rosetta): updated rosetta to support icrc3 standard (#2607)
  Review: Looks fine + matches description
  Notes: Updates the ICRC ledger to return a certificate which is compatible with the ICRC-3 spec. This is currently hidden behind the icrc3-compatible-data-certificate feature flag.

• e3ee0072c Interface,Message Routing: Unify checkpointed_state and state in commit_and_certify (#2827)
  Review: Looks fine + matches description
  Notes: During checkpointing a second replicated state instance is created by loading from disk. This commit makes it such that the 2nd instance is dropped as soon as the 2 instances are validated to be equal. By consolidating into a single instance there is no risk of divergence.

• 90c56a39f Interface,Networking: enable the hashes-in-blocks feature everywhere (#2854)
  Review: Looks fine + matches description
  Notes: Sets the HASHES_IN_BLOCKS_FEATURE_ENABLED feature flag to true so that it is enabled everywhere.

• 260f1cc09 Interface,Node: Upgrade build container to 24.04 (#1946)
  Review: Looks fine + matches description
  Notes: Updates the Ubuntu images to version 24.04.

• 245e13ebd Interface,Node: Replace mac_address with deterministic_ips (#2757)
  Review: Looks fine + matches description
  Notes: Removes the mac_address package and switches usages over to the deterministic_ips package which provides the same functionality for determining IP addresses and parsing strings into IP addresses.

• e9f61b877 Interface,Node(IDX): don’t rely on bazel cache for large test deps (#2752)
  Review: Looks fine + matches description
  Notes: Modifies the system tests runner to take a url for each IC-OS image required so that the image may be downloaded from any remote source rather than requiring it to be in the Bazel cache.

• a0be7baf6 Node: Allow local network to access metrics through firewall (#2703)
  Review: I don’t know enough to say for sure that this is doing what it says it is doing, but it is definitely only updating firewall rules, and it doesn’t look suspicious in any way, so I’m going to go ahead and say this is fine.

Bugfixes:

• 052b85232 Consensus,Interface(node): onboarding log typo (#2869)
  Review: Looks fine + matches description
  Notes: Simply fixes a typo, “ONBOADING” → “ONBOARDING”.

• a55ac1812 Consensus,Interface(recovery): Clear recovery directory instead of deleting it (#2823)
  Review: Looks fine + matches description
  Notes: During the clean-up phase, make ic-recovery clear all files in the recovery directory rather than deleting the directory.

• c1d7c5ec6 Execution,Interface,Message Routing: Drop duplicate responses in induct_messages_to_self (#2725)
  Review: Looks fine + matches description
  Notes: When inducting messages, first checks should_enqueue_input and drops the message if that returns false, which happens for duplicate messages or best-effort messages where the callback has gone.

• e7ce87bf2 Interface,Networking: remove the app handshake (#2790)
  Review: Looks fine + matches description
  Notes: Skips the “Gruezi handshake” step when setting up a connection between 2 nodes which was adding unnecessary latency.

• e9a0f1894 Interface,Networking(quic-transport): Correct the error code sent for reset stream frames. (#2761)
  Review: Looks fine + matches description
  Notes: Fixes the underlying int value of the QUIC_STREAM_CANCELLED error code (6 → 0x80000006).

• 8869fff65 Interface,Node: Revert "feat: Replace mac_address with deterministic_ips (#2785)
  Review: Looks fine + matches description
  Notes: This commit simply reverts 245e13ebd since it introduced some issues in tests.

• 0210728ba Node: Depend on //:bazel-timestamp instead of //:.git/index when calculating commit_timestamp_txt (#2765)
  Review: Looks fine + matches description
  Notes: Previously the Bazel build was depending on .git/index so that the commit timestamp is regenerated each time a new branch is checked out, but that is now switched to instead depend on bazel-timestamp so that the commit timestamp is not cached.

Chores:

• cdf85cbaf Consensus,Interface(node): update onboarding log (#2830)
  Review: Looks fine + matches description
  Notes: Updates an orchestrator log message to highlight the fact that onboarding is not yet complete.

• 32cf9b0f8 Consensus,Interface(consensus): remove unnecessary variable in ingress selector (#2683)
  Review: Looks fine + matches description
  Notes: Removes the num_messages variable because this value can be determined by simply calling messages_in_payload.len().

• bef38d077 Crypto,Interface: upgrade thiserror and anyhow (#2732)
  Review: Looks fine + matches description
  Notes: Bumps thiserror to 2.0.3 and anyhow to 1.0.93.

• 5f4e13e98 Execution,Interface(EXC): Remove IPC calls for compilation in canister sandbox (#2853)
  Review: Looks fine + matches description
  Notes: Removes open_wasm from the SandboxManager which would make an inter-process request to perform the wasm compilation, and instead calls into wasm_utils::compile_wasm followed by the pre-existing open_wasm_serialized, because this compilation already takes place in a separate sandbox process.

• 9d768e5d8 Execution,Interface: Make request metadata non optional (#2622)
  Review: Looks fine + matches description
  Notes: Makes the Request::metadata non-optional now that all instances still in existence will have the field populated.

• e636d3f34 Execution,Interface: Drop redundant CanisterQueuesTesting::output_message_count() (#2737)
  Review: Looks fine + matches description
  Notes: Drops CanisterQueuesTesting::output_message_count() in favour of CanisterQueues::output_queues_message_count() which returns the exact same value.

• ac6341599 Interface: upgrade crates (#2788)
  Review: Looks fine + matches description
  Notes: Bumps the versions of a few dependencies (axum, hyper, tower, rustls, …).

• 385cf07ae Interface,Message Routing(fuzzing): Allow no_op_logger in StateMachine (#2736)
  Review: Looks fine + matches description
  Notes: Previously, if no log level was passed to the state machine tests it would fallback to Level::Warning, this change makes it instead default to using a no-op logger so that logs don’t get bloated during long fuzz tests.

• 0cdfbb49f Interface,Networking: add a logging layer for tracing (#2650)
  Review: Looks fine + matches description
  Notes: Adds the ic-tracing-logging-layer package which adds a logging layer to the replica which sets up tracing based on the config passed in (eg. log to stdout with min level INFO).

• bbae0dd15 Interface,Networking: upgrade crates (#2742)
  Review: Looks fine + matches description
  Notes: Updates a few more dependencies.

• 647fec3b1 Node: bump ic-bn-lib & ic-gateway (#2767)
  Review: Looks fine + matches description
  Notes: Updates ic-gateway to 0.1.60 and updates the ic-bn-lib git dependency to a later commit.

• 27e7499a7 Node: Add docs on nightly benchmarking (#2755)
  Review: Looks fine + matches description
  Notes: Adds more details to the benchmarking docs.

• acdbe1f72 Node: Update Base Image Refs [2024-11-21-0808] (#2731)
  Review: Looks fine + matches description
  Notes: Updates the base IC-OS image references.

• 2202c3f93 Node(IDX): clarify launch-remote-vm script (#2745)
  Review: Looks fine + matches description
  Notes: Simplifies part of the VM dev tools by splitting some of the logic out into the launch-remote-vm.sh script.

Refactoring:

• ccd5e2921 Consensus,Interface: Deprecate pb::TaggedNiDkgTranscript and implement new deserialization (#2838)
  Review: Looks fine + matches description
  Notes: Deprecates TaggedNiDkgTranscript in favour of NiDkgTranscript and implements deserialization from the old fields to the new ones.

• cf2727ce9 Consensus,Interface(CON): Rename dkg::Dealings to dkg::DataPayload (#2793)
  Review: Looks fine + matches description
  Notes: Renames the dkg::Dealings struct to dkg::DataPayload plus also renames a few fields.

• 7d5f65756 Crypto,Interface(crypto): rename VetKdArgs::encryption_key to encryption_public_key (#2789)
  Review: Looks fine + matches description
  Notes: Renames VetKdArgs::encryption_key to VetKdArgs::encryption_public_key.

• 2f4c1fca1 Interface,Node: Replace mac_address with deterministic_ips (#2810)
  Review: Looks fine + matches description
  Notes: This is almost the same as commit 245e13ebd which was reverted due to a MAC generation bug being picked up by tests, but this time with a fix in place for the bug.

Tests:

• e6f76957a Execution,Interface: Precompile universal canister (#2816)
  Review: Looks fine + matches description
  Notes: Adds the with_precompiled_universal_canister flag to the execution environment test builder and implements its usage such that when it is set to true the universal canister wasm is compiled and inserted into the compilation cache so that the compilation is only done once rather than by lots of threads in parallel.

• ac64a1d95 Execution,Interface: Move long execution tests (#2814)
  Review: Looks fine + matches description
  Notes: This is fine because it just shuffles some tests around and splits the some_ic0_calls_fail_if_called_with_huge_size into multiple tests.

• 627fae2f7 Execution,Interface,Message Routing: Nicer SystemState tests (#2748)
  Review: Looks fine + matches description
  Notes: Just cleans up some system state tests.

• e16e7f255 Interface,Networking(quic-transport): Add assertion for reset reason in drop handle test (#2764)
  Review: Looks fine + matches description
  Notes: Adds a check within an assertion to make sure the error code is the newly updated value (0x80000006 rather than 6).

Documentation:

• 7fb898bdf Execution,Interface,Message Routing: Improve documentation around message and callback expiry (#2763)
  Review: Looks fine + matches description
  Notes: Updats some CanisterQueues doc comments.

• ef0c2e44c Interface,Networking: improve error names and track other errors via metrics (#2797)
  Review: Looks fine + matches description
  Notes: Updates some doc comments, increments the connection_handle_errors_total in case of a failure, and renames PeerIdMismatch to InvalidIncomingPeerId.

• b2a094f7e Interface,Networking: improve some comments and remove infallible condition from metrics (#2773)
  Review: Looks fine + matches description
  Notes: Updates some comments on the Quic transport metrics + removes a label which was never used.

Proposal 134337: Adopt

I have successfully run the build script and the only additional commit disables the HASHES_IN_BLOCKS_FEATURE_ENABLED feature flag as a safeguard in case anything goes wrong in the next release in which the feature will be enabled everywhere.

[ilbert]

Proposal 134336

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
For the Execution and Runtime layers, the main changes introduced in this proposal are:

• Continues the implementation of vetKeys by implementing the management canister’s VetKdPublicKey method
• Optimizes canister sandboxes memory usage

I’ve reviewed all the commits listed in the proposal, for the Execution and Runtime layers:

3a9f843de:
Adds the derive_vetkd_public_key function to the canister_threshold_sig utils, which takes a public key and the derivation path made of the caller principal and the derivation path. Returns a derived public key obtained from the DerivedPublicKey::compute_derived_key method. It also checks that the public key passed as input is valid by checking that its algorithm is ThresBls12_381 and that it can be deserialized to G2Affine.
Adds the get_vetkd_public_key method to the ExecutionEnvironment struct. This method calls the derive_vetkd_public_key function and is called in the execute_subnet_message function, in case the input message method is Ic00Method::VetKdPublicKey.
Renames variables from idkg_* prefix to chain_key_* in the execution environment. A similar renaming was already done in the previous release for the routing functions inside the system_api.

8bf9965c2:
Turns the CompilationCache struct into an enum, in order to keep track of the type of storage used for the cache. Splits the insert method of the enum into insert_err and insert_ok to simplify access to the variants of the enum. The insert_ok method now returns the new StoredCompilation enum, which contains the serialized module based on the type of memory chosen for the storage. If the StoredCompilation is of type Disk, it uses the newly introduced OnDiskSerializedModule struct, which stores all the data of the initial state inside a file along with the compiled bytes in another file. Right now, the disk storage is still not used and will be used from the next releases.

6a725785d:
Changes the loop that cycles through the candidates array passed to the evict function. Now, candidates are added to the evicted array only after having checked that the minimum amount of candidates has been evicted and that we have reached the first non-idle canister. This way, the canisters that are idle are inserted in the evicted array and hence removed from the sandbox processes.
It also removes the check on the active sandboxes and total sandboxes’ RSS before calling the evict function in the evict_sandbox_processes. This way, the evict function will cycle through all the candidates and eventually add the idle processes to the list to evict.

003c70ac1:
These changes were already introduced in the previous release.

13006ee4d:
Changes the time_out_messages method of the ReplicatedState struct to check if the subnet_queues have expired deadlines and time out messages in those queues if the condition is met.

c1d7c5ec6:
Changes the logic inside the induct_messages_to_self method of the SystemState struct to use the should_enqueue_input to check if the message can be inducted into the input queue. The function stops the execution if the there was an error with the check, and stops iterating through the output queue if there is a best-effort message without a callback. In the latter case, it also removes the output message from the output queue using the newly introduced pop_canister_output method of the CanisterQueues struct. This method removes the next message in the output queue that was directed to the dst_cansiter passed as input.

5f4e13e98:
Removes the OpenWasm and CreateExecutionState variants from the Request enum and their inner data structs. Removes the open_wasm and create_execution_state methods from the SandboxManager struct and the SandboxService trait.

9d768e5d8:
Changes the metadata field of the Request struct inside the inter_canister module to be non optional. Same for the RequestMetadataStats struct.

e636d3f34:
Matches description.

e6f76957a:
Matches description.

ac64a1d95:
Matches description.

627fae2f7:
Matches description.

7fb898bdf:
Matches description.

---

Proposal 134337

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
This release is the same as the one in 134336 with the addition of the commit f9085dbe8, which matches its description.

[zenithcode]

Proposal: 134336

Summary:

1. Build Hash: Build has from the proposal, local build and CDN matches and is “b7482882b643da341027cf782a3e9cbb933668988f73f9ef19d2140aa0334305”.
2. Summary: The release notes matches the code changes
3. Vote: I vote to adopt the proposals
   
   

Detailed Review:

Features:

• 925acaec1 Consensus,Interface(orchestrator): Call sync and fstrim during node unassignment (#2720)
  Notes: sync and fstrim is called on the replica’s data directory from a separate shell script with root access when the node is unassigned. This is done to discard the blocks not used by the filesystem. In addition there are some changes around collecting metrics for failed sync and fstrim calls.
  Review: Code changes look good and match release notes.
• e3540ad62 Consensus,Interface: Introduce new context type for VetKD requests (#2629)
  Notes: Another variant ThresholdArguments of type VetKdArguments has been added to existing SignWithThreshholdContext. To separately handle tSchnorr and tEcdsa in consensus, a new type is also added IDkgSignWithThresholdContext which wraps the reference to SignWithThresholdContext .
  Review: Code changes look good and match release notes.
• fd4d35f4a Consensus,Interface,Node: SOCKS proxy on the API BNs (#2191)
  Notes: Code changes has been done to API boundary node to have dante socks server in the base image. ALso firewall port 1080 will now be opened when the node is API boundary node to make ipv4 calls via proxy, and only opened for nodes in the subnet.
  Review: Code changes look good and match release notes.
• fd6561a04 Crypto,Interface(crypto): Add sig support to secp256k1 utility crate (#2756)
  Notes: Function sign_message_with_bip341 has been added to sign a message with BIP340 with Taproot derivation to the utility crate.
  Review: Code changes look good and match release notes.
• 3a9f843de Execution,Interface: Implement Ic00Method::VetKdPublicKey (#2693)
  Notes: The PR implements VetKdPublicKey in the management canister. Also have some minor renames idkg_subnet_keys to chain_key_subnets_keys.
  Review: Code changes look good and match release notes.
• 8bf9965c2 Execution,Interface: Store compiled Wasms on disk (#2673)
  Notes: To reduce the sandbox memory usage, now the compiled wasm modules will be stores on the disk itself in temporary files. For this modifications have been made to CompilationCache. This is disabled as well for now, as it will involve some follow up work in the future proposals.
  Review: Code changes look good and match release notes.
• 6a725785d Execution,Interface: Evict sandbox cache of all idle candidates (#2744)
  Notes: The sandbox cache is now evicted for all idle candidates instead of relying on a calculation of total sandbox and total memory. This will reduce the unnecessary overhead in async thread which monitors the state of cache.
  Review: Code changes look good and match release notes.
• 003c70ac1 Execution,Interface: Evict sandboxes based on the available memory (#2651)
  Notes: Unless the available memory is less than 250GB, sandbox eviction based on RSS is avoided to optimize sandbox cache hit ratio.
  Review: Code changes look good and match release notes.
• 13006ee4d Execution,Interface,Message Routing: Time out messages in subnet queues (#2708)
  Notes: When timing out messages, also ensure that messages in the the management canister queues aka subnet_queues are timed out, as these may include best-effort responses that should be discarded if they have expired.
  Review: Code changes look good and match release notes.
• 7d8134765 Interface(ckbtc): re-evaluate discarded UTXOs (#2674)
  Notes: The UTXO will be re-evaluated with this change which were previously discarded and not minted to ckBTC because of either being very small in value or being marked as tainted. The rules have been changed and those utxo’s can now be valid.
  Review: Code changes look good and match release notes.
• f68da752b Interface(ICRC-Rosetta): updated rosetta to support icrc3 standard (#2607)
  Notes: Now the Rosseta implementation will support ICRC3 standard as well. For this some changes has been done to ICRC-1 ledger to use latest_block_hash instead of tip_hash in the ICRC3 certificate, to use leb128 encoding. Also a feature flag has been added for this new ICRC3 support.
  Review: Code changes look good and match release notes.
• e3ee0072c Interface,Message Routing: Unify checkpointed_state and state in commit_and_certify (#2827)
  Notes: two versions of ReplicatedState has been combined now by eliminating checkpointed_state while creating a checkpoint. This ensures that there are no diversions in both the state.
  Review: Code changes look good and match release notes.
• 90c56a39f Interface,Networking: enable the hashes-in-blocks feature everywhere (#2854)
  Notes: HASHES_IN_BLOCKS_FEATURE_ENABLED is now set to true, thus enabling hashes in block features for all the subnets on IC network.
  Review: Code changes look good and match release notes.
• 260f1cc09 Interface,Node: Upgrade build container to 24.04 (#1946)
  Notes: Updates the bases OS to version 24.04 ( ubuntu ).
  Review: Code changes look good and match release notes.
• 245e13ebd Interface,Node: Replace mac_address with deterministic_ips (#2757)
  Notes: the use of mac_address has been replaced with deterministic_ips all over the ic-os codebase and rs files.
  Review: Code changes look good and match release notes.
• e9f61b877 Interface,Node(IDX): don’t rely on bazel cache for large test deps (#2752)
  Notes: A little behaviour change has been done around bazel cache. Rather than relying on bazel cache for large dependencies like ic-os, it is now ensured by upload_systest_dep.sh along with bazel-remote to be always available for the tests.
  Review: Code changes look good and match release notes.
• a0be7baf6 Node: Allow local network to access metrics through firewall (#2703)
  Notes: Changes have been made to nftables and firewall conifers, to allow metrics collection.
  Review: Code changes look good and match release notes.

Bugfixes:

• 052b85232 Consensus,Interface(node): onboarding log typo (#2869)
  Notes: Minor typo change from ONBODING to ONBOARDING.
  Review: Code changes look good and match release notes.
• a55ac1812 Consensus,Interface(recovery): Clear recovery directory instead of deleting it (#2823)
  Notes: To avoid permission errors, the recovery directory is now cleaned up rather than delete.
  Review: Code changes look good and match release notes.
• c1d7c5ec6 Execution,Interface,Message Routing: Drop duplicate responses in induct_messages_to_self (#2725)
  Notes: Duplicate response message is dropped in induct_messages_to_self. This is done by checking if there is a match for should_enqueue_input and dropping the message if it returns false.
  Review: Code changes look good and match release notes.
• e7ce87bf2 Interface,Networking: remove the app handshake (#2790)
  Notes: Custom handshake called GRUEZI_HANDSHAKE is now removed as it is not needed anymore.
  Review: Code changes look good and match release notes.
• e9a0f1894 Interface,Networking(quic-transport): Correct the error code sent for reset stream frames. (#2761)
  Notes: Error code on QUIC_STREAM_CANCELLED has been corrected to 0x80000006
  Review: Code changes look good and match release notes.
• 8869fff65 Interface,Node: Revert "feat: Replace mac_address with deterministic_ips (#2785)
  Notes: Its a simple revert of a previous commit 245e13ebdd34b9114b9a0f8a68bb379b21657092. Will require some rework.
  Review: Code changes look good and match release notes.
• 0210728ba Node: Depend on //:bazel-timestamp instead of //:.git/index when calculating commit_timestamp_txt (#2765)
  Notes: The build dependency has been changed from .git/index to bazel-timestamp.
  Review: Code changes look good and match release notes.

Chores:

• cdf85cbaf Consensus,Interface(node): update onboarding log (#2830)
  Notes: A simple log change to enhance the onboarding behaviour.
  Review: Code changes look good and match release notes.
• 32cf9b0f8 Consensus,Interface(consensus): remove unnecessary variable in ingress selector (#2683)
  Notes: A simple change to remove unwanted variable num_messages and replace it with .len() function.
  Review: Code changes look good and match release notes.
• bef38d077 Crypto,Interface: upgrade thiserror and anyhow (#2732)
  Notes: thiserror has been upgraded from 1.0.68 to 2.0.3, and anyhow has been upgraded from ^1 to 1.0.93.
  Review: Code changes look good and match release notes.
• 5f4e13e98 Execution,Interface(EXC): Remove IPC calls for compilation in canister sandbox (#2853)
  Notes: Refactors the SandboxManager to eliminate the use of open_wasm for making inter-process requests during Wasm compilation. Instead, it directly invokes wasm_utils::compile_wasm, followed by the existing open_wasm_serialized, as the compilation process is already handled within a separate sandbox process.
  Review: Code changes look good and match release notes.
• 9d768e5d8 Execution,Interface: Make request metadata non optional (#2622)
  Notes: The request meta_data which was previously optional has been made non optional now as old certificate versions are now completely removed, and there are no very old requests in output queues / streams /input queues.
  Review: Code changes look good and match release notes.
• e636d3f34 Execution,Interface: Drop redundant CanisterQueuesTesting::output_message_count() (#2737)
  Notes: CanisterQueues::output_queues_message_count() is now used instead of output_message_count() which returns the same value.
  Review: Code changes look good and match release notes.
• ac6341599 Interface: upgrade crates (#2788)
  Notes: Various crates have been upgraded.
  Review: Code changes look good and match release notes.
• 385cf07ae Interface,Message Routing(fuzzing): Allow no_op_logger in StateMachine (#2736)
  Notes: In StateMachine the log_level is now set to default Some(Level::Warning), and if None is passed it uses no_op_logger.
  Review: Code changes look good and match release notes.
• 0cdfbb49f Interface,Networking: add a logging layer for tracing (#2650)
  Notes: Adds a new logging_layer ic-tracing-logging-layer to setup tracing based on passed in LoggingConfig. This config directs what needs to be logged in trace.
  Review: Code changes look good and match release notes.
• bbae0dd15 Interface,Networking: upgrade crates (#2742)
  Notes: Various crates have been bumped up.
  Review: Code changes look good and match release notes.
• 647fec3b1 Node: bump ic-bn-lib & ic-gateway (#2767)
  Notes: Bumps up ic-bn-lib and ic-gateway along with minor version bumps to rusls
  Review: Code changes look good and match release notes.
• 27e7499a7 Node: Add docs on nightly benchmarking (#2755)
  Notes: Readme has been updated in guesos and bare_metal_deployment for performance benchmarking.
  Review: Code changes look good and match release notes.
• acdbe1f72 Node: Update Base Image Refs [2024-11-21-0808] (#2731)
  Notes: Base images references are updated.
  Review: Code changes look good and match release notes.
• 2202c3f93 Node(IDX): clarify launch-remote-vm script (#2745)
  Notes: Use of genrule is replaced with sh_binary in VM developer tools, and some logic has been moved to launch-remote-vm.sh
  Review: Code changes look good and match release notes.

Refactoring:

• ccd5e2921 Consensus,Interface: Deprecate pb::TaggedNiDkgTranscript and implement new deserialization (#2838)
  Notes: Deprecates the use of TaggedNiDkgTranscript and new vector NiDkgTranscripts is used.
  Review: Code changes look good and match release notes.
• cf2727ce9 Consensus,Interface(CON): Rename dkg::Dealings to dkg::DataPayload (#2793)
  Notes: dkg::Dealings has been renamed to dkg::DataPayload and some other fields.
  Review: Code changes look good and match release notes.
• 7d5f65756 Crypto,Interface(crypto): rename VetKdArgs::encryption_key to encryption_public_key (#2789)
  Notes: Minor renaming VetKdArgs::encryption_key to VetKdArgs::encryption_public_key
  Review: Code changes look good and match release notes.
• 2f4c1fca1 Interface,Node: Replace mac_address with deterministic_ips (#2810)
  Notes: 245e13ebd is reintroduced with some fixes.
  Review: Code changes look good and match release notes.

Tests:

• e6f76957a Execution,Interface: Precompile universal canister (#2816)
  Notes: Introduces the with_precompiled_universal_canister flag to the execution environment test builder and integrates its functionality. When enabled, this flag ensures that the universal canister Wasm is precompiled and stored in the compilation cache. This approach eliminates redundant parallel compilations by multiple threads, optimizing the process.
  Review: Code changes look good and match release notes.
• ac64a1d95 Execution,Interface: Move long execution tests (#2814)
  Notes: some_ic0_calls_fail_if_called_with_huge_size is splitted into smaller tests.
  Review: Code changes look good and match release notes.
• 627fae2f7 Execution,Interface,Message Routing: Nicer SystemState tests (#2748)
  Notes: Some systemstate tests has been refactored to make them more informative.
  Review: Code changes look good and match release notes.
• e16e7f255 Interface,Networking(quic-transport): Add assertion for reset reason in drop handle test (#2764)
  Notes: New error code 0x80000006 is used to respond back with reason in an assertion.
  Review: Code changes look good and match release notes.

Documentation:

• 7fb898bdf Execution,Interface,Message Routing: Improve documentation around message and callback expiry (#2763)
  Notes: Minor documentation change around message and callback expiry.
  Review: Code changes look good and match release notes.

• ef0c2e44c Interface,Networking: improve error names and track other errors via metrics (#2797)
  Notes: This improves error names by adding more description + some metrics has been updated with additional details such as connection_handle_errors_total .
  Review: Code changes look good and match release notes.

• b2a094f7e Interface,Networking: improve some comments and remove infallible condition from metrics (#2773)
  Notes: Inline comments have been changed in observe_conn_error() and removes unused constant ERROR_CLOSED_STREAM.
  Review: Code changes look good and match release notes.

Proposal: 134337

Summary:

1. Build Hash: Build has from the proposal, local build and CDN matches and is “94f7aa25005c02c17464844d43875b1daaa022d4e99fce3a6873ba1467637aa6”.
2. Summary: The release notes matches the code changes
3. Vote: I vote to adopt the proposals
   
   

Detailed Review:

Other changes:

• f9085dbe8 Interface,Networking: Revert “feat: enable the hashes-in-blocks feature everywhere (#2854)”
  Notes: Its a revert of previous commit where HASHES_IN_BLOCKS_FEATURE_ENABLED is set to false. This is just for precaution in case something goes wrong in the next release.
  Review: Code changes look good and match release notes.

[ZackDS]

Proposal #134336 for release-2024-11-28_03-15-base

Vote: Adopted
Reason: Builds fine and the hash matches for GUESTOS, HOSTOS and SETUPOS along with all the listed commits.

Review of commits

to be edited

[ZackDS]

Proposal #134337 for release-2024-11-28_03-15-revert-hashes-in-blocks**

Vote: Adopted
Reason: Builds fine and the hash matches for GUESTOS, HOSTOS and SETUPOS along with the one listed commit.

The only change is
Other changes:
f9085dbe8 Interface,Networking: Revert “feat: enable the hashes-in-blocks feature everywhere ([#2854)
As explained this will be used as a “fallback” to have in case something breaks and it needs to be disabled.

[ipsita]

Proposal 134336

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Commits

Features

• 925acaec1
  Summary: Call sync and fstrim during node unassignment
  Notes: When a node is unassigned after unlinking the files we should sync the file systen and issue trim command so that the blocks not used by filesystem are discarded. A separate script is implemented which calls sync and fstrim since root access is required to run these commands.
  Review: Code changes matches the commit message.

• e3540ad62
  Summary: Introduce new context type for VetKD requests
  Notes: SignWithThresholdContext is extended by adding ThresholdArguments variant of type VetKdArguments. To handle tSchnorr and tEcdsa contexts and ignore KD contexts a new type IDkgSignWithThresholdContext is added to use only when context can be handled by IDKG.
  Review: Code changes matches the commit message.

• fd4d35f4a
  Summary: SOCKS proxy on the API BNs
  Notes: SOCKS proxy is moved to the API boundary nodes for the new node architecture to enable making http calls to IPv4. until now http calls were only supported for IPv6 destinations. Firewall is also updated so that port 1080 is only opened for node which are API boundary node.
  Review: Code changes matches the commit message.

• fd6561a04
  Summary: Add BIP-341 sig support to secp256k1 utility crate
  Notes: Add method sign_message_with_bip341 to secp256k1 sign the messages
  Review: Code changes matches the commit message.

• 3a9f843de
  Summary: Implement Ic00Method::VetKdPublicKey
  Notes: VetKdPublicKey management canister method is added which is similar to GetThresholdKey. idkg_subnet_keys is also renames to chain_key_subnets_keys since these keys does not hold idkg type keys.
  Review: Code changes matches the commit message.

• 8bf9965c2
  Summary: Store compiled Wasms on disk
  Notes: CompilationCache is modified to enable storing compiled wasm modules on disk. This option is disabled until next pr which will address passing these files to sandbox and will also reduce memory usage and limit space used on disk by compilation cache.
  Review: Code changes matches the commit message.

• 6a725785d
  Summary: Evict sandbox cache of all idle candidates
  Notes: All the idle candidates are removed from the sandbox cache as soon as they become idle. earlier these candidates were kept in the cache until they were replaced by another candidate which created overhead in the async thread which monitors the cache state.
  Review: Code changes matches the commit message.

• 003c70ac1
  Summary: Evict sandboxes based on the available memory
  Notes: Avoids RSS based eviction if there is sufficient memory available 250GiB or more since RSS based eviction can be pessimistic and negatively impacts sandbox hit ratio.
  Review: Code changes matches the commit message.

• 13006ee4d
  Summary: Time out messages in subnet queues
  Notes: Sheds best-effort messages in subnet input queues which are already expired.
  Review: Code changes matches the commit message.

• 7d8134765
  Summary: re-evaluate discarded UTXOs
  Notes: Discarded UTXOs which are not minted to ckBTC are re-evaluated since they can be valid and discarded because their value was too small or KYT check marked then as being tainted.
  Review: Code changes matches the commit message.

• f68da752b
  Summary: updated rosetta to support icrc3 standard
  Notes: updates on ICRC-1 ledger to use latest_block_hash and leb128 encoding in ICRC3 certificate. Feature flag for ICRC updates and updates the icrc agent to support icrc1 ledger changes.
  Review: Code changes matches the commit message.

• e3ee0072c
  Summary: Unify checkpointed_state and state in commit_and_certify
  Notes: checkpointed_state is eliminated to have only one ReplicatedState when a check point is created.
  Review: Code changes matches the commit message.

• 90c56a39f
  Summary: enable the hashes-in-blocks feature everywhere
  Notes: HASHES_IN_BLOCKS_FEATURE_ENABLED feature flag is set to true
  Review: Code changes matches the commit message.

• 260f1cc09
  Summary: Upgrade build container to 24.04
  Notes: Updated the OS to ubuntu 24.04
  Review: Code changes matches the commit message.

• 245e13ebd
  Summary: Replace mac_address with deterministic_ips
  Notes: mac_address is replaced with deterministic_ips in multiple files.
  Review: Code changes matches the commit message.

• e9f61b877
  Summary: don’t rely on bazel cache for large test deps
  Notes: upload_systest_dep.sh script is added to ensure that images have been stored in bazel-remote cache. This ensures that the images are always ready when the tests are run even after the cache is purned.
  Review: Code changes matches the commit message.

• a0be7baf6
  Summary: Allow local network to access metrics through firewall
  Notes: Updates firewall rules so that the metrics are accessible on local network.
  Review: Code changes matches the commit message.

Bugfixes

• 052b85232
  Summary: onboarding log typo
  Notes: onboarding typo is fixed in orchestrator.rs
  Review: Code changes matches the commit message.

• a55ac1812
  Summary: Clear recovery directory instead of deleting it
  Notes: ic-recovery uses var/lib/ic/data/ic_state directory by default, during cleanup ic-recovery tries to delete the directory which results in permission error. Instead we remove the contents of this directory.
  Review: Code changes matches the commit message.

• c1d7c5ec6
  Summary: Drop duplicate responses in induct_messages_to_self **Notes:** CanisterQueuescheck for duplicate response messages for a callback but reply onSystemStateto ensure they are valid.SystemStatechecks this inpush_input()this check is also added ininduct_messages_to_self()`
  Review: Code changes matches the commit message.

• e7ce87bf2
  Summary: remove the app handshake
  Notes: Code for custom handshake is removed since it is not longer needed.
  Review: Code changes matches the commit message.

• e9a0f1894
  Summary: Correct the error code sent for reset stream frames
  Notes: QUIC_STREAM_CANCELLED error code value is changed from 6 to 0x80000006
  Review: Code changes matches the commit message.

• 8869fff65
  Summary: Revery: Replace mac_address with deterministic_ips
  Notes: This reverts commit 245e13e
  Review: Code changes matches the commit message.

• 0210728ba
  Summary: Depend on //:bazel-timestamp instead of //:.git/index when calculating commit_timestamp_txt
  Notes: //:.git/index is replaced with //:bazel-timestamp when calculating commit timestamp.
  Review: Code changes matches the commit message.

Chores

• cdf85cbaf
  Summary: update onboarding log
  Notes: onboarding log is updated to emphasize onboarding is not yet complete
  Review: Code changes matches the commit message.

• 32cf9b0f8
  Summary: remove unnecessary variable in ingress selector
  Notes: calculates the number of ingress messages by calling messages_in_payload.len() instead of num_messages
  Review: Code changes matches the commit message.

• bef38d077
  Summary: upgrade thiserror and anyhow
  Notes: upgrades thiserror from 1.0.68 to 2.0.3 and anyhow to 1.0.93
  Review: Code changes matches the commit message.

• 5f4e13e98
  Summary: Remove IPC calls for compliation in canister sandbox
  Notes: IPC requests for compiling in canister sandbox are deprecated since we have a separate compilation sandbox.
  Review: Code changes matches the commit message.

• 9d768e5d8
  Summary: Make request metadata non optional
  Notes: request metadata is now non optional since old requests without metadata does not exists anymore.
  Review: Code changes matches the commit message.

• e636d3f34
  Summary: Drop redundant CanisterQueuesTesting::output_message_count()
  Notes: updates canister_queues.output_message_count() to canister_queues.output_queues_message_count() which returns the exact same value.
  Review: Code changes matches the commit message.

• ac6341599
  Summary: upgrade crates
  Notes: some dependencies like hyper , rustls , tower etc are upgraded.
  Review: Code changes matches the commit message.

• 385cf07ae
  Summary: Allow no_op_logger in StateMachine
  Notes: defaults to warning log level and if none is passed it uses no_op_logger.
  Review: Code changes matches the commit message.

• 0cdfbb49f
  Summary: add a logging layer for tracing
  Notes: Adds ic-tracing-logging-layer package logging layer to tracing which can show log level >= INFO from 3rd party dependencies.
  Review: Code changes matches the commit message.

• bbae0dd15
  Summary: upgrade crates
  Notes: upgrades some dependencies
  Review: Code changes matches the commit message.

• 647fec3b1
  Summary: bump ic-bn-lib & ic-gateway
  Notes: upgrades ic-bn-lib, ic-gateway and rustls
  Review: Code changes matches the commit message.

• 27e7499a7
  Summary: Add docs on nightly benchmarking
  Notes: Adds documentation in read.me file
  Review: Code changes matches the commit message.

• acdbe1f72
  Summary: Update Base Image Refs [2024-11-21-0808]
  Notes: updates base ic-os image references.
  Review: Code changes matches the commit message.

• 2202c3f93
  Summary: clarify launch-remote-vm script
  Notes: code refactoring to simplify logic by adding logic to launch-remote-vm.sh
  Review: Code changes matches the commit message.

Refactoring

• ccd5e2921
  Summary: Deprecate pb::TaggedNiDkgTranscript and implement new deserialization
  Notes: TaggedNiDkgTranscript is deprecated and NiDkgTranscript is and introduces decentralization function to consider both the fields.
  Review: Code changes matches the commit message.

• cf2727ce9
  Summary: Rename dkg::Dealings to dkg::DataPayload
  Notes: renames DKG payload type from Dealings to DkgDataPayload since the block also contains height in addition to dealings and will be extended in future to contain transcripts.
  Review: Code changes matches the commit message.

• 7d5f65756
  Summary: rename VetKdArgs::encryption_key to encryption_public_key
  Notes: Renames VetKdArgs::encryption_key to
  VetKdArgs::encryption_public_key to make it clear that this is not sensitive information.
  Review: Code changes matches the commit message.

• 2f4c1fca1
  Summary: Replace mac_address with deterministic_ips
  Notes: Bugfixes to replace mac_address with deterministic_ips which was adding in commit #2757 but reverted in #2785
  Review: Code changes matches the commit message.

Tests

• e6f76957a
  Summary: Precompile universal canister
  Notes: Adds build target to compile universal canister from wasm to Wasmtime::Module and includes this module to compilation cache for execution tests.
  Review: Code changes matches the commit message.

• ac64a1d95
  Summary: Move long execution tests
  Notes: some longer running tests are splitted to multiple short test cases.
  Review: Code changes matches the commit message.

• 627fae2f7
  Summary: Nicer SystemState tests
  Notes: code refactoring for test cases
  Review: Code changes matches the commit message.

• e16e7f255
  Summary: Add assertion for reset reason in drop handle test
  Notes: Adds assertion check to make sure that the error code is the updated value 0x80000006 instead to 6
  Review: Code changes matches the commit message.

Documentation

• 7fb898bdf
  Summary: Improve documentation around message and callback expiry
  Notes: Documentation improvements like which canister messages expire and try_push_deadline_expired_input() only applies to best-effort callbacks etc.
  Review: Code changes matches the commit message.

• ef0c2e44c
  Summary: improve error names and track other errors via metrics
  Notes: updates some comments and updates PeerIdMismatch to InvalidIncomingPeerId
  Review: Code changes matches the commit message.

• ef0c2e44c
  Summary: improve some comments and remove infallible condition from metrics
  Notes: updates some comments on metrics.rs file.
  Review: Code changes matches the commit message.

Proposal 134337

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Commits

Other changes

• f9085dbe8
  Summary: Revert "feat: enable the hashes-in-blocks feature everywhere
  Notes: reverts commit 90c56a3
  Review: Code changes matches the commit message.

[yuvika]

Proposal 134336

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits

Commits

Features:

1. 925acaec1
   Summary: Call sync and fstrim during node unassignment
   Notes: When a node is unassigned, sync and fstrim are called on the replica’s data directory from a separate shell script with root access . This is done to ensure the blocks not used by the filesystem are discarded.
   Review: The description matches the code changes.

2. e3540ad62
   Summary: Introduce new context type for VetKD requests.
   Notes: Extend SignWithThresholdContext by adding another variant ThresholdArguments of type VetKdArguments . Added a new type IDkgSignWithThresholdContext which wraps the reference to a
   SignWithThresholdContext.
   Review: The description matches the code changes.

3. fd4d35f4a
   Summary: SOCKS proxy on the API BNs.
   Notes: Added dante to both in the base image and the normal image, however this will be updated to be only in the base image. Install Dante and adjust the firewall to allow port 1080 to be opened when the node is an API boundary node, to make ipv4 calls using SOCKS proxy and only for nodes that are part of a system subnet.Review: The description matches the code changes.

4. fd6561a04
   Summary: Add sig support to secp256k1 utility crate
   Notes: secp256k1 utility crate updated, added sign_message_with_bip341.
   Review: The description matches the code changes.

5. 3a9f843de
   Summary: Implement Ic00Method::VetKdPublicKey.
   Notes: Implement management canister method VetKdPublicKey . Rename idkg_subnet_keys to chain_key_subnets_keys .
   Review: The description matches the code changes.

6. 8bf9965c2
   Summary: Store compiled Wasms on disk.
   Notes: This is disabled for now, will be completed in following proposals. Changed made to CompilationCacheCompiled, for wasm modules to be stored as temporary files on disk to reduce sandbox memory usage.
   Review: The description matches the code changes.

7. 6a725785d
   Summary: Evict sandbox cache of all idle candidates.
   Notes: To reduce overhead in the asynchronous thread which monitors the cache state, evict the sandbox processes of all idle canisters regardless of the number or memory they consume.
   Review: The description matches the code changes.

8. 003c70ac1
   Summary: Evict sandboxes based on the available memory.
   Notes: If 250 GiB or more memory is available, avoid RSS-based sandbox eviction.
   Review: The description matches the code changes.

9. 13006ee4d
   Summary: Time out messages in subnet queues.
   Notes: Time out best-effort message requests in subnet input queues which are already expired.
   Review: The description matches the code changes.

10. 7d8134765
    Summary: re-evaluate discarded UTXOs .
    Notes: Re-evaluate already discarded UTXOs, since they were either very small in value or being marked as tainted. Some updates to whether a UTXO is tainted, both the KYT fee and the decision of whether a UTXO is tainted will change.
    Review: The description matches the code changes.

11. f68da752b
    Summary: updated rosetta to support icrc3 standard.
    Notes: Updates to ICRC-1 ledger: to use latest_block_hash instead of tip_hash, to use leb128 encoding for the latest_block_index in the ICRC3 certificate. Add a feature flag to ICRC3.
    Review: The description matches the code changes.

12. e3ee0072c
    Summary: Unify checkpointed_state and state in commit_and_certify.
    Notes: Eliminate checkpointed_state and have only one ReplicatedState when a check point is created, and should they diverge a critical error must be raised.
    Review: The description matches the code changes.

13. 90c56a39f
    Summary: enable the hashes-in-blocks feature everywhere.
    Notes: Set HASHES_IN_BLOCKS_FEATURE_ENABLED to true, which will enable the hashes in block feature everywhere.
    Review: The description matches the code changes.

14. 260f1cc09
    Summary: Upgrade build container to 24.04.
    Notes: Update Ubuntu image to version 24.04.
    Review: The description matches the code changes.

15. 245e13ebd
    Summary: Replace mac_address with deterministic_ips.
    Notes: Switch usage from the mac_address package to the deterministic_ips package.
    Review: The description matches the code changes.

16. e9f61b877
    Summary: don’t rely on bazel cache for large test deps.
    Notes: Remove dependancy on bazel cache and instead use stored images that are in the bazel-remote cache. This is done through the upload_systest_dep script. This simplifies the bazel build as we don’t need both a URL & a sha256 file for images and also gives more flexibility with the implementation of Bazel cache.
    Review: The description matches the code changes.

17. a0be7baf6
    Summary: Allow local network to access metrics through firewall.
    Notes: Modify firewall rules such that the metrics can be accessible on local network.
    Review: The description matches the code changes.

Bugfixes:

1. 052b85232
   Summary: onboarding log typo.
   Notes: Logging typo in orchestrator.rs, changed from ONBOADING to ONBOARDING.
   Review: The description matches the code changes.

2. a55ac1812
   Summary: Clear recovery directory instead of deleting it.
   Notes: During clean-up the default recovery directory used by ic-recovery in var/lib/ic/data/ic_state , is deleted which results in a permission error. We should instead, clear the directory’s contents.
   Review: The description matches the code changes.

3. c1d7c5ec6
   Summary: Drop duplicate responses in induct_messages_to_self.
   Notes: Drop duplicate response message in induct_messages_to_self. Check the reurn type of should_enqueue_input and if returns false, silently drop it.
   Review: The description matches the code changes.

4. e7ce87bf2
   Summary: remove the app handshake.
   Notes: Remove GRUEZI_HANDSHAKE which was a custom handshake and is not needed anymore.
   Review: The description matches the code changes.

5. e9a0f1894
   Summary: Correct the error code sent for reset stream frames.
   Notes: Update error code QUIC_STREAM_CANCELLED from 6 to 0x80000006.
   Review: The description matches the code changes.

6. 8869fff65
   Summary: Revert "feat: Replace mac_address with deterministic_ips.
   Notes: Revert commit 245e13e.
   Review: The description matches the code changes.

7. 0210728ba
   Summary: Depend on //:bazel-timestamp instead of //:.git/index when calculating commit_timestamp_txt.
   Notes: Update build dependency from .git/index to bazel-timestamp so that the commit timestamp is not cached.
   Review: The description matches the code changes.

Chores:

1. cdf85cbaf
   Summary: update onboarding log.
   Notes: Clarify the onboarding log to emphasize onboarding is not yet complete./
   Review: The description matches the code changes.

2. 32cf9b0f8
   Summary: remove unnecessary variable in ingress selector.
   Notes: Compute the number of ingress messages by using messages_in_payload.len() instead of using a separate variable num_messages to track it.
   Review: The description matches the code changes.

3. bef38d077
   Summary: upgrade thiserror and anyhow.
   Notes: Upgrade thiserror from 1.0.68 to 2.0.3 and anyhow from 1 to 1.0.93.
   Review: The description matches the code changes.

4. 5f4e13e98
   Summary: Remove IPC calls for compilation in canister sandbox.
   Notes: Deprecate IPC requests for compiling in the canister sandbox since there is a separate compilation sandbox.
   Review: The description matches the code changes.

5. 9d768e5d8
   Summary: Make request metadata non optional.
   Notes: Request::metadata is made non-optional now since old requests without metadata do not exist anymore and all instances of it will have the field populated.
   Review: The description matches the code changes.

6. e636d3f34
   Summary: Drop redundant CanisterQueuesTesting::output_message_count().
   Notes: Use CanisterQueues::output_queues_message_count() instead of CanisterQueuesTesting::output_message_count() as it returns the exact same value.
   Review: The description matches the code changes.

7. ac6341599
   Summary: upgrade crates.
   Notes: Upgrade multiple crates such as hyper, rustls, axum, tower etc.
   Review: The description matches the code changes.

8. 385cf07ae
   Summary: Allow no_op_logger in StateMachine.
   Notes: In StateMachine set the log_level default to Some(Level::Warning) and if None is passed, use the no_op_logger unlike earlier where log_level: None was the default and
   it would switch to Warning in fn replica_logger.
   Review: The description matches the code changes.

9. 0cdfbb49f
   Summary: add a logging layer for tracing.
   Notes: Add ic-tracing-logging-layer package to tracing as it can provide more logs from 3rd party dependencies (log level >= INFO).
   Adds logging layer
   Review: The description matches the code changes.

10. bbae0dd15
    Summary: upgrade crates.
    Notes: Upgrade various crates.
    Review: The description matches the code changes.

11. 647fec3b1
    Summary: bump ic-bn-lib & ic-gateway.
    Notes: Upgrade ic-gateway to 0.1.60, ic-bn-lib to a later commit and rustls from 0.23.16 to 0.23.17.
    Review: The description matches the code changes.

12. 27e7499a7
    Summary: Add docs on nightly benchmarking.
    Notes: Improve nightly benchmarking documentation by adding further details.
    Review: The description matches the code changes.

13. acdbe1f72
    Summary: Update Base Image Refs [2024-11-21-0808].
    Notes: Update base container image references.
    Review: The description matches the code changes.

14. 2202c3f93
    Summary: clarify launch-remote-vm script.
    Notes: Refactor launch-remote-vm.sh script.
    Review: The description matches the code changes.

Refactoring:

1. ccd5e2921
   Summary: Deprecate pb::TaggedNiDkgTranscript and implement new deserialization.
   Notes: Deprecate TaggedNiDkgTranscript vectors and instead use new vectors of NiDkgTranscript.
   Review: The description matches the code changes.

2. cf2727ce9
   Summary: Rename dkg::Dealings to dkg::DataPayload.
   Notes: Rename dkg::Dealings to dkg::DataPayload and some other fields.
   Review: The description matches the code changes.

3. 7d5f65756
   Summary: rename VetKdArgs::encryption_key to encryption_public_key.
   Notes: Rename VetKdArgs::encryption_key to VetKdArgs::encryption_public_key.
   Review: The description matches the code changes.

4. 2f4c1fca1
   Summary: Replace mac_address with deterministic_ips.
   Notes: Replace mac_address with deterministic_ips . Similar to 245e13ebd is reintroduced with the necessary fixes.
   Review: The description matches the code changes.

Tests:

1. e6f76957a
   Summary: Precompile universal canister.
   Notes: Add a build target which can compile the universal canister from Wasm to a Wasmtime::Module and include this module to the compilation cache for ExecutionTests by default.
   Review: The description matches the code changes.

2. ac64a1d95
   Summary: Move long execution tests.
   Notes: Refactor the tests; move some longer execution tests into integration tests and shorten others by splitting them into multiple testcases.
   Review: The description matches the code changes.

3. 627fae2f7
   Summary: Nicer SystemState tests.
   Notes: Refactor SystemState tests, improve log messages etc.
   Review: The description matches the code changes.

4. e16e7f255
   Summary: Add assertion for reset reason in drop handle test.
   Notes: Add a check an assertion to verify if the updated error code is returned (0x80000006 instead of `6`` ) as it was updated in e9a0f1894.
   Review: The description matches the code changes.

Documentation:

1. 7fb898bdf
   Summary: Improve documentation around message and callback expiry.
   Notes: Update CanisterQueues documentation on message expiry, try_push_deadline_expired_input() which applies to best-effort callbacks etc.
   Review: The description matches the code changes.

2. ef0c2e44c
   Summary: improve error names and track other errors via metrics.
   Notes: Rename error code from PeerIdMismatch to InvalidIncomingPeerId + use other metrics to track errors + update comments etc.
   Review: The description matches the code changes.

3. b2a094f7e
   Summary: improve some comments and remove infallible condition from metrics.
   Notes: Update comments in metrics.rs file. + remove ERROR_CLOSED_STREAM unused constant.
   Review: The description matches the code changes.

Proposal 134337

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits

Commits

Other Changes:

1. f9085dbe8
   Summary: Revert "feat: enable the hashes-in-blocks feature everywhere.
   Notes: This reverts commit 90c56a3. Set HASHES_IN_BLOCKS_FEATURE_ENABLED to False. This is a fallback in case issues are seen.
   Review: The description matches the code changes.

[timk11]

Proposal 134336

Vote: Adopt

Reason: Build is successful, hashes match, commits match descriptions and the reasoning behind the changes is sound.

I’ve selectively reviewed Consensus, Crypto and Interface commits as detailed below.

Review

Features:

[925acaec1]
Adds a new function sync_and_trim_fs which calls a new script sync_fstrim.sh which calls sync and fstrim (filesystem trim) from root. This function is now called via async fn remove_state when a node is to be unassigned. Error logging and tests are added.

[e3540ad62]
Adds a variant VetKd(VetKdArguments) to enum ThresholdArguments. Adds type IDkgSignWithThresholdContext, wrapping SignWithThresholdContext and using flag is_idkg() so that this is only used for tEcdsa and tSchnorr contexts given that these rely on IDKG, and not for vetKD contexts, which rely on NiDKG.

[fd4d35f4a]
Installs dante as a SOCKS proxy and applies it to API boundary nodes as per the commit notes.

[fd6561a04]
Adds functions to rs/crypto/secp256k1/src/lib.rs to enable Taproot / BIP-341 signature support. Changes relevant dependencies and adds tests.

[7d8134765]
Various changes, matching the description such that previously discarded bitcoin UTXOs are re-evaluated when the user calls update_balance.

[f68da752b]
Changes to support the ICRC-3 standard for certification of blocks, as per description, although “latest_…” should read “last_…” in the text of these notes. In particular, last_block_hash is used instead of tip_hash in fn construct_hash_tree, and LEB128 encoding is used for last_block_index in fn get_certified_chain_tip when tip_hash is not present in the hash tree.

[e3ee0072c]
Replaces checkpointed_state field with state in CreateCheckpointResult.

[90c56a39f]
Enables the hashes-in-blocks feature, intended to be used for the NNS subnet as has already been done for all the other subnets.

[260f1cc09]
Upgrades IC build container version.

[245e13ebd]
Shifts deterministic_ips out of dev_test_tools and utilises it in place of mac_address. Adds logic (as fn resolve_mgmt_mac) to get the management MAC address either from a provided parameter or from ipmitool.

[e9f61b877]
Test changes, as per commit notes.

Bugfixes:

[052b85232]
Typo correction.

[a55ac1812]
Adds clear_dir function, which serves to clear the recovery directory’s contents instead of attempting to delete the directory.

[e7ce87bf2]
Removes Grüezi handshake function, deemed unnecessary now that SEV attestation is no longer used.

[e9a0f1894]
Changes QUIC_STREAM_CANCELLED error code.

[8869fff65]
Reverts commit 245e13e (replace mac_address with deterministic_ips) above.

Chores:

[cdf85cbaf]
Adds further emphasis to the log message indicating that node onboarding is not yet complete.

[32cf9b0f8]
Replaces redundant num_messages variable with messages_in_payload.len().

[bef38d077]
Version changes for thiserror and anyhow + minor error log code changes.

[ac6341599]
Version updates to several crates + minor boundary node code changes.

[385cf07ae]
Fuzz testing code changes, matching description.

[0cdfbb49f]
Adds crate ic-tracing-logging-layer, used for tracing in ‎rs/replica/src/main.rs.

[bbae0dd15]
Version updates to several crates.

Refactoring:

[ccd5e2921]
Flags TaggedNiDkgTranscript types as deprecated and adds ‘_new’ equivalents, flagged as such.

[cf2727ce9]
Renames dkg::Dealings to dkg::DataPayload.

[7d5f65756]
Renames VetKdArgs::encryption_key to VetKdArgs::encryption_public_key to clarify that sensitive information is not being exposed.

[2f4c1fca1]
Re-attempts replacement of mac_address with deterministic_ips as per 245e13e above, with additional changes to hostos-scripts/ and setupos-scripts/.

Tests:

[e16e7f255]
Adds assertion for reset reason in dropped_connection_handle_resets_the_stream QUIC transport test.

Documentation:

[ef0c2e44c]
Added more detail to error tracking and error messages.

[b2a094f7e]
Changes to comments + relabeling of WriteError::ClosedStream and WriteError::ZeroRttRejected as infallible.

Proposal 134337

Vote: Adopt

Reason: Build is successful and hashes match. The proposal contains a single commit, serving to revert the enabling of the hashes-in-blocks feature, in case this is needed as a fallback for some reason.

[Zane]

Proposal 134336

Vote: ADOPT

Reason: Build is successful and hashes match, reviewed commits also match their descriptions, therefore I’ve voted to adopt.

Features:

3a9f843de Added derive_vetkd_public_key method which can be used to derive a public key given a master public key and derivation path, it first ensures the master key uses the correct signature algorithm and that it can be deserialized, the derivation path is constructed by providing the caller and an additional path as derivation indexes, finally the public key is derived and encoded into a vector.
This method is used to implement the VetKdPublicKey endpoint of management canister, now when it is called payload is decoded and the derivation path, canister and key id to use are extracted from it. Then the master public key is retrieved and a call is made to get_vetkd_public_key method, which is a light wrapper for derive_vetkd_public_key, used to map possible VetKdPublicKeyDeriveError returned by the latter into UserError. Some methods and variables have been renamed to use chain key nomenclature instead of idkg.

8bf9965c2 Heavily refactored CompilationCache to store compiled wasm on disk. While it used to be a struct it has now been turned into an enum with two variants, one for the in memory cache and one for the on disk cache. All its methods exception made for the constructor have been updated to handle both caching mechanisms. insert has also been separated into two new methods, insert_err and insert_ok the former is used to insert a compilation error into the cache while the latter is used to insert a successful compilation result. StoredCompilation has been introduced to abstract read operations on wasm module based on storage type. In memory WASMs still rely on SerializedModule struct, while for on disk ones OnDiskSerializedModule struct has been added to encapsulate all data and logic required to read and write a wasm module to disk.
Finally some code has been updated to use the new insert method variants and panic the replica whenever the cache returns StoredCompilation::Disk since the feature is still not currently supported.

6a725785d Revert some of last week’s optimization for process eviction, now even if total rss and number of active sandboxes are below the imposed limits, the eviction logic will still be triggered and all idle canisters will be evicted.

003c70ac1 Duplicate of commit a3478d571, which was part of last week’s build and has already been reviewed.

13006ee4d In ReplicatedState::time_out_messages call timeout messages for subnet queues if they have at least one expired message in their pool.

e3ee0072c Get rid of checkpointed_state in commit_and_certify so that there is only a version of ReplicateState after a checkpoint is created. The checkpointed state is only kept around in create_checkpoint_and_switch until equality between the version loaded from disk and the one in memory is verified.

90c56a39f Enabled feature flag for hashes in blocks feature.

Bugfixes:

c1d7c5ec6 Modified induct_messages_to_self to prevent it from enqueueing duplicate messages, when looping over all responses in the output queue, a call to should_enqueue_input has been added and if the response isn’t safe to induct it is skipped.
Logic from pop_canister_output has been moved to a new method with the same name in CanisterQueues struct, operations performed are the same but some debug asserts have been added to ensure invariants are respected.

Chores:

5f4e13e98 Removed code from canister sandbox used to decode and compile a wasm binary since it is now performed in the compiler sandbox.

9d768e5d8 Changed RequestMetadata to no longer be optional and updated code accordingly. Change is safe cause old certification versions have been deprecated and the field has been populated for a while now so no stale requests that don’t have it should be present. Furthermore protobuf definition and conversion code have been updated to also make RequestMetadata’s field call_tree_depth and call_tree_start_time_nanos mandatory, this is a preliminary step to change their type in future releases.

e636d3f34 Replaced all instances of output_message_count with output_queues_message_count and eliminated the method entirely.

ac6341599 Bump version of several crates.

385cf07ae Modified replica_logger to use no_op_logger if no log level is specified instead of defaulting to Level::Warning.

Tests:

e6f76957a Added build target to precompile the universal canister. ExecutionTestBuilder has been extended with precompiled_universal_canister field, which acts as a feature flag to determine whether the compiled universal canister should be added to the hypervisor compilation cache.

ac64a1d95 Moved orthogonal persistence and hypervisor related tests to integration tests. Reduced available_wasm_custom_sections_memory from 20KiB to 1KiB in install_code_respects_wasm_custom_sections_available_memory. Divided some_ic0_calls_fail_if_called_with_huge_size test into smaller functions, one containing actual validation logic and 3 acting as tests for an individual sys call.

627fae2f7 Added prepare_call helper method to SystemStateFixture to build and return request/response pair to a provided canister id backed by a callback with a configurable deadline.

Documentation:

7fb898bdf Improved some comments, no code changes.

ef0c2e44c Renamed PeerIdMismatch variant of ConnectionEstablishError to InvalidIncomingPeerId. Increase connection_handle_errors_total when to_response returns an error. Improved doc comments for errors.

Proposal 134337

Vote: ADOPT

Reason: Build is successful and hashes match, the proposed changes are the same as 134336 but with feature flag for hashes in block feature disabled.