Proposal to elect new release rc--2024-11-21_03-11

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 134250.

Here is a summary of the changes since the last release:

Release Notes for release-2024-11-21_03-11-base (a3478d571670053ebd09b9771bec489ef69425fa)

This release is based on changes since release-2024-11-14_03-07-base (cb3cb61009d904bcb726781ad379de10e1b745ff).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

This release diverges from the latest release. Merge base is 13c0601e09de77b2011f5282b175d569c4a2bb6a. Changes were removed from this release.

Features:

• f3bb40251 Consensus,Interface: Instrument payload size and batch delivery interval (#2669)
• e44d2bc85 Crypto,Interface(crypto): create VetKdProtocol trait (#2569)
• be026d014 Crypto,Interface(crypto): Use NiDkg-specific MasterPublicKeyId in NiDkgTag (#2620)
• af542cdad Crypto,Interface(crypto): Extend NiDkgTag with HighThresholdForKey variant (#2445)
• a3478d571 Execution,Interface: Evict sandboxes based on the available memory (#2651)
• caca44da3 Execution,Interface: Add vetKeys related management canister endpoints (#2633)
• a1e516f92 Execution,Interface: Evict sandboxed based on scheduler priorities (#2590)
• 826d9a503 Execution,Interface: Restrict legacy ICQC code to one subnet (#2647)
• 77164cdf7 Execution,Interface: Penalize canisters at the end of the round (#2601)
• e391f4cdf Execution,Interface: Define and enforce callback limits (#1838)
• 1ee1fe368 Execution,Interface: Don’t apply priority credit on abort (#2597)
• f8c4eb15e Interface,Node(node): add node_reward_type to config tool (#2641)
• 1cebd7342 Node: Update GuestOS base images with 6.11 kernel (#2667)
• 4e46b92fc Node: Add node_reward_type to config.ini (#1787)

Bugfixes:

• 338b77f6c Consensus,Interface(consensus): Don’t increase the notary delay during upgrades (#2677)
• 545f2fad6 Consensus,Interface: Handle ChainKeyInitializations conversion errors (#2635)
• 9870f1ea8 Execution,Interface: Correctly observe block maker to induction latency (#2663)
• a078ed82b Interface: stop doing u32 subtraction in bitcoin headers validation (#2634)
• 2902a2a37 Interface,Networking: increase the keep alive probes. (#2713)
• b679ad3a3 Interface,Networking: Remove the async lock in the AdapterMetricsRegistry (#2613)
• 1c2346cab Node: HSM onboarding (#2636)

Benchmarking:

• 211a01015 Execution,Interface: Add criterion load benchmark (#2587)

Chores:

• 9732c2c26 Consensus,Interface(consensus): Remove a no longer needed metric (#2679)
• 85c272c78 Consensus,Interface(consensus): Remove some clones from ingress selector (#2627)
• 5a5e0be15 Consensus,Interface(node): Add HostOS console message clarifying onboarding success (#2640)
• 87270d904 Consensus,Interface: Avoid handling VetKdKeyIds in the IDKG component (#2388)
• 659d9b143 Execution,Interface: Fix default scheduler priority in evict (#2719)
• 5813a429f Execution,Interface: Rename and test routing function for vetKD (#2680)
• 2f00d6815 Execution,Interface: Optimize evict_sandbox_processes (#2653)
• a10fbc291 Execution,Interface(fuzzing): Reconstruct system API imports using Wasmtime::Linker (#2575)
• 0d14c11bd Execution,Interface,Message Routing: fixing typos and cleanups (#2682)
• 0eedbb674 Interface,Message Routing: Always use request metadata in backward compatibility test (#2690)
• aacbed376 Interface,Message Routing: Trim canonical state framework (#2539)
• 7dd4dfe62 Interface,Networking: add a jaeger exporter package/crate (#2691)
• 8b94d60b6 Interface,Networking: upgrade opentelemetry and num_cpus crates (#2656)
• 45dca07dd Interface,Networking: Do not shutdown the joinset when the event loop exists in the request handler (#2649)
• 72ec446d6 Interface,Networking: Add additional logging in the case when the version message is not valid (#2596)
• f96dec1db Interface,Node(node): Remove setupos/hostos network bonding and clean up docs (#2579)
• c43bd9b0c Owners: Bump ic-cdk-timers (#2654)
• 7dd4fd0f8 Node: Update Base Image Refs [2024-11-20-2036] (#2727)
• c00695e45 Node: Update Base Image Refs [2024-11-20-0147] (#2705)
• 0d127b8fd Node: Add log_and_halt_installation_on_error sleep (#2605)
• 4c6c5dae2 Node: Update Base Image Refs [2024-11-14-0808] (#2606)

Refactoring:

• 81686f56f Crypto,Interface(crypto): Move ExtendedDerivationPath to ic_types::crypto (#2676)
• 3397eb8bf Crypto,Interface(crypto): Inline ThresholdSigDataStoreImpl::new_with_max_size (#2625)
• dab484292 Interface,Node(node): rename use_nns_public_key and use_node_operator_private_key (#2700)
• 03f3debdf Node: move node_index to ic.json (#1947)

Tests:

• d0acab2b2 Interface: Inline SNS system-tests (#2646)
  Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/a3478d571670053ebd09b9771bec489ef69425fa/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c a3478d571670053ebd09b9771bec489ef69425fa --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 134251.

Here is a summary of the changes since the last release:

Release Notes for release-2024-11-21_03-11-hashes-in-blocks (6e2c0d9fbea99f799a987a365ff6ad2b883067c9)

This release is based on changes since release-2024-11-21_03-11-base (a3478d571670053ebd09b9771bec489ef69425fa).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

This release diverges from the latest release. Merge base is 0dc55e08ab70c0f561e3617a3d6f41c58d7b2b38. Changes were removed from this release.

Features:

• 6e2c0d9fb Execution,Interface: Evict sandboxes based on the available memory (#2651)
• b16d21c2a Interface,Networking(Consensus): Enable the hashes-in-blocks feature

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/6e2c0d9fbea99f799a987a365ff6ad2b883067c9/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 6e2c0d9fbea99f799a987a365ff6ad2b883067c9 --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 134259.

Here is a summary of the changes since the last release:

Release Notes for release-2024-11-21_03-11-24.04-base-kernel (5d202894864f4db4a5a46f44422aebc80c3d321b)

This release is based on changes since release-2024-11-21_03-11-base (a3478d571670053ebd09b9771bec489ef69425fa).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

This RC is a fallback to deploy just in case we have issues with the 6.11 kernel. The image variants used in this RC include images with the default 6.8 kernel generated before this commit.

Other changes:

• 5d2028948 Node: Revert “chore: Update Base Image Refs [2024-11-20-2036] (#2727)”

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/5d202894864f4db4a5a46f44422aebc80c3d321b/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 5d202894864f4db4a5a46f44422aebc80c3d321b --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[hpeebles]

Proposal 134250: Adopt

I have successfully run the build verification script and in my opinion all of the commits look fine.

Features:

• f3bb40251 Consensus,Interface: Instrument payload size and batch delivery interval (#2669)
  Review: Looks fine + matches description
  Notes: Adds 3 new metrics, batch_delivery_interval (which measures the time between finalized batches), payload_size_bytes (which measures the size of each consensus block), and payload_section_size_bytes (which measures the sizes of the sections within each consensus block).

• e44d2bc85 Crypto,Interface(crypto): create VetKdProtocol trait (#2569)
  Review: Looks fine + matches description
  Notes: Adds the VetKdProtocol trait and for now implements the methods by returning default values, the proper implementation will follow in subsequent commits.

• be026d014 Crypto,Interface(crypto): Use NiDkg-specific MasterPublicKeyId in NiDkgTag (#2620)
  Review: Looks fine + matches description
  Notes: Introduces the NiDkgMasterPublicKeyId enum (which currently only contains VetKd), allowing the NiDkg keys to avoid being mixed in with the other MasterPublicKeyId types (eg. Ecdsa and Schnorr).

• af542cdad Crypto,Interface(crypto): Extend NiDkgTag with HighThresholdForKey variant (#2445)
  Review: Looks fine + matches description
  Notes: Adds the HighThresholdForKey variant to the NiDkgTag which is used to instruct the NiDkg protocol to generate a transcript with a high threshold for a specified master public key.

• a3478d571 Execution,Interface: Evict sandboxes based on the available memory (#2651)
  Review: Looks fine + matches description
  Notes: Avoids evicting canister sandbox processes based on the total sandbox RSS unless the available subnet memory is below a threshold (currently 250GB).

• caca44da3 Execution,Interface: Add vetKeys related management canister endpoints (#2633)
  Review: Looks fine + matches description
  Notes: Adds method stubs for 3 new management canister functions, ReshareChainKey, VetKdPublicKey, and VetKdDeriveEncryptedKey. The implementations will follow in later commits.

• a1e516f92 Execution,Interface: Evict sandboxed based on scheduler priorities (#2590)
  Review: Looks fine + matches description
  Notes: Evict sandbox processes for idle canisters first, then evict those with the lowest scheduler priorities since those with higher priorities will be executed earlier.

• 826d9a503 Execution,Interface: Restrict legacy ICQC code to one subnet (#2647)
  Review: Looks fine + matches description
  Notes: Restricts the legacy ICQC feature to only the subnet that Distrikt is on since they are currently making use of it, this will allow other verified application subnets to be made public without worrying about more devs starting to use this soon to be deprecated feature.

• 77164cdf7 Execution,Interface: Penalize canisters at the end of the round (#2601)
  Review: Looks fine, but this commit itself doesn’t “Penalize canisters at the end of the round”, it is a follow-on commit required now that canisters are penalized at the end of each round.
  Notes: This commit modifies the scheduler such that during each reset round, each canister’s priority is set to equal its compute allocation rather than zero + adds a few minor optimisations.

• e391f4cdf Execution,Interface: Define and enforce callback limits (#1838)
  Review: Looks fine + matches description
  Notes: Adds a subnet wide limit on the number of pending callbacks (1M) and a per canister guaranteed minimum (50), so even if the subnet limit is exceeded, canisters only making a few calls are still able to proceed and only those canisters attempting to make many calls will have some rejected.

• 1ee1fe368 Execution,Interface: Don’t apply priority credit on abort (#2597)
  Review: Looks fine + matches description
  Notes: Avoids applying priority credit if a long-running call is aborted due to reaching a checkpoint, since now the LongExecutionMode is persisted across checkpoints and is used to ensure aborted tasks are retried after the checkpoint.

• f8c4eb15e Interface,Node(node): add node_reward_type to config tool (#2641)
  Review: Looks fine + matches description
  Notes: Adds the node_reward_type arg to the IC-OS config tool now that it has been added to the config.ini file.

• 1cebd7342 Node: Update GuestOS base images with 6.11 kernel (#2667)
  Review: Looks fine + matches description
  Notes: Bumps the GuestOS Linux Kernel version to 6.11.

• 4e46b92fc Node: Add node_reward_type to config.ini (#1787)
  Review: Looks fine + matches description
  Notes: Adds the node_reward_type arg to config.ini and updates various setup scripts to handle it + updates a few comments.

Bugfixes:

• 338b77f6c Consensus,Interface(consensus): Don’t increase the notary delay during upgrades (#2677)
  Review: Looks fine + matches description
  Notes: Avoids increasing the notary delay if the subnet is about to upgrade or is in the process of upgrading, since slowing down the block rate would only low down the upgrade.

• 545f2fad6 Consensus,Interface: Handle ChainKeyInitializations conversion errors (#2635)
  Review: Looks fine + matches description
  Notes: Make consistent all handling of ChainKeyInitialization failures so that they always return errors, rather than returning errors in some instances and panicking in others.

• 9870f1ea8 Execution,Interface: Correctly observe block maker to induction latency (#2663)
  Review: Looks fine + matches description
  Notes: Fixes the calculation of the message latencies which gets recorded in metrics for ingress messages which was previously relying on a field that wasn’t yet set.

• a078ed82b Interface: stop doing u32 subtraction in bitcoin headers validation (#2634)
  Review: Looks fine + matches description
  Notes: Fixes an underflow error in the Bitcoin header validation because on the testnet a block appeared which had a later timestamp than the subsequent block, it is infeasible that this would ever happen on the Bitcoin mainnet though.

• 2902a2a37 Interface,Networking: increase the keep alive probes. (#2713)
  Review: Looks fine + matches description
  Notes: Increases the quic heartbeat interval which checks if the connection is still alive from 200ms to 1 second.

• b679ad3a3 Interface,Networking: Remove the async lock in the AdapterMetricsRegistry (#2613)
  Review: Looks fine + matches description
  Notes: Avoids using an async lock within AdapterMetricsRegistry, and instead simply clones the Vec of AdapterMetrics which apparently is cheap to do so.

• 1c2346cab Node: HSM onboarding (#2636)
  Review: Looks fine + matches description
  Notes: Since the upgrade to Ubuntu 24.04 deployments via HSM have been broken, this commit fixes this by disabling polkit.

Benchmarking:

• 211a01015 Execution,Interface: Add criterion load benchmark (#2587)
  Review: Looks fine + matches description
  Notes: Adds the load_simulator_canisters_bench benchmark which runs may canisters which use timers and access stable memory to simulate load.

Chores:

• 9732c2c26 Consensus,Interface(consensus): Remove a no longer needed metric (#2679)
  Review: Looks fine + matches description
  Notes: Removes the missing_ingress_messages which was temporarily added to gather data for the hashes in blocks feature.

• 85c272c78 Consensus,Interface(consensus): Remove some clones from ingress selector (#2627)
  Review: Looks fine + matches description
  Notes: Small refactor to avoid a few clones within the IngressSelector.

• 5a5e0be15 Consensus,Interface(node): Add HostOS console message clarifying onboarding success (#2640)
  Review: Looks fine + matches description
  Notes: Adds a console message during onboarding to clarify a case which some node providers thought was successful when in fact it wasn’t yet.

• 87270d904 Consensus,Interface: Avoid handling VetKdKeyIds in the IDKG component (#2388)
  Review: Looks fine + matches description
  Notes: Introduces the IDkgMasterPublicKeyId type and then switches IDKG functions to take it as an arg rather than the pre-existing MasterPublicKeyId type which contains some variants which are not compatible with IDKG.

• 659d9b143 Execution,Interface: Fix default scheduler priority in evict (#2719)
  Review: Looks fine + matches description
  Notes: Sets the default schedular priority to i64::MIN rather than 0.

• 5813a429f Execution,Interface: Rename and test routing function for vetKD (#2680)
  Review: Looks fine + matches description
  Notes: Renames a few types from IDkgKey to ChainKey so that they can apply to any chain key type + adds some tests covering VetKD routing.

• 2f00d6815 Execution,Interface: Optimize evict_sandbox_processes (#2653)
  Review: Looks fine + matches description
  Notes: Exit the sandbox eviction process early, before creating the eviction candidates, if we detect than we don’t need to evict any.

• a10fbc291 Execution,Interface(fuzzing): Reconstruct system API imports using Wasmtime::Linker (#2575)
  Review: Looks fine + matches description
  Notes: In the fuzz tests, uses wasmtime::Linker to reconstruct the system API rather than manually typing the methods out.

• 0d14c11bd Execution,Interface,Message Routing: fixing typos and cleanups (#2682)
  Review: Looks fine + matches description
  Notes: Tiny refactor to make the reserve_cycles function more human-readable + fixes some typos in comments.

• 0eedbb674 Interface,Message Routing: Always use request metadata in backward compatibility test (#2690)
  Review: Looks fine + matches description
  Notes: Modifies a test to always have the request metadata populated because going forward the plan is to make this field non-optional.

• aacbed376 Interface,Message Routing: Trim canonical state framework (#2539)
  Review: Looks fine + matches description
  Notes: Removes old certification versions that are no longer supported leaving only V17, V18 and V19 + removes all the code required to define and handle these old versions.

• 7dd4dfe62 Interface,Networking: add a jaeger exporter package/crate (#2691)
  Review: Looks fine + matches description
  Notes: Adds the jaeger_exporter package for pushing telemetry data to Jaeger.

• 8b94d60b6 Interface,Networking: upgrade opentelemetry and num_cpus crates (#2656)
  Review: Looks fine + matches description
  Notes: Bumps opentelemetry to 0.27.0 and num_cpus to 1.16.0 + does a small refactor to work with the updated versions.

• 45dca07dd Interface,Networking: Do not shutdown the joinset when the event loop exists in the request handler (#2649)
  Review: Looks fine + matches description
  Notes: Avoids shutting down inflight requests if the bi-stream receives an error, also returns Ok rather than Err if send stream stops.

• 72ec446d6 Interface,Networking: Add additional logging in the case when the version message is not valid (#2596)
  Review: Looks fine + matches description
  Notes: Adds more details to the warning message output by the Bitcoin adapter when it receives an invalid version message.

• f96dec1db Interface,Node(node): Remove setupos/hostos network bonding and clean up docs (#2579)
  Review: Looks fine + matches description
  Notes: Removes generate_bond6_netdev_content during network setup since it is not needed + updates docs.

• c43bd9b0c Owners: Bump ic-cdk-timers (#2654)
  Review: Looks fine + matches description
  Notes: Bumps ic-cdk-timers from 0.7.0 to 0.11.0.

• 7dd4fd0f8 Node: Update Base Image Refs [2024-11-20-2036] (#2727)
  Review: Looks fine + matches description
  Notes: Updates the base IC-OS image references.

• c00695e45 Node: Update Base Image Refs [2024-11-20-0147] (#2705)
  Review: Looks fine + matches description
  Notes: Updates the base IC-OS image references.

• 0d127b8fd Node: Add log_and_halt_installation_on_error sleep (#2605)
  Review: Looks fine + matches description
  Notes: If setup fails, wait 5 seconds before exiting so that any log messages have time to be written.

• 4c6c5dae2 Node: Update Base Image Refs [2024-11-14-0808] (#2606)
  Review: Looks fine + matches description
  Notes: Updates the base IC-OS image references.

Refactoring:

• 81686f56f Crypto,Interface(crypto): Move ExtendedDerivationPath to ic_types::crypto (#2676)
  Review: Looks fine + matches description
  Notes: Moves ExtendedDerivationPath one level higher, out of the canister_threshold_sig module, since it will also be used for other sigs such as VetKeys.

• 3397eb8bf Crypto,Interface(crypto): Inline ThresholdSigDataStoreImpl::new_with_max_size (#2625)
  Review: Looks fine + matches description
  Notes: Removes ThresholdSigDataStoreImpl::new_with_max_size where the size could be chosen and instead always creates instances using the CAPACITY_PER_TAG_OR_KEY constant.

• dab484292 Interface,Node(node): rename use_nns_public_key and use_node_operator_private_key (#2700)
  Review: Looks fine + matches description
  Notes: Renames nns_public_key_exists to use_nns_public_key and node_operator_private_key_exists to use_node_operator_private_key within the IC-OS settings args.

• 03f3debdf Node: move node_index to ic.json (#1947)
  Review: Looks fine + matches description
  Notes: Defaults the node_index to 0 within ic.json rather than doing so in various Rust files.

Tests:

• d0acab2b2 Interface: Inline SNS system-tests (#2646)
  Review: Looks fine + matches description
  Notes: Moves the location of SNS system tests to the new sns_system_test_lib package.

Proposal 134251: Adopt

I have successfully run the build verification script and the 2 additional commits look fine, 6e2c0d9fb seems to be a duplicate of a3478d571, I’m not sure what exactly has happened here but either way the commit is fine, the other commit simply enables the HASHES_IN_BLOCKS_FEATURE_ENABLED feature flag.

Proposal 134259: Adopt

I have successfully run the build verification script and the only additional commit reverts the change which bumps the Linux Kernel to 6.11.

[ZackDS]

Proposal #134250 for release-2024-11-21_03-11-base.

Vote: Adopted
Reason: Builds fine and the hash matches for GUESTOS, HOSTOS and SETUPOS along with all the listed commits.

Review of commits

pending…

[ZackDS]

Proposal #134251 for release-2024-11-21_03-11-hashes-in-blocks.

Vote: Adopted
Reason: Builds fine and the hash matches for GUESTOS, HOSTOS and SETUPOS .

Review:

The two additional changes are:

Features:
6e2c0d9fb Execution,Interface: Evict sandboxes based on the available memory (#2651)

b16d21c2a Interface,Networking(Consensus): Enable the hashes-in-blocks feature

[ZackDS]

Proposal #134259 for release-2024-11-21_03-11-24.04-base-kernel.

Vote: Adopted
Reason: Builds fine and the hash matches for GUESTOS, HOSTOS and SETUPOS .

Review:

The only one additional change is:

Other changes:
5d2028948 Node: Revert “chore: Update Base Image Refs [2024-11-20-2036] (#2727)”

[yuvika]

Proposal 134250

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits
   
   

Commits

Features:

1. f3bb40251
   Summary: Instrument payload size and batch delivery interval.
   Notes: Add 3 histograms: batch_delivery_interval , payload_size_bytes and payload_section_size_bytes .
   Review: The description matches the code changes.
2. e44d2bc85 Crypto,Interface(crypto):
   Summary: create VetKdProtocol trait.
   Notes: Implement VetKdProtocol trait and some methods under CryptoReturningOk. The complete implementation will be added later.
   Review: The description matches the code changes.
3. be026d014
   Summary: Use NiDkg-specific MasterPublicKeyId in NiDkgTag.
   Notes: Add NiDkgMasterPublicKeyId enum to be explicit on which types of keys can be used with NiDkg.
   Review: The description matches the code changes.
4. af542cdad
   Summary: Extend NiDkgTag with HighThresholdForKey variant.
   Notes: Add new variant NiDkgTag::HighThresholdForKey that holds a MasterPublicKeyId.
   Review:
5. a3478d571
   Summary: Evict sandboxes based on the available memory.
   Notes: Avoid RSS-based eviction if there is sufficient memory (250 GiB or more). Added DEFAULT_MIN_MEM_AVAILABLE_TO_EVICT_SANDBOXES which is 250 GiB and added logic for sandbox eviction trigger_sandbox_eviction and other utilities.
   Review: The description matches the code changes.
6. caca44da3
   Summary: Add vetKeys related management canister endpoints.
   Notes: Add 3 new functions ReshareChainKey, VetKdPublicKey and VetKdEncryptedKey. All functions are unimplemented for now.
   Review: The description matches the code changes.
7. a1e516f92
   Summary: Evict sandboxed based on scheduler priorities.
   Notes: Evict Sandbox processes based on the lowest priority as this will decrease the number of cache misses.
   Review: The description matches the code changes.
8. 826d9a503
   Summary: Restrict legacy ICQC code to one subnet.
   Notes: Restrict the ICQC legacy feature to be only used by Distrikt until the migration is done.
   Review: The description matches the code changes.
9. 77164cdf7
   Summary: Penalize canisters at the end of the round.
   Notes: Modify the scheduler to set each canister’s priority equal its compute allocation and some minor optimizations.
   Review: The description matches the code changes. (check)
10. e391f4cdf
    Summary: Define and enforce callback limits.
    Notes: Set soft limit on the subnet-wide number of callbacks SUBNET_CALLBACK_SOFT_LIMIT = 1_000_000 and the number of callbacks that are guaranteed to each canister CANISTER_GUARANTEED_CALLBACK_QUOTA = 50 , virtually there is no practical limit for canisters except under a few extreme cases. (check)
    Review: The description matches the code changes.
11. 1ee1fe368
    Summary: Don’t apply priority credit on abort.
    Notes: Priority credit and LongExecutionMode are persisted and can be maintained across checkpoints hence, a priority credit doesn’t need to applied for long-running calls that reach a checkpoint. (check)
    Review: The description matches the code changes.
12. f8c4eb15e
    Summary: add node_reward_type to config tool.
    Notes: Add node_reward_type to the new config tool as it has already been added to config.ini.
    Review: The description matches the code changes.
13. 1cebd7342
    Summary: Update GuestOS base images with 6.11 kernel.
    Notes: Upgrade GuestOS Linux Kernel version to 6.11.
    Review: The description matches the code changes.
14. 4e46b92fc
    Summary: Add node_reward_type to config.ini.
    Notes: Add node_reward_type field to config.ini. SetupOS will return fail, if this field is left empty or is invalid. Some other scripts were also updated to incorporate this change.
    Review: The description matches the code changes.

Bugfixes:

1. 338b77f6c
   Summary: Don’t increase the notary delay during upgrades.
   Notes: Prevent adjusting the notary delay if a subnet is upgrading since slowing down the block rate would only delay the upgrade…
   Review: The description matches the code changes.
2. 545f2fad6
   Summary: Handle ChainKeyInitializations conversion errors.
   Notes: Improve error handling and make it consistent for inspect_idkg_chain_key_initializations ; always return an Error if something unexpected happens.
   Review: The description matches the code changes.
3. 9870f1ea8
   Summary: Correctly observe block maker to induction latency.
   Notes: Correctly calculate the message latency using the current block time instead of the received_time which wasn’t populated.
   Review: The description matches the code changes.
4. a078ed82b
   Summary: stop doing u32 subtraction in bitcoin headers validation.
   Notes: Fix u32 subtraction underflow issue, caused due to a recent testnet block was having a smaller timestamp than an older block.
   Review: The description matches the code changes.
5. 2902a2a37
   Summary: increase the keep alive probes.
   Notes: Increase KEEP_ALIVE_INTERVAL to from 200 ms to 1 sec, this is the interval to check if the connection is idle for more than 1 sec.
   Review: The description matches the code changes.
6. b679ad3a3
   Summary: Remove the async lock in the AdapterMetricsRegistry.
   Notes: Clone the Vec of AdapterMetrics instead of using async mutex since that is cheaper.
   Review: The description matches the code changes.
7. 1c2346cab
   Summary: HSM onboarding.
   Notes: This issue was introduced with the Ubuntu 24.04 upgrade, disable polkit to fix HSM onboardings.
   Review: The description matches the code changes.Benchmarking:
   211a01015
   Summary: Add criterion load benchmark
   Notes: Add a new benchmark: load_simulator_canisters.rs which is useful for debugging and benchmarking scheduler and sandbox eviction changes.
   Review: The description matches the code changes.

Chores:

1. 9732c2c26
   Summary: Remove a no longer needed metric.
   Notes: Remove Histogram missing_ingress_messages , as it was only temporarily added to get some additional metrics for the hashes in blocks feature.
   Review: The description matches the code changes.
2. 85c272c78
   Summary: Remove some clones from ingress selector.
   Notes: Improve performance by removing call to .clone() from IngressSelector .
   Review: The description matches the code changes.
3. 5a5e0be15
   Summary: Add HostOS console message clarifying onboarding success.
   Notes: Improve logging and clarify the onboarding message.
   Review: The description matches the code changes.
4. 87270d904
   Summary: Avoid handling VetKdKeyIds in the IDKG component .
   Notes: Add a new type IDkgMasterPublicKeyId which wraps MasterPublicKeyId (passing it as an arg instead of the pre-existing type) and uses only an IDKG compatible variant. The key only needs to be checked once, and after that it is guaranteed to be IDKG compatible key.
   Review: The description matches the code changes.
5. 659d9b143
   Summary: Fix default scheduler priority in evict.
   Notes: Set the default scheduler priority min_scheduler_priority to i64::MIN rather than 0.
   Review: The description matches the code changes.
6. 5813a429f
   Summary: Rename and test routing function for vetKD.
   Notes: Rename functions and types form IDKG to Chain Key , this also applies to vetKD. Some tests were also updated.
   Review: The description matches the code changes.
7. 2f00d6815
   Summary: Optimize evict_sandbox_processes.
   Notes: Check if evicting any candidate is necessary before creating the Candidates vector.
   Review: The description matches the code changes.
8. a10fbc291
   Summary: Reconstruct system API imports using Wasmtime::Linker.
   Notes: Use Wasmtime::Linker to reconstruct system APIs instead of using a static list in the fuzz tests.
   Review: The description matches the code changes.
9. 0d14c11bd
   Summary: fixing typos and cleanups.
   Notes: Fix typos and refactor reserve_cycles method.
   Review: The description matches the code changes.
10. 0eedbb674
    Summary: Always use request metadata in backward compatibility test.
    Notes: This change is to support making the request metadata non optional, which will be implemented.
    Review: The description matches the code changes.
11. aacbed376
    Summary: Trim canonical state framework.
    Notes: Remove exceptions for unsupported certification versions (V16 and below) and clean up testing framework.
    Review: The description matches the code changes.
12. 7dd4dfe62
    Summary: add a jaeger exporter package/crate.
    Notes: Add jaeger_exporter package.
    Review: The description matches the code changes.
13. 8b94d60b6
    Summary: upgrade opentelemetry and num_cpus crates.
    Notes: Upgrade opentelemetry from 0.26.0 to 0.27.0 and num_cpus from 1.13.1 to 1.16.0 and some refactor to work with the upgraded versions.
    Review: The description matches the code changes.
14. 45dca07dd
    Summary: Do not shutdown the joinset when the event loop exists in the request handler.
    Notes: Returns Ok instead of an error is send_stream stops and if there is an error accepting bi streams, it just logs an error and doesn’t shut down the request handler.
    Review: The description matches the code changes.
15. 72ec446d6
    Summary: Add additional logging in the case when the version message is not valid.
    Notes: Improve logging for the Bitcoin adapter when an invalid version message is received.
    Review: The description matches the code changes.
16. f96dec1db
    Summary: Remove setupos/hostos network bonding and clean up docs.
    Notes: Remove generate_bond6_netdev_content since it added unnecessary complexity and update README.md.
    Review: The description matches the code changes.
17. c43bd9b0c
    Summary: Bump ic-cdk-timers.
    Notes: Upgrade ic-cdk-timers from 0.7.0 to 0.11.0.
    Review: The description matches the code changes.
18. 7dd4fd0f8
    Summary: Update Base Image Refs [2024-11-20-2036].
    Notes: Update base container image references.
    Review: The description matches the code changes.
19. c00695e45
    Summary: Update Base Image Refs [2024-11-20-0147] .
    Notes: Update base container image references.
    Review: The description matches the code changes.
20. 0d127b8fd
    Summary: Add log_and_halt_installation_on_error sleep.
    Notes: If there are any issues with the setup and it fails, before the exit is called add a sleep for 5s for all the logs to be written and displayed on the console.
    Review: The description matches the code changes.
21. 4c6c5dae2
    Summary: Update Base Image Refs [2024-11-14-0808].
    Notes: Update base container image references.
    Review: The description matches the code changes.

Refactoring:

1. 81686f56f
   Summary: Move ExtendedDerivationPath to ic_types::crypto
   Notes: Move ExtendedDerivationPath from ic_types::crypto::canister_threshold_sig to ic_types::crypto as it will be used for other features as well such as the vetKeys.
   Review: The description matches the code changes.
2. 3397eb8bf
   Summary: Inline ThresholdSigDataStoreImpl::new_with_max_size
   Notes: Create new instances using CAPACITY_PER_TAG_OR_KEY const instead of using ThresholdSigDataStoreImpl::new_with_max_siz with a chosen size.
   Review: The description matches the code changes.
3. dab484292
   Summary: rename use_nns_public_key and use_node_operator_private_key
   Notes: Rename nns_public_key_exists to use_nns_public_key and node_operator_private_key_exists to use_node_operator_private_key .
   Review: The description matches the code changes.
4. 03f3debdf
   Summary: move node_index to ic.json
   Notes: Set node_index to 0 within ic.json instead of doing it in various other places.
   Review: The description matches the code changes.

Tests:

1. d0acab2b2
   Summary: Inline SNS system-tests (#2646) Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.
   Notes: Create new sns_system_test_lib package and move the SNS system-tests there.
   Review: The description matches the code changes.

Proposal 134251

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits
   
   

Commits

Features:

1. 6e2c0d9fb
   Summary: Evict sandboxes based on the available memory.
   Notes: Duplicate of a3478d571.
   Review: The description matches the code changes.
2. b16d21c2a
   Summary: Enable the hashes-in-blocks feature.
   Notes: Set HASHES_IN_BLOCKS_FEATURE_ENABLED to true, thereby enabling the hashes in blocks feature.
   Review: The description matches the code changes.

Proposal 134259

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits
   
   

Commits

Other changes:

1. 5d2028948
   Summary: Revert "chore: Update Base Image Refs [2024-11-20-2036].
   Notes: Revert commit 7dd4fd0.
   Review: The description matches the code changes.

[ipsita]

Proposal 134250

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Commits

Features

• f3bb40251
  Summary: Instrument payload size and batch delivery interval
  Notes: Added three new metrics to the IngressHistoryWriterImpl structure: batch_delivery_interval measures the time gap between two consecutive batch deliveries, payload_size_bytes records the size of the consensus block payload, and payload_section_size_bytes tracks the size of specific sections in the payload.
  Review: Code changes matches the commit message.

• e44d2bc85
  Summary: Create VetKdProtocol trait
  Notes: Introduced a new VetKdProtocol trait, implemented for the CryptoReturningOk structure, designed to return generic values in cryptographic operations.
  Review: Code changes matches the commit message.

• be026d014
  Summary: Use NiDkg-specific MasterPublicKeyId in NiDkgTag
  Notes: Introduced NiDkgMasterPublicKeyId as a distinct enum to eliminate dependencies on ECDSA or Schnorr in the context of NI-DKG. This simplifies the implementation and avoids unnecessary matching of unrelated key types.
  Review: Code changes matches the commit message.

• af542cdad
  Summary: Extend NiDkgTag with HighThresholdForKey variant
  Notes: Added a HighThresholdForKey variant to the NiDkgTag implementation. This enhancement facilitates the return of transcript configurations for the vetkeys feature.
  Review: Code changes matches the commit message.

• a3478d571
  Summary: Evict sandboxes based on the available memory
  Notes: Sandbox eviction logic now considers available memory before evicting canister sandbox processes. Evictions occur only when the subnet’s available memory drops below 250GB, ensuring efficient memory usage.
  Review: Code changes matches the commit message.

• caca44da3
  Summary: Add vetKeys-related management canister endpoints
  Notes: Added new vetkeys management canister endpoints: ReshareChainKey, VetKdPublicKey, and VetKdDeriveEncryptedKey. Implementation details will follow in future proposals.
  Review: Code changes matches the commit message.

• a1e516f92
  Summary: Evict sandboxes based on scheduler priorities
  Notes: Adjusted sandbox eviction logic to align with scheduler priorities. This update enhances the cache hit rate by propagating scheduler priorities during the eviction process and ensuring lower-priority sandboxes are evicted first.
  Review: Code changes matches the commit message.

• 826d9a503
  Summary: Restrict legacy ICQC code to one subnet
  Notes: The legacy ICQC code, now deprecated and primarily used by the Distrikt app, is restricted to a single subnet. This change ensures the code is not unnecessarily available across all verified subnets.
  Review: Code changes matches the commit message.

• 77164cdf7
  Summary: Penalize canisters at the end of the round
  Notes: Modified logic to allocate canisters’ accumulated priorities to their compute allocations at the end of each round. Additional minor updates were made to improve the proper implementation of penalization mechanisms.
  Review: Code changes matches the commit message.

• e391f4cdf
  Summary: Define and enforce callback limits
  Notes: Introduced new limits for subnet-wide and per-canister callbacks to manage memory usage effectively. Subnet-wide callbacks are capped at 1M, while each canister is guaranteed a minimum quota of 50 callbacks, ensuring sufficient resources even during heavy usage or attacks.
  Review: Code changes matches the commit message.

• 1ee1fe368
  Summary: Don’t apply priority credit on abort
  Notes: Updated logic to retain LongExecutionMode across checkpoints, ensuring aborted tasks retire properly after the checkpoint. This eliminates the need to apply priority credits to long-running calls that were aborted due to checkpoint constraints.
  Review: Code changes matches the commit message.

• f8c4eb15e
  Summary: Add node_reward_type to config tool
  Notes: Introduced a new configuration option, node_reward_type, in ic_os. This addition is expected to be utilized in subsequent updates.
  Review: Code changes matches the commit message.

• 1cebd7342
  Summary: Update GuestOS base images with 6.11 kernel
  Notes: The GuestOS kernel version has been upgraded to 6.11.
  Review: Code changes matches the commit message.

• 4e46b92fc
  Summary: Add node_reward_type to config.ini
  Notes: The node_reward_type configuration has been added to setup_os for use during node installation.
  Review: Code changes matches the commit message.

Bugfixes

• 338b77f6c
  Summary: Don’t increase the notary delay during upgrades.
  Notes: Ensures that the notary delay is not adjusted when a subnet is undergoing an upgrade, as reducing the block rate would unnecessarily extend the upgrade duration.
  Review: Code changes matches the commit message.

• 545f2fad6
  Summary: Handle ChainKeyInitializations conversion errors.
  Notes: Enhances error handling in inspect_idkg_chain_key_initializations to ensure it consistently returns an error for any unexpected scenario.
  Review: Code changes matches the commit message.

• 9870f1ea8
  Summary: Correctly observe block maker to induction latency.
  Notes: Updates latency calculations to use the current block time instead of received_time, which was previously not populated, ensuring accurate latency measurements.
  Review: Code changes matches the commit message.

• a078ed82b
  Summary: Stop doing u32 subtraction in Bitcoin headers validation.
  Notes: Fixes an issue with u32 subtraction underflow that occurred due to a recent testnet block having a smaller timestamp than a previous block.
  Review: Code changes matches the commit message.

• 2902a2a37
  Summary: Increase the keep-alive probes.
  Notes: Adjusts KEEP_ALIVE_INTERVAL from 200ms to 1 second, setting the interval for checking if a connection has been idle beyond the one-second threshold.
  Review: Code changes matches the commit message.

• b679ad3a3
  Summary: Remove the async lock in the AdapterMetricsRegistry.
  Notes: Replaces the use of an async mutex by cloning the Vec of AdapterMetrics, reducing computational overhead and improving efficiency.
  Review: Code changes matches the commit message.

• 1c2346cab
  Summary: HSM onboarding.
  Notes: Resolves an issue introduced with the Ubuntu 24.04 upgrade by disabling polkit, addressing problems with HSM onboarding.
  Review: Code changes matches the commit message.

Benchmarking

• 211a01015
  Summary: Add criterion load benchmark.
  Notes: Introduces a new benchmark, load_simulator_canisters.rs, designed to assist with debugging and evaluating the performance of scheduler and sandbox eviction updates.
  Review: Code changes matches the commit message.

Chores

• 9732c2c26
  Summary: Remove a no longer needed metric.
  Notes: Eliminates the missing_ingress_messages histogram, which was introduced temporarily for monitoring the hashes-in-blocks feature.
  Review: Code changes matches the commit message.

• 85c272c78
  Summary: Remove some clones from ingress selector.
  Notes: Enhances performance by removing unnecessary .clone() calls in the IngressSelector implementation.
  Review: Code changes matches the commit message.

• 5a5e0be15
  Summary: Add HostOS console message clarifying onboarding success.
  Notes: Refines logging output to clearly indicate when the onboarding process is successful.
  Review: Code changes matches the commit message.

• 87270d904
  Summary: Avoid handling VetKdKeyIds in the IDKG component.
  Notes: Introduces IDkgMasterPublicKeyId to encapsulate MasterPublicKeyId, ensuring it is exclusively compatible with IDKG and simplifying key validation.
  Review: Code changes matches the commit message.

• 659d9b143
  Summary: Fix default scheduler priority in evict.
  Notes: Adjusts the default scheduler priority to use i64::MIN instead of 0 to ensure accurate eviction behavior.
  Review: Code changes matches the commit message.

• 5813a429f
  Summary: Rename and test routing function for vetKD.
  Notes: Updates function and type names from IDKG to ChainKey, including for vetKD, and modifies relevant tests to align with these changes.
  Review: Code changes matches the commit message.

• 2f00d6815
  Summary: Optimize evict_sandbox_processes.
  Notes: Improves efficiency by checking whether eviction is necessary before creating the Candidates vector.
  Review: Code changes matches the commit message.

• a10fbc291
  Summary: Reconstruct system API imports using Wasmtime::Linker.
  Notes: Replaces the static list of system APIs with dynamic reconstruction using Wasmtime::Linker in fuzz tests.
  Review: Code changes matches the commit message.

• 0d14c11bd
  Summary: Fixing typos and cleanups.
  Notes: Resolves typos and simplifies the reserve_cycles method for improved readability and maintainability.
  Review: Code changes matches the commit message.

• 0eedbb674
  Summary: Always use request metadata in backward compatibility test.
  Notes: Introduces changes to ensure request metadata is consistently included, supporting future changes to make it mandatory.
  Review: Code changes matches the commit message.

• aacbed376
  Summary: Trim canonical state framework.
  Notes: Removes exceptions for unsupported certification versions (V16 and earlier) and cleans up the associated testing framework.
  Review: Code changes matches the commit message.

• 7dd4dfe62
  Summary: Add a Jaeger exporter package/crate.
  Notes: Adds a new package named jaeger_exporter for tracing and observability purposes.
  Review: Code changes matches the commit message.

• 8b94d60b6
  Summary: Upgrade opentelemetry and num_cpus crates.
  Notes: Updates opentelemetry to version 0.27.0 and num_cpus to version 1.16.0, making necessary adjustments to maintain compatibility.
  Review: Code changes matches the commit message.

• 45dca07dd
  Summary: Do not shut down the joinset when the event loop exits in the request handler.
  Notes: Ensures stability by logging errors instead of shutting down the request handler upon encountering issues in send_stream or bidirectional stream acceptance.
  Review: Code changes matches the commit message.

• 72ec446d6
  Summary: Add additional logging in the case when the version message is not valid.
  Notes: Improves error visibility for the Bitcoin adapter by adding detailed logs for invalid version messages.
  Review: Code changes matches the commit message.

• f96dec1db
  Summary: Remove setupos/hostos network bonding and clean up docs.
  Notes: Eliminates the generate_bond6_netdev_content method to reduce complexity and updates the README.md for clarity.
  Review: Code changes matches the commit message.

• c43bd9b0c
  Summary: Bump ic-cdk-timers.
  Notes: Upgrades ic-cdk-timers from version 0.7.0 to 0.11.0.
  Review: Code changes matches the commit message.

• 7dd4fd0f8
  Summary: Update Base Image Refs [2024-11-20-2036].
  Notes: Refreshes the references to base container images for compatibility and stability.
  Review: Code changes matches the commit message.

• c00695e45
  Summary: Update Base Image Refs [2024-11-20-0147].
  Notes: Updates references to the base images to reflect the latest changes.
  Review: Code changes matches the commit message.

• 0d127b8fd
  Summary: Add log_and_halt_installation_on_error sleep.
  Notes: Introduces a 5-second delay in case of setup errors, allowing logs to fully propagate before exiting.
  Review: Code changes matches the commit message.

• 4c6c5dae2
  Summary: Update Base Image Refs [2024-11-14-0808].
  Notes: Adjusts base container image references to match the latest updates.
  Review: Code changes matches the commit message.

Refactoring

• 81686f56f
  Summary: Move ExtendedDerivationPath to ic_types::crypto
  Notes: Relocates ExtendedDerivationPath from ic_types::crypto::canister_threshold_sig to ic_types::crypto to make it accessible for additional features like vetKeys.
  Review: Code changes matches the commit message.

• 3397eb8bf
  Summary: Inline ThresholdSigDataStoreImpl::new_with_max_size
  Notes: Replaces the use of ThresholdSigDataStoreImpl::new_with_max_size by initializing new instances directly with the CAPACITY_PER_TAG_OR_KEY constant.
  Review: Code changes matches the commit message.

• dab484292
  Summary: Rename use_nns_public_key and use_node_operator_private_key
  Notes: Updates the variable names: nns_public_key_exists is now use_nns_public_key, and node_operator_private_key_exists is renamed to use_node_operator_private_key.
  Review: Code changes matches the commit message.

• 03f3debdf
  Summary: Move node_index to ic.json
  Notes: Configures node_index to default to 0 in ic.json, consolidating this initialization in one place.
  Review: Code changes matches the commit message.

Tests

• d0acab2b2
  Summary: Inline SNS system-tests
  Notes: Introduces a new sns_system_test_lib package and transfers the SNS system-tests into this package for better organization. The full list of changes, including those unrelated to GuestOS, is available on GitHub.
  Review: Code changes matches the commit message.

Proposal 134251

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Commits

Features

• 6e2c0d9fb
  Summary: Evict sandboxes based on the available memory
  Notes
  Avoid RSS based eviction if there is sufficient memory (250 GiB or
  more) and instead evict sandboxes based on the available memory since RSS based eviction can negatively impact sandbox cache hit ratio.
  Review
  Code changes matches the commit message.

• b16d21c2a
  Summary: Enable the hashes-in-blocks feature
  Notes
  HASHES_IN_BLOCKS_FEATURE_ENABLED feature flag is set to true. When the feature flag is enabled we strip all ingress messages from the block before sending it to peer, on the reciever side these ingress messages are reconstructed from the ingress pool and if the message is not found in the ingress pool we fetch it from the peer which is advertising that message.
  Review
  Code changes matches the commit message.

Proposal 134259

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Commits

Other changes

• 5d2028948
  Summary: Revert "chore: Update Base Image Refs [2024-11-20-2036]
  Notes: Revert of commit 7dd4fd0 which updated base image references.
  Review: Code changes matches the commit message.

[zenithcode]

Proposal: 134250

Summary:

1. Build Hash: Build has from the proposal, local build and CDN matches and is “4f497704a89f88beb7aa8d541c44beb72f5dd68a339170430fbc4e9c8515cae3”.
2. Summary: The release notes matches the code changes
3. Vote: I vote to adopt the proposals

Detailed Review:

Features:

• f3bb40251 Consensus,Interface: Instrument payload size and batch delivery interval (#2669)
  Notes: Three more metrics have been added. batch_delivery_interval for Time elapsed since the delivery of the previous batch, in seconds , payload_size_bytes for the size of consensus block payload size in bytes, and payload_section_size_bytes for the size of consensys payload section in bytes.
  Review: Code changes look good and match release notes.
• e44d2bc85 Crypto,Interface(crypto): create VetKdProtocol trait (#2569)
  Notes: vetKdProtocol trait has meen introduced and implemented for CryptoReturningOk returning generic values.
  Review: Code changes look good and match release notes.
• be026d014 Crypto,Interface(crypto): Use NiDkg-specific MasterPublicKeyId in NiDkgTag (#2620)
  Notes: MasterPublicKeyId has been introduced as a dedicated enum for NI-DKG aks NiDkgMasterPublicKeyId. This has been done to avoid the need to match on Ecdsa or Schnorr in the context of NI-DKG.
  Review: Code changes look good and match release notes.
• af542cdad Crypto,Interface(crypto): Extend NiDkgTag with HighThresholdForKey variant (#2445)
  Notes: HighThresholdForKey variant has been added to NiDkgTag implementation which will be used to return transcript configs for vetkeys.
  Review: Code changes look good and match release notes.
• a3478d571 Execution,Interface: Evict sandboxes based on the available memory (#2651)
  Notes: The total RSS will not be used to evict canister sandbox process unless the available memory of the subnet is below 250GB ( current implementation ).
  Review: Code changes look good and match release notes.
• caca44da3 Execution,Interface: Add vetKeys related management canister endpoints (#2633)
  Notes: Added a few vetkeys related functions in management canister. ReshareChainKey, VetKdPublicKey, VetKdDeriveEncryptedKey. The implementation will happen in further proposals.
  Review: Code changes look good and match release notes.
• a1e516f92 Execution,Interface: Evict sandboxed based on scheduler priorities (#2590)
  Notes: Currently the canister scheduling and sandbox eviction logic had a mismatch. To improve the cache hit rate significantly to 75% changes have been made to propodate scheduler priorities as well where the sanddbox process are evicted and do evictions based on lowers priority.
  Review: Code changes look good and match release notes.
• 826d9a503 Execution,Interface: Restrict legacy ICQC code to one subnet (#2647)
  Notes: The legacy ICQC code which is the predecessor of composite query is now restricted to only one subnet, since Distrikt is the only dapp using those code. This is anyways deprecated thats is why have no sense to be available in all the verified app subnets.
  Review: Code changes look good and match release notes.
• 77164cdf7 Execution,Interface: Penalize canisters at the end of the round (#2601)
  Notes: This PR adds the functionality to allocate the canister accumulated priority to its compute allocation rather than 0. There are a few other minor changes too.
  Review: Code changes look good and but the release note of PR could have been proper stating that this change is necessary to properly implement the penalization on canisters at each round…
• e391f4cdf Execution,Interface: Define and enforce callback limits (#1838)
  Notes: New call backs limits are defined and enforced. Now the subnet wide callbacks are set to 1M in order to limit the memory usage. Per-canister guaranteed callback quota is set to 50. Thats how even in the case of high usage / ddos canisters will be able to make sufficient concurrent calls.
  Review: Code changes look good and match release notes.
• 1ee1fe368 Execution,Interface: Don’t apply priority credit on abort (#2597)
  Notes: Across checkpoints LongExecutionMode is persisted to ensure retirement of aborted task after the checkpoint, hance there is now no need of applying priority credit to a long running call aborted due to reaching the checkpoint.
  Review: Code changes look good and match release notes.
• f8c4eb15e Interface,Node(node): add node_reward_type to config tool (#2641)
  Notes: A new config is added to ic_os which is node_reward_type. This might be used later on in further proposals.
  Review: Code changes look good and match release notes.
• 1cebd7342 Node: Update GuestOS base images with 6.11 kernel (#2667)
  Notes: The GuestOs Kernal has been bumped to 6.11
  Review: Code changes look good and match release notes.
• 4e46b92fc Node: Add node_reward_type to config.ini (#1787)
  Notes: node_reward_type is also added to setup OS, which will not be used while installing the node ic-os.
  Review: Code changes look good and match release notes.

Bugfixes:

• 338b77f6c Consensus,Interface(consensus): Don’t increase the notary delay during upgrades (#2677)
  Notes: The noterly delay ( finality_adjusted_delay ) is now removed, as it adds a further delay in the subnet upgrade process. By design the execution halt at the CUP height while consensus delivers empty blocks.
  Review: Code changes look good and match release notes.
• 545f2fad6 Consensus,Interface: Handle ChainKeyInitializations conversion errors (#2635)
  Notes: The PR addresses consistency in handling chainkeyinitialization conversion errors so as to always return errors instead of unexpected panic via expect.
  Review: Code changes look good and match release notes.
• 9870f1ea8 Execution,Interface: Correctly observe block maker to induction latency (#2663)
  Notes: The PR addresses a bug of received_time being used before its been set when a new ingress message was inducted. Changes are done in IngressHistoryWriterImpl.
  Review: Code changes look good and match release notes.
• a078ed82b Interface: stop doing u32 subtraction in bitcoin headers validation (#2634)
  Notes: On testnet a block appeared which had a higher timestamp than a subsequent block. That is why the u32 subtraction in bitcoin header underflew. Changes are made to fix this. This is never anticipated to happen on mainnet.
  Review: Code changes look good and match release notes.
• 2902a2a37 Interface,Networking: increase the keep alive probes. (#2713)
  Notes: The keep alive probes interval has been increased from 200ms to 1sec in Quic heartbeats.
  Review: Code changes look good and match release notes.
• b679ad3a3 Interface,Networking: Remove the async lock in the AdapterMetricsRegistry (#2613)
  Notes: Async lock is removed in AdapterMetricsRegistry and simply use the clones of Vec of AdpaterMetrics as it is cheap.
  Review: Code changes look good and match release notes.
• 1c2346cab Node: HSM onboarding (#2636)
  Notes: Since the upgraded pcscd was switched from using D-bus to polkit, the HSM onboarding was broken in Ubuntu upgrade from 20.04 to 24.04. Its a quick fix to disable polkit.
  Review: Code changes look good and match release notes.

Benchmarking:

• 211a01015 Execution,Interface: Add criterion load benchmark (#2587)
  Notes: load_sumilator has been added using canister_creator to measure total time taken to complete the rounds and the throughput.
  Review: Code changes look good and match release notes.

Chores:

• 9732c2c26 Consensus,Interface(consensus): Remove a no longer needed metric (#2679)
  Notes: Minor metric has been removed ( missing_ingress_messages )
  Review: Code changes look good and match release notes.
• 85c272c78 Consensus,Interface(consensus): Remove some clones from ingress selector (#2627)
  Notes: Minor changes to avoid clones of IngressSelector.
  Review: Code changes look good and match release notes.
• 5a5e0be15 Consensus,Interface(node): Add HostOS console message clarifying onboarding success (#2640)
  Notes: A minor messaging change to change HostOS clarifying onboarding success.
  Review: Code changes look good and match release notes.
• 87270d904 Consensus,Interface: Avoid handling VetKdKeyIds in the IDKG component (#2388)
  Notes: This update introduces the IDkgMasterPublicKeyId type and modifies IDKG functions to use it as an argument instead of the existing MasterPublicKeyId type. The change ensures compatibility, as MasterPublicKeyId includes variants that are not supported by IDKG.
  Review: Code changes look good and match release notes.
• 659d9b143 Execution,Interface: Fix default scheduler priority in evict (#2719)
  Notes: Default scheduler priority has been changed to i64::MIN) from 0.
  Review: Code changes look good and match release notes.
• 5813a429f Execution,Interface: Rename and test routing function for vetKD (#2680)
  Notes: Renames several types from IDkgKey to ChainKey to generalize their applicability to any chain key type. Also, adds tests to cover VetKD routing.
  Review: Code changes look good and match release notes.
• 2f00d6815 Execution,Interface: Optimize evict_sandbox_processes (#2653)
  Notes: Code changes have been made for early eviction of sandbox process.
  Review: Code changes look good and match release notes.
• a10fbc291 Execution,Interface(fuzzing): Reconstruct system API imports using Wasmtime::Linker (#2575)
  Notes: In the fuzz tests, the wasmtime::Linker was utilized to reconstruct the system API instead of manually defining the methods.
  Review: Code changes look good and match release notes.
• 0d14c11bd Execution,Interface,Message Routing: fixing typos and cleanups (#2682)
  Notes: Minorrefactor has been made to reserve_cycles function to make it more readable plus there are some more typo fixes.
  Review: Code changes look good and match release notes.
• 0eedbb674 Interface,Message Routing: Always use request metadata in backward compatibility test (#2690)
  Notes: Updates a test to ensure the request metadata is always populated, aligning with the plan to make this field mandatory moving forward.
  Review: Code changes look good and match release notes.
• aacbed376 Interface,Message Routing: Trim canonical state framework (#2539)
  Notes: Removes outdated certification versions no longer supported, retaining only V17, V18, and V19. Additionally, eliminates all code associated with defining and managing these deprecated versions.
  Review: Code changes look good and match release notes.
• 7dd4dfe62 Interface,Networking: add a jaeger exporter package/crate (#2691)
  Notes: Minor change to add jaeger_exporter package to push telemetry data.
  Review: Code changes look good and match release notes.
• 8b94d60b6 Interface,Networking: upgrade opentelemetry and num_cpus crates (#2656)
  Notes: Minor bumps to opentelemetry to 0.27.0 and num_cups to 1.16.0, additionally some refactor to for these upgrades.
  Review: Code changes look good and match release notes.
• 45dca07dd Interface,Networking: Do not shutdown the joinset when the event loop exists in the request handler (#2649)
  Notes: Prevents shutting down in-flight requests when the bi-directional stream encounters an error and ensures an Ok response is returned instead of an Err if the send stream halts.
  Review: Code changes look good and match release notes.
• 72ec446d6 Interface,Networking: Add additional logging in the case when the version message is not valid (#2596)
  Notes: Enhances the warning message displayed by the Bitcoin adapter when it encounters an invalid version message, providing additional context and details to aid in debugging and issue resolution.
  Review: Code changes look good and match release notes.
• f96dec1db Interface,Node(node): Remove setupos/hostos network bonding and clean up docs (#2579)
  Notes: Removes generate_bond6_netdev_content from the network setup process as it is unnecessary, and updates the documentation accordingly.
  Review: Code changes look good and match release notes.
• c43bd9b0c Owners: Bump ic-cdk-timers (#2654)
  Notes: ic-cdk-timers have been bumped from 0.7.0 to 0.11.0
  Review: Code changes look good and match release notes.
• 7dd4fd0f8 Node: Update Base Image Refs [2024-11-20-2036] (#2727)
  Notes: Updates to base image ref.
  Review: Code changes look good and match release notes.
• c00695e45 Node: Update Base Image Refs [2024-11-20-0147] (#2705)
  Notes: Updates to base image ref.
  Review: Code changes look good and match release notes.
• 0d127b8fd Node: Add log_and_halt_installation_on_error sleep (#2605)
  Notes: If the setup fails, pause for 5 seconds before exiting to allow sufficient time for log messages to be recorded.
  Review: Code changes look good and match release notes.
• 4c6c5dae2 Node: Update Base Image Refs [2024-11-14-0808] (#2606)
  Notes: base image ref update
  Review: Code changes look good and match release notes.

Refactoring:

• 81686f56f Crypto,Interface(crypto): Move ExtendedDerivationPath to ic_types::crypto (#2676)
  Notes: Moves ExtendedDerivationPath one level higher, relocating it out of the canister_threshold_sig module, as it will also be utilized for other signatures, such as VetKeys.
  Review: Code changes look good and match release notes.
• 3397eb8bf Crypto,Interface(crypto): Inline ThresholdSigDataStoreImpl::new_with_max_size (#2625)
  Notes: Replaces ThresholdSigDataStoreImpl::new_with_max_size, which allowed specifying a custom size, with a simplified approach that always uses the predefined CAPACITY_PER_TAG_OR_KEY constant for instance creation.
  Review: Code changes look good and match release notes.
• dab484292 Interface,Node(node): rename use_nns_public_key and use_node_operator_private_key (#2700)
  Notes: Renames nns_public_key_exists to use_nns_public_key and node_operator_private_key_exists to use_node_operator_private_key within the IC-OS settings arguments.
  Review: Code changes look good and match release notes.
• 03f3debdf Node: move node_index to ic.json (#1947)
  Notes: Sets the default node_index to 0 in ic.json instead of handling it separately across multiple Rust files.
  Review: Code changes look good and match release notes.

Tests:

d0acab2b2 Interface: Inline SNS system-tests (#2646) Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.
Notes: Relocates SNS system tests to the newly created sns_system_test_lib package.
Review: Code changes look good and match release notes.

Proposal: 134251

Summary:

1. Build Hash: Build has from the proposal, local build and CDN matches and is “791f412858fd3197e1b27c675726d0910b9d7731eabac8fc877662ab834c8853”.
2. Summary: The release notes matches the code changes
3. Vote: I vote to adopt the proposals
   
   

Detailed Review:

Features:

• 6e2c0d9fb Execution,Interface: Evict sandboxes based on the available memory (#2651)
• b16d21c2a Interface,Networking(Consensus): Enable the hashes-in-blocks feature

Proposal: 134259

1. Build Hash: Build has from the proposal, local build and CDN matches and is “1b482cefe1eae8deb269988beb78bc779fa2a87f16eb05faf611db168a7aa1fd”.
2. Summary: The release notes matches the code changes
3. Vote: I vote to adopt the proposals

Other changes:

• 5d2028948 Node: Revert “chore: Update Base Image Refs [2024-11-20-2036] (#2727)”

[cyberowl]

proposal - 134250

Vote: ADOPT

Reason:

Although I have some concerns over the logging of the keys, I do think it is intentional and the author has good reason to do it for commit e44d2bc85. The build was successful and all the changes align with their commit description.

Hash Match: MATCH

Feedback:

feat(crypto): CRP-2596 create VetKdProtocol trait (#2569)

Proposer Check: MATCH

Overall Summary:

More changes related to vetkd and introduction of new metrics for batch delivery intervals, payload sizes, and sandbox eviction logic improvements to refine memory management. Enhancements to priority-based scheduling and callback management optimize performance and resource allocation across the network.

Commits Summary

f3bb40251
New metrics are introduced to measure batch delivery intervals, payload sizes, and sizes of payload sections like ingress, XNet, and others. Helper functions to track payload sizes and manage their limits.

e44d2bc85
Added a new module, vetkd, defining the VetKd protocol and its associated types and traits, including VetKdArgs, VetKdEncryptedKeyShare, and VetKdEncryptedKey, along with error types like VedKdKeyShareCreationError.

Updated the crypto interfaces to include the VetKdProtocol trait, enabling operations like creating, verifying, and combining encrypted key shares under the VetKd protocol.

*** Potential Issue with logging keys ***

be026d014
New NiDkgMasterPublicKeyId enum specifically designed for NI-DKG, replacing the general MasterPublicKeyId type in relevant contexts.

af542cdad
Update code to accommodate the new tag, including differentiating between different NiDkgTag variants such as LowThreshold, HighThreshold, and the new HighThresholdForKey.

Changes to the Protobuf definitions to include the HighThresholdForKey and its related data. Rust restricts the use of as casting on enums that have non-unit variants. Since NiDkgTag now includes HighThresholdForKey, which is a non-unit variant, it can no longer be cast directly into numeric types like i32.

a3478d571
Updated sandbox eviction logic avoids unnecessary RSS-based evictions when sufficient memory (250 GiB or more) is available. By refining how memory usage is estimated—defaulting to a conservative 50 MiB per sandbox. Matches description.

caca44da3
Add new APIs (VetKdPublicKey, VetKdDeriveEncryptedKey, and ReshareChainKey) to support functionality related to Verifiable Encrypted Key Derivation (VetKD) and chain key resharing. These APIs are integrated into the management canister system, though they are not yet implemented, and currently return rejection messages when invoked.

a1e516f92
Scheduler priorities are now incorporated to rank and decide the eviction order, alongside metrics like recency of usage and RSS.

826d9a503
Restricts the usage of legacy inter-canister query calls (ICQC) exclusively to Distrikt’s subnet, ensuring that no other subnets have access to this deprecated feature. It introduces a principal ID check for the subnet and refines the associated tests to align with this limitation while focusing on composite queries for broader compatibility.

77164cdf7
Penalizing canisters for full execution and redistributing priority now occurs at the end of each round, aligning with the requirements for priority-based eviction.

Newly created canisters start with an accumulated priority equal to their compute allocation instead of zero during reset rounds, ensuring fair initial resource allocation while maintaining compatibility with compute allocation policies.

e391f4cdf
To manage memory usage effectively, a subnet-wide soft cap on callbacks has been established, along with a guaranteed per-canister callback quota. These measures ensure fair distribution when outgoing requests are enqueued while preserving flexibility under normal conditions.

1ee1fe368
abort_canister no longer does apply_priority_credit. Matches commit description.

f8c4eb15e
The ConfigIniSettings, GenerateTestnetConfigArgs, and ICOSSettings structures now include a node_reward_type field, which determines the type of node rewards.

A regex pattern is introduced to validate the node_reward_type value, ensuring it matches a specific format (^type[0-9]+(\.[0-9])?$). An error is raised if the validation fails during runtime.

1cebd7342
Update Guest OS base images matching description.

4e46b92fc
Add node_reward_type to config.

Bugfixes:

338b77f6c
Adjustments to the get_adjusted_notary_delay logic, incorporating membership and logger dependencies to refine the delay calculation and ensure it considers additional conditions, such as halting scenarios during upgrades or excessive gaps between certified and finalized heights.

545f2fad6
Replaces panics and ignored cases in inspect_idkg_chain_key_initializations with robust error handling.

9870f1ea8
Clarify the measured latencies for ingress messages, specifically emphasizing "per-ingress-message wall-clock duration”.

Adds a direct calculation and observation of the latency (wall-clock time) from when a message was included in a block to its induction (Received state), using system_time_now().saturating_duration_since(time).

a078ed82b
Fix for bitcoin test-net issue. More error handling for invalid headers message. target_adjustment_interval_time now uses i64.

2902a2a37
KEEP_ALIVE_INTERVAL increase.

b679ad3a3
The name field of AdapterMetrics was changed from String to &'static str, making it more efficient and simplifying its initialization and usage.

Replaced tokio::sync::RwLock with parking_lot::RwLock

Add parking_lot to Cargo.toml and Bazel files

1c2346cab
Disabled polkit to fix HSM onboarding. Matches description.

211a01015
load_simulator canister and its associated components for benchmarking and debugging. This includes updates to the Cargo.toml and BUILD.bazel files to include dependencies and build rules for the load_simulator canister, as well as a new benchmarking script in load_simulator_canisters.rs to measure throughput and performance under simulated load conditions.

Chores:

9732c2c26
Removal of the IngressSelector interface from the Validator and associated components. This includes eliminating the has_message method and its references in the ValidatorMetrics and IngressSelector implementations, as it is no longer used or needed.

Cleanup of metrics related to missing_ingress_messages, along with removing logic for observing these metrics in the consensus process.

85c272c78
Removed some clones from IngressSelector for better performance.

5a5e0be15
Add logging to help clarify onboarding success.

Refactoring:

81686f56f
Reorganization of the ExtendedDerivationPath struct, which has been relocated from specific canister_threshold_sig modules into a broader crypto namespace.

3397eb8bf
Ensures that the capacity constraints are validated at compile time rather than at runtime.

dab484292
The variables use_nns_public_key and use_node_operator_private_key were renamed to better reflect their purpose. Makes role more explicit and aligned with their function.

03f3debdf
Node Id set to 0 and removed node_index.

d0acab2b2
Integration of a new sns_system_test_lib crate to streamline SNS (Service Nervous System) system tests. Existing test modules were refactored to use this library.

proposal - 134251

Vote: ADOPT

Reason:

I successfully built and verified the guest image. Commit changes look fine.

Hash Match: MATCH

2 Urls: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits:

6e2c0d9fb
Similar commit to the following:
a3478d571
Updated sandbox eviction logic avoids unnecessary RSS-based evictions when sufficient memory (250 GiB or more) is available. By refining how memory usage is estimated—defaulting to a conservative 50 MiB per sandbox. Matches description.

b16d21c2a
Enables HASHES_IN_BLOCKS_FEATURE_ENABLED

proposal - 134259

Vote: ADOPT

Reason:

I successfully built and verified the guest image. Commit changes look fine.

Hash Match: MATCH

2 Urls: MATCH

Feedback: NONE

Proposer Check: MATCH

Commits:

5d2028948
This reverts commit 7dd4fd0.

[timk11]

Proposal 134250

Vote: Adopt

Reason: Build is successful, hashes match, commits match descriptions and the reasoning behind the changes is sound.

@raymondk I’ve also verified HostOS and SetupOS images for these 3 proposals as requested separately. Local and CDN hashes match for all of these.

Review

I’ve selected reviewed Consensus, Crypto and Interface commits as follows.

Features:

[f3bb40251]
Adds to finaliser, notary and payload builder metrics, including histograms for payload and payload section byte size and batch delivery interval.

[e44d2bc85]
Adds a trait VetKdProtocol, implemented on CryptoReturningOk with placeholder methods for creating, verifying and sharing encrypted key shares and verifying the combined encrypted key. Adds VetKdArgs, VetKdEncryptedKeyShare, VetKdEncryptedKey and error types, along with logging and tests.
@franzstefan Are there any potential concerns with exposing keys during logging here?

[be026d014]
Adds enum NiDkgMasterPublicKeyId and conversion methods between this and MasterPublicKeyId, for use with HighThresholdForKey variant as detailed in the next commit.

[af542cdad]
Adds a variant HighThresholdForKey(MasterPublicKeyId) to enum NiDkgTag. The tag relates to different executions of non-interactive distributed key generation, in this case requiring more participants (i.e. high threshold) for operations using a specific key.

[f8c4eb15e]
Adds field node_reward_type to ConfigIniSettings.

Bugfixes:

[338b77f6c]
Adds a halting flag in order to prevent the notary delay from being increased (which usually happens when the gap between the finalised height and the certified height has gone beyond an acceptable limit) in cases where the subnet is upgrading. Adds related tests.

[545f2fad6]
Replaces .expect with .ok_or in function inspect_idkg_chain_key_initializations in order to always return an error in cases of protobuf conversion errors.

[a078ed82b]
Change types in time calculations in the bitcoin implementation from u32 to i64 in order to correct/prevent an underflow error that occurred due to a bitcoin testnet block storm.

[2902a2a37]
Increase KEEP_ALIVE_INTERVAL from 0.2 s to 1 s in the QUIC transport connection manager.

[b679ad3a3]
Changes AdapterMetrics name type to string slice + related changes.

Chores:

[9732c2c26]
Removes missing_ingress_messages from consensus validator metrics as this has now been superceded by ingress_messages_in_a_block_count in stripped artifact metrics.

[85c272c78]
Removes .clone() from parts of rs/ingress_manager/src/ingress_selector.rs in order to improve performance.

[5a5e0be15]
Adds a logging message advising node providers to wait for a “Join request successful!” message before presuming that onboarding has been successful.

[87270d904]
Adds type IDkgMasterPublicKeyId, serving as a wrapper around MasterPublicKeyId, along with conversion methods between them and other related logic. The intention is to distinguish between IDKG compatible and incompatible variants, given that the VetKD variant will be incompatible with some of the existing IDKG code once it is fully implemented.

[0eedbb674]
Change such that RequestMetadata is always used in test_backward_compatibility instead of just 4 times out of 5.

[aacbed376]
Efficiency changes to canonical state certification version exceptions and tests.

[7dd4dfe62]
Addition of a Jaeger exporter crate.

[8b94d60b6]
Version update to opentelemetry and related crates.

[45dca07dd]
Renames run_stream_acceptor to start_stream_acceptor and removes shutdown in the event of the request handler event loop being exited.

[72ec446d6]
Added detail to the invalid version logging message in the bitcoin adaptor connection manager.

[f96dec1db]
Removes generate_bond6_netdev_content function and related code from rs/ic_os/network/src/systemd.rs + documentation changes.

Refactoring:

[81686f56f]
Moves struct ExtendedDerivationPath from rs/types/types/src/crypto/canister_threshold_sig.rs to rs/types/types/src/crypto.rs as the feature will be used more broadly once vetKeys is implemented.

[3397eb8bf]
Simplifies impl ThresholdSigDataStoreImpl by placing assertions prior to the impl and moving from two constructors to just one.

[dab484292]
Renames nns_public_key_exists to use_nns_public_key and node_operator_private_key_exists to use_node_operator_private_key.

Tests:

[d0acab2b2]
Removal of various tests from ‎rs/tests/src/nns_tests/ to ‎rs/tests/nns/.

Proposal 134251

Vote: Adopt

Reason: Build is successful, hashes match, commits match descriptions and the reasoning behind the changes is sound.

Review

[6e2c0d9fb]
Changes to canister wasm sandboxing, so as to evict sandbox processes only when available memory is low.

[b16d21c2a]
Enables the hashes-in-blocks feature.

Proposal 134259

Vote: Adopt

Reason: Build is successful and hashes match. The proposal consists of a single commit which reverts the base container image references to a previous version.

[ilbert]

Proposal 134250

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
For the Execution and Runtime layers, the main changes introduced in this proposal are:

• Optimizations of the sandbox eviction logics
• Starting the work on the vetKeys API

I’ve reviewed all the commits listed in the proposal, for the Execution and Runtime layers:

a3478d571:
Adds the trigger_sandbox_eviction method to the SandboxedExecutionController struct, which calls the newly introduced total_active_sandboxes function to get the count of active sandboxes. If the count is greater than the max_sandbox_count, it triggers the eviction. Otherwise, it triggers the eviction if the total sandboxes RSS is greated than the maximum allowed and if the available memory is less than the DEFAULT_MIN_MEM_AVAILABLE_TO_EVICT_SANDBOXES (newly introduced and equal to 250GiB). The available memory is calculated with the newly introduced available_memory_wrapper method of the SandboxedExecutionController. The trigger_sandbox_eviction method is called in the get_sandbox_process method.

caca44da3:
Starts the work in the Execution layer to implement the APIs for vetKeys. It adds the ReshareChainKey, VetKdPublicKey and VetKdDeriveEncryptedKey and their relative arguments and return types to the management canister API. Calls to these methods will fail for now.

a1e516f92:
Changes the logic inside the evict function to sort the candidate canisters using the following logic: prioritize the canisters that have last_used value below the threshold (”idle”) ordering them by their last_used value, then sort the remaining candidates based on their scheduler_priority first and last_used valued after.
Adds the state_reader field to the SandboxedExecutionController struct. This field is used in the evict_sandbox_processes function in order to populate the scheduler_priority field of the EvictionCandidate struct when filtering out the candidates to pass to the evict function.
Adds the get_scheduler_priorities to the ReplicatedState struct, which maps the canister_states into a BTreeMap of (canister id, accumulated priority from the scheduler state).

826d9a503:
Limits the availability of the legacy inter-canister query calls only to the subnet where Distrikt is deployed, which is the only application that still uses it. In order to do this check, the QueryContext needs the own_subnet_id field, which is introduced in this commit. This field is populated when calling the new method from the InternalHttpQueryHandler::new method. For this reason, the own_subnet_id has been added to the InternalHttpQueryHandler struct too.
Thanks to this change, new subnets can be opened up with the legacy feature already disabled.

77164cdf7:
Changes the logic of the apply_scheduling_strategy method of the RoundSchedule struct to set the accumulated priority of a canister to its compute allocation every time the accumulated priority are reset.
It also removes a for loop and moves the skipped_round_due_to_no_messages canister metric calculation inside the first for loop. This removes the need of iterating again on the canister states.
It also optimizes another for loop at the end of the method to cycle through the ordered_long_execution_canister_ids.

e391f4cdf:
Adds the subnet_callback_soft_limit and canister_guaranteed_callback_quota to the Config struct of the execution environment. Their default values are 1_000_000 and 50, respectively.
Adds the subnet_available_callbacks field to the RoundLimits struct.
Adds the callback_count method to the ReplicatedState struct, which returns the total number of the callbacks of all the canisters.
Adds the callbacks_created method to the SystemStateChanges, which returns the length of the requests.
Matches description.

1ee1fe368:
Removes the call to RoundSchedule::apply_priority_credit inside the abort_canister method of the ExecutionEnvironment struct.

9870f1ea8:
Matches description.

211a01015:
Matches description.

659d9b143:
Matches description.

5813a429f:
Renames route_idks_message into route_chain_key_message, IDkgSubnetKind into ChainKeySubnetKind. merges EcdsaKeyError and IDkgKeyError into ChainKeyError in the ResolveDestinationError enum.

2f00d6815:
Changes the evict function to first check if there are any candidates to evict. The check is: candidates amount must be above the max_count_threshold and the total_rss must be above the max_sandboxes_rss. The same check is also made in the evict_sandbox_processes function, to avoid calling the evict and creating the candidates vector if there are no sandboxes to evict.

a10fbc291:
Matches description.

0d14c11bd:
Matches description.

---

Proposal 134251

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
This release is the same as the one in 134250 with the addition of the commits 6e2c0d9fb and b16d21c2a, which match their description.

---

Proposal 134259

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
This release is the same as the one in 134250 with the addition of the commit 5d2028948, which matches its description.

[Zane]

Proposal 134250

Vote: ADOPT

Reason: Build is successful and hashes match, reviewed commits also match their descriptions, therefore I’ve voted to adopt.

Features:

a3478d571 Added DEFAULT_MIN_MEM_AVAILABLE_TO_EVICT_SANDBOXES const and set it to 250GiB. Updated logic used to determine when sandbox eviction should be triggered in get_sandbox_process so that it doesn’t only rely on the total resident set size updated asynchronously but also checks whether available memory metric from process_os is below the threshold. The new logic has been incapsulated in a new method trigger_sandbox_eviction. Furthermore when evictions are triggered without validating actual memory usage, the rss logic is skipped, this is done by passing u64::MAX to evict_sandbox_processes as max_sandboxes_rss argument, the value is then propagated to evict method which uses it to determine when enough candidates have been evicted, since the value is always greater than total_rss the condition is always satisfied.
Added helper method total_active_sandboxes to calculate number of active sandboxes.

In monitor_and_evict_sandbox_processes and update_sandbox_processes_rss the variable used to hold sandbox processes rss and id has been changed from an hashmap to a vector, this is more efficient considering the only operations performed on it are inserts and iterations.

caca44da3 Added scaffholding required to support 3 new VetKeys related management canister endpoint: ReshareChainKey, VetKdPublicKey, and VetKdDeriveEncryptedKey. All 3 of them are currently not implemented.

a1e516f92 Extended EvictionCandidate struct with scheduler_priority field and modified ReplicatedState to expose a new method get_scheduler_priorities which returns a BTreeMap linking canister ids to their accumulated priority in the scheduler. SandboxedExecutionController has also been modified to take in a StateReader to make use of the new method.

The listed changes were aimed at improving eviction logic to achieve more efficient cache hits, to do so evict_sandbox_processes gets the canister id-priority mapping from the StateReader and assigns the scheduled priority to all active eviction candidates. In evict the candidates are sorted by their last_used value for idle processes and by their scheduler_priority for active processes. Furthermore the method has been slightly optimized, in case the number of processes to evict is 0 and total_rss is less than max_sandboxes_rss, it returns early since effectively there is nothing to evict and by doing so it skips the sorting step. In the eviction loop, the candidates are added early to the eviction list, this is possible cause the method returns early if no candidate has to be evicted, this makes it possible to avoid a new unnecessary iteration.

826d9a503 Previously legacy ICQCs were allowed on system and verified application subnets, since they have to be deprecated to prevent new canisters from using them, they have been restricted to the subnet Distrikt runs on since it is the only service still relying on the feature.

77164cdf7 apply_scheduling_strategy no longer updates priority_credit of canisters as it is now done in finish_round. During reset rounds the accumulated_priority is now set in function of the canister’s computer allocation instead of using the default zero priority.

e391f4cdf Added subnet_callback_soft_limit and canister_guaranteed_callback_quota fields to execution environment config, these are used to impose limits on how many pending callbacks a subnet can have and in case the limit is reached provide each canister a guaranteed quota of callbacks so it can still make calls. The guaranteed quota for each canister is calculated starting by a max of 50 and then subtracting the number of pending callbacks for that canister, the value is only used as an effective limit if its greater than the subnet wide limit.

1ee1fe368 Removed apply_priority_credit call from abort_canister.

Bugfixes:

9870f1ea8 Updated description for block induction histograms and fix how latencies are calculated by using system time instead of received_time.

Benchmarking

211a01015 Added load simulator canister and use it in load_simulator_canisters_bench to benchmark performance of a set amount of canister for n rounds whose only workload is setting up a timer and writing to stable memory once every 50 timer calls.

Chores:

659d9b143 Use AccumulatedPriority::new(i64::MIN) as default value for scheduler_priority in evict_sandbox_processes instead of 0.

5813a429f Change terminology of some structs, methods and comments to “chain key” in place of “IDKG” or “Ecdsa”.

2f00d6815 Optimized evict_sandbox_processes by having it return early if no eviction is necessary.

a10fbc291 Use Wasmtime::Linker to retrieve all system API imports instead of hardcoding them.

0d14c11bd Slightly improved code readability in reserve_cycles and fix a minor typo in a comment.

0eedbb674 Always populate RequestMetadata in test_backward_compatibility.

aacbed376 Deleted unused code related to old certification versions.

Proposal 134251

Vote: ADOPT
Reason: Build is successful and hashes match, the proposed changes are the same as 134250 but with feature flag for hashes in block feature enabled.

Proposal 134259

Vote: ADOPT
Reason: Reason: Build is successful and hashes match, the proposed changes are the same as 134250 with the only difference being a commit which reverts changes introduced in a previous release.