Proposal to elect new release rc--2024-10-31_03-09

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133900.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-base (88227422ae6e3bfc6c74a5216309a3b86a93744b)

This release is based on changes since release-2024-10-23_03-07-base (75dd48c38f296fc907c269263f96633fa8a29d0e).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

This release diverges from the latest release. Merge base is 524aa689c5781255d9db638f06eb2d24b6bec4bd. Changes were removed from this release.

Features:

• e7b16eab4 Consensus,Interface(consensus): adjust dynamic block maker delay (#2336)
• e3c408cd0 Consensus,Interface(consensus): push all ingress messages (#2233)
• c5e9e2af1 Consensus,Interface(consensus): reduce consensus bouncer function refresh period from 3 seconds to 1 second. (#1885)
• 237990ccb Crypto,Interface(crypto): skip ingress_expiry check for anonymous queries and read_state requests (#1768)
• 161b12ce8 Crypto,Interface(crypto): Add support for derivation to ecdsa_secp256r1 crate (#1730)
• d361dd692 Execution,Interface: Update cycles cost for compute (#2308)
• 74751adc0 Execution,Interface: Skip per-canister overhead if no instructions used (#2322)
• 71f3b769c Execution,Interface: [v3] Make subnet replica version of latest block available to execution environment (#2248)
• b29e83ac7 Execution,Interface: enable allowed_viewers feature for canister log visibility (#2244)
• 3006ab867 Execution,Interface: Evict sandboxes based on their RSS (#2197)
• 39f67a628 Interface(crypto): Introduce master key ID variant for vetKD (#2108)
• 0515bc380 Interface,Message Routing: Introduce a minimum supported certification version. (#1502)
• fa22c9fa9 Interface,Message Routing: Introduce msg_limit for application subnets in payload builder (#1798)
• ac6c9d1e5 Interface,Message Routing: Reduce the limit for messages in Stream from 50k to 10k. (#2185)
• 8c2e0c98e Interface,Networking(https-outcalls): re-enable H/2 support for outcalls (#2200)
• 993fc8586 Interface,Node: initial draft of custom metric tool and its systemd timer (#1963)

Bugfixes:

• f306fcb8c Consensus,Interface(recovery): Use metric for highest validated certification for determining node heights (#2332)
• 3d590ec5a Consensus,Interface: remove the async locks from the orchestrator (#2325)
• e70f04d9f Consensus,Interface(recovery): Print example of expected ssh key format during recovery (#2187)
• 55297ccfd Consensus,Interface: update ic.json5 nns_urls references (#2192)
• ec12a4eea Interface,Networking(https-outcalls): Increase allowed header size limit for HTTP/2 requests (#2292)
• 218fdbcaa Interface,Networking: remove the idle future (#2285)
• 52bc5275e Interface,Networking: call make_idle if the adapter is idle initially (#2278)
• b56d5e1ba Interface,Networking: synchronize properly with tokio::Notify instead of sleeping (#2178)
• 582ce51a6 Interface,Node: Fix confusing logging of ipmitool (#2297)
• 677aa6862 Owners(IDX): bump to thiserror v1.0.65 in Bazel (#2305)
• 7be43416d Owners(IDX): bump to typenum v1.17.0 in Bazel (#2296)
• 53f97714f Owners(IDX): work around spurious rebuilds in rustix (#2287)
• dac2f36f9 Node: Make regex for special address ranges more strict (#2315)
• 361d09aeb Node: Improve prestorecon performance (#2218)
• 3753b4388 Node(setupos): update default ipv6 prefix and gw for setupos testnets (#1805)

Chores:

• 80dc25b52 Consensus,Interface(consensus): move notary related functions from consensus_utils crate to notary.rs in consensus crate (#2310)
• d3cfffac1 Consensus,Interface(consensus): Replace generic pool metrics in certification pool (#2312)
• 4f7cebee1 Consensus,Interface: Don’t log default replica version in batch delivery (#2276)
• 10150fb34 Consensus,Interface(consensus): Rename persistent certification pool section for consistency (#2262)
• c5e6242f5 Crypto,Interface: remove unneeded clippy allows (#2326)
• 4899bba2c Interface: upgrade tower-rs crates (#1773)
• 54f0fae1c Interface: disallow async locks (RwLock) (#2144)
• c84916512 Interface,Message Routing: Keep extra in-memory states (#2061)
• 619190192 Owners(IDX): bump rules_rust to 0.53.0 (#2307)
• 35a25eaf6 Owners(IDX): proc-macro2: 1.0.88 -> 1.0.89 & remove rules_rust.patch (#2224)
• 7dede601c Node: add more logging for check-network.sh (#2302)
• 60ce9d351 Node: revert removal of old API BN certificates (#2209)
• 36cbd803b Node: Update Base Image Refs [2024-10-24-0808] (#2229)

Refactoring:

• 6c4efce2e Interface,Message Routing: split load_checkpoint (#2210)

---

Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/88227422ae6e3bfc6c74a5216309a3b86a93744b/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 88227422ae6e3bfc6c74a5216309a3b86a93744b --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133938.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-hashes-in-blocks (cc1319059ee8283cf96481109f98d0b14b967859)

This release is based on changes since release-2024-10-31_03-09-base (88227422ae6e3bfc6c74a5216309a3b86a93744b).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Features:

• cc1319059 Interface,Networking(Consensus): Enable the hashes-in-blocks feature

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/cc1319059ee8283cf96481109f98d0b14b967859/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c cc1319059ee8283cf96481109f98d0b14b967859 --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133939.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-ubuntu20.04 (51f6f4e4ab7fa2a8ad4cf573e04fc2686e14fa57)

This release is based on changes since release-2024-10-31_03-09-base (88227422ae6e3bfc6c74a5216309a3b86a93744b).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

• 51f6f4e4a Node: Revert “chore: Fixup services (#1837)”
• c514a532a Node: Revert “fix(): allow read access to more hardware info for node_exporter (#2121)”
• 372b9a574 Node: Revert “feat: Upgrade GuestOS to 24.04 (#938)”

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/51f6f4e4ab7fa2a8ad4cf573e04fc2686e14fa57/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 51f6f4e4ab7fa2a8ad4cf573e04fc2686e14fa57 --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133902.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-6.11-kernel (f910b32efbd32183962b74464b1044b900a58a5b)

This release is based on changes since release-2024-10-31_03-09-base (88227422ae6e3bfc6c74a5216309a3b86a93744b).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

• f910b32ef Node: Squashed commit of the following: Try using 6.11 kernel. Use 6.11 base images

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/f910b32efbd32183962b74464b1044b900a58a5b/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c f910b32efbd32183962b74464b1044b900a58a5b --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[ZackDS]

Proposal #133900

Vote: Adopted
Reason: Builds fine and the hash matches, as do all the listed commits.

Reviews :

[ZackDS]

Proposal #133901

Vote: Adopted
Reason: Builds fine and the hash matches, as does the by now usual commit for hashes in blocks.

Review:

Features:
cc1319059 Interface,Networking(Consensus): Enable the hashes-in-blocks feature
Done by setting HASHES_IN_BLOCKS_FEATURE_ENABLED to true.

[ZackDS]

What happened here with release-2024-10-31_03-09-ubuntu20.04 ?

Update:

Proposal #133939

Vote: Adopted
Reason: Builds fine and the hash matches, as do all the listed commits.

Reviews:

Other changes:
51f6f4e4a Node: Revert “chore: Fixup services (#1837)”
c514a532a Node: Revert “fix(): allow read access to more hardware info for node_exporter (#2121)”
372b9a574 Node: Revert “feat: Upgrade GuestOS to 24.04 (#938)”
All three reverts match their description as they are the usual for this release.

[ZackDS]

Proposal #133902

Vote: Adopted
Reason: Builds fine and the hash matches, as does the listed commit.

Review:

Other changes:
f910b32ef Node: Squashed commit of the following: Try using 6.11 kernel. Use 6.11 base images
Settings for the GuestOS dockerfile to force using the specific 6.11 kernel for the base image, it matches description but no motivation given. It’s fine for testing purposes.

[cyberowl]

proposal - 133900

Vote: ADOPT

Reason:

The changes appeared sound, with all hashes matching and no cause for concern. I noted, however, that the canister_creation_fee was left unchanged despite prior discussions on the forum about its potential increase, though this isn’t a reason to halt deployment. On the whole, the update introduces promising features like VetKeys, which I am excited about.

Hash Match: MATCH

2 Urls: MATCH

Feedback:

canister_creation_fee stayed the same at 100_000_000_000 even though blog mentions increase.

Proposer Check: MATCH

Overall Summary:

Introduces a range of changes primarily to improve subnet performance, implement cycle fee adjustments, and add foundational support for VetKeys, a new cryptographic feature for enhanced data privacy on the Internet Computer. Fee increases, particularly for message execution and instruction fees, aim to balance operational costs with increased demand on resources, although the canister_creation_fee remains unchanged despite prior discussions. VetKeys integration has been a long time coming and it’s exciting to see progress being made.

Commits Summary

Features:

e7b16eab4
Increased DYNAMIC_DELAY_MAX_NON_RANK_0_BLOCKS from 5 to 10. With the threshold increased, the additional delay for higher-ranked proposals will trigger less frequently, allowing non-rank-0 blocks to proceed more often without the dynamic delay penalty.

e3c408cd0
Currently, only small ingress messages are pushed directly, while larger ones require an advert and request, adding latency. Expanding direct message pushing could reduce this delay and improve validation efficiency, though it may occasionally increase bandwidth use for unnecessary messages. So is_latency_sensitive is set to true to activate.

c5e9e2af1
This change is in combination with commit e3c408cd0. The refresh_period in the ConsensusBouncer struct defines how frequently the bouncer function checks the pool to identify unnecessary or expired ingress messages. Setting this period to 1 second means the function will check for these messages every second, allowing it to quickly identify and discard expired or unnecessary messages.

237990ccb
Checks for ingress_expiry in Query and ReadState requests when the sender is not anonymous, enhancing validation of expiration times for non-anonymous users.

161b12ce8
derivation path handling through the DerivationIndex and DerivationPath structures, allowing for BIP32-style key derivation in PrivateKey and PublicKey with derive_subkey and derive_subkey_with_chain_code methods. ckd and ckd_pub helper functions to support key derivation along a path, providing SLIP-10-compatible hierarchical deterministic key generation.

d361dd692
Substantial increases in cycle fees across multiple configurations and test files, including UNIVERSAL_CANISTER_CYCLE_MARGIN and various execution fees for operations like update_message_execution_fee and ten_update_instructions_execution_fee. This got discussed at length on the forum FEES$$$.

Message Base Fee: Increasing from 590K cycles to 5M cycles.
Instruction Fee: Increasing from 0.4 cycles to 1 cycle per instruction.

canister_creation_fee stayed the same at 100_000_000_000 even though blog mentions increase.

74751adc0
By skipping the per-canister overhead deduction when total_instructions_used is zero, the system avoids unnecessary overhead for canisters that lack sufficient cycles. Matches description.

71f3b769c
Tracking the ReplicaVersion in various components of the Internet Computer’s execution and scheduling processes. This includes updating structs like Batch to store the replica_version and modifying test utilities and test builders to account for this new field.

b29e83ac7
Enables allowed_viewers for canister log visibility.

3006ab867
Introduce memory-based eviction thresholds for sandbox processes by adding a max_sandboxes_rss limit to manage memory usage, in addition to the existing process count thresholds. The evict_sandbox_processes function and associated sandbox stats are now enhanced to track and handle memory usage per sandbox, with new logic to determine when to evict based on combined RSS limits. Additionally, default values for max_sandbox_count and max_sandboxes_rss are increased.

39f67a628
These are the fun VetKey changes for end to end encryption on the IC. This is the start and more changes are to come. This code introduces a preliminary structure to integrate vetKD (vetKeys) by adding a new variant, VetKd, to the MasterPublicKeyId type used in registry configurations and management canister types. It updates KeyConfig, ChainKeyConfig, and relevant APIs, enabling future support.

0515bc380
The goal of this change is to phase out older certification versions, beginning with setting the minimum supported version to V17, as a step toward simplifying and aligning with the new messaging model.

fa22c9fa9
This change sets a limit on the number of messages that can be inducted in a stream slice to ensure it doesn’t exceed a specified maximum, by calculating an upper bound based on stream indices and halting message induction when that bound is reached.

ac6c9d1e5
More realistic 10,000 limit for handling stream messages, accounting for actual bottlenecks and practical usage patterns.

8c2e0c98e
This change got introduced in a prev proposal but then got reverted. This commit introduces the change again to enable H/2 support for outcalls.

993fc8586
Custom metrics_tool service to the system. This tool collects specific metrics, such as TLB shootdowns, writes them in Prometheus format, and integrates them into the monitoring system by generating metrics files compatible with node_exporter. Metrics collection every minute.

Bugfixes:

f306fcb8c
Adds tracking of an additional metric, certification_share_height, by including it in NodeMetrics and parsing it from lines labeled with artifact_pool_certification_height_stat{pool_type="validated",stat="max",type="certification_share"}.

3d590ec5a
Use synchronous RwLock operations across several modules, simplifying the handling of shared state by replacing tokio::RwLock with std::sync::RwLock. This matches the description. Removes async locks.
e70f04d9f
UX changes to enter key

55297ccfd
Field fix to match spec.

ec12a4eea
Increase MAX_HEADER_LIST_SIZE because it broke sending of headers greater than 16KB. This fixes that.

218fdbcaa
Reverts the idle changes from commit 52bc5275e.

52bc5275e
Allows the adapter to remain idle until activity is detected, thus reducing resource usage when no new events are received.

b56d5e1ba
AdapterState to use a watch channel, allowing it to track the time of the last received request asynchronously and avoid panics on MacOS related to time calculations. This update introduces two new asynchronous methods, idle and active, that leverage the watch channel to monitor and manage adapter idle and active states based on the idle_seconds threshold. The improvement is a more responsive idle-state management system, allowing the adapter to efficiently await activity and transition in and out of idle mode without unnecessary delays.

582ce51a6
Detailed error context message in the get_ipmi_mac function for failed ipmitool executions, capturing the status and standard error output to aid debugging if parsing the IPMI LAN MAC address fails.

677aa6862
Update thiserror

7be43416d
Update typenum

53f97714f
rustix patch to fix determinism issues

dac2f36f9
Stricter matching criteria for filtering IPv6 addresses.

361d09aeb
Improve perf by adding timing measurements to the tool, specifically by recording and printing the total elapsed time in milliseconds, from the start to the end of processing.

3753b4388
Updates the IPv6 network configuration by modifying the ipv6_prefix and ipv6_gateway values to a new subnet, adjusting them from 2a00:fb01:400:200 to 2a00:fb01:400:44.

Chores:

80dc25b52
Change relates to commit chore(consensus): move notary related functions from consensus_utils crate to notary.rs in consensus crate (#2310). It relocates notary-related functions from the consensus_utils crate directly into the notary.rs file within the consensus crate, effectively consolidating notary logic into a more specific module and reducing dependencies on consensus_utils.

d3cfffac1
Addition of a PerTypeMetrics struct, which tracks the minimum, maximum, and count metrics for both certification and certification share artifacts. These metrics are integrated into the PoolMetrics struct, allowing for enhanced observability of certification pool states.

4f7cebee1
Remove the current_replica_version parameter from several functions in the consensus modules, instead using the replica_version directly obtained from the Block structure. This refactor reduces redundancy by accessing the version dynamically within each function.

10150fb34
Renaming persistent_pool to validated throughout the code, emphasizing that the pool stores only validated certification data.

c5e6242f5
Remove the #![allow(clippy::unwrap_used)] directive from various test files, ensuring that unwrapping operations are avoided where possible for better error handling practices. clippy.toml file is modified to allow expect and unwrap in tests.

4899bba2c
Upgrade tower dependency from version 0.4.13 to 0.5.1 and tower-http from 0.5.2 to 0.6.1, along with adjustments to dependencies and configurations to align with these versions. Mutex and RwLock, are now used directly instead of tower::buffer::Buffer wrappers in several places.

54f0fae1c
Add #![allow(clippy::disallowed_types)] to multiple modules to suppress specific Clippy lint warnings related to the use of discouraged types, particularly asynchronous locks like tokio::sync::Mutex and tokio::sync::RwLock.

c84916512
remove_inmemory_states_below function now accepts an additional parameter, extra_heights_to_keep, allowing specific states to be retained even if they are below the standard height threshold. The goal is to optimize memory usage by discarding in-memory states more promptly at checkpointed heights, while selectively retaining necessary states for current CUP creation and validation.

619190192
Bump rules_rust-v0.53.0

35a25eaf6
Update of the proc-macro2 dependency from version 1.0.88 to 1.0.89 across multiple entries in the dependency files, updating the corresponding checksum for integrity verification. URLs associated with the proc-macro2 package download have been updated to match the new version reference.

7dede601c
Enhancement of the eval_command_with_retries function in the updated script, which now captures and logs additional diagnostic information (such as the output of ip -6 addr show, ip -6 route show, and configured DNS servers) when a command fails after multiple retries.

60ce9d351
Revert for remove existing certificates

36cbd803b
Update base image refs

proposal - 133901

Vote: ADOPT

Hash Match: MATCH

2 Urls: MATCH

Proposer Check: MATCH

[cc1319059]
Enable HASHES_IN_BLOCKS_FEATURE_ENABLED

proposal - 133902

Vote: ADOPT

Hash Match: MATCH

2 Urls: MATCH

Proposer Check: MATCH

[f910b32ef]
The updated Dockerfile adds a step to download, verify, and install the 6.11 Linux kernel and related modules, ensuring the new kernel version is included in the base image.

[ilbert]

Proposal 133900

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
For the Execution and Runtime layers, the main changes introduced in this proposal are:

• Increasing the execution cycles fees
• Optimizing the canisters execution
• Preparing the execution layer to introduce the management canister’s subnet_info endpoint
• Enabling the allowed viewers setting in the canister logs visibility API

I’ve reviewed all the commits listed in the proposal, for the Execution and Runtime layers:

d361dd692:
Changes the application subnet configuration in the subnet_config to increase the update_message_execution_fee from 590’000 to 5’000’000 cycles and the ten_update_instructions_execution_fee from 4 to 10 cycles.
This commit also updates a lot of tests where the cycles are hard-coded, in order to accommodate the new fees.

74751adc0:
Changes the execute_canisters_on_thread function in the scheduler to decrease the execution round’s left instructions only if the canister used some instructions. during the execution. The total_instructions_used variable is introduced to keep track if the canister has used any instructions when looping over the canister’s messages. The variable is checked at the end of the canister messages’ loop, before starting executing messages for the next scheduled canister.
This change optimizes the round capacity, as it does not reduce the instructions left for the round if a canister fails without consuming cycles.

71f3b769c:
Adds the replica_version field to the Batch struct, which is constructed in the consensus layer. Inside the execute_round method of the StateMachine struct, the replica version value is read and passed to the execute_round of the Scheduler struct.
The value is then passed to the execute_subnet_message method of the Scheduler struct, where the value is passed to the execute_subnet_message method of the ExecutionEnvironment struct. Here, the value is ignored for now, but a PR has already been opened to use and expose it in the management canister’s API.
The value is also passed to the inner_round method of the Scheduler struct, where it is subsequently passed to the drain_subnet_queues method.

b29e83ac7:
Enables the allowed viewers feature for the canister logs visibility by setting the allowed_viewers_feature flag of the execution environment’s Config struct to enabled.

3006ab867:
Introduces the same changes introduced in the proposal 133798.

---

Proposal 133901

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
This release is the same as the one in 133900 with the addition of the commit cc1319059, which matches its description.

---

Proposal 133902

Vote: ADOPTED.
Reason: All reviewed commits match their description and hashes match.

Review:
This release is the same as the one in 133900 with the addition of the commit f910b32ef, which matches its description.

[hpeebles]

Proposal 133900: Adopt

I have reviewed all commits within proposal 133900 and in my opinion they all look fine.
I have also successfully run the build verification script, so I have voted to adopt the proposal.

Full review:

Features:

• e7b16eab4 Consensus,Interface(consensus): adjust dynamic block maker delay (#2336)
  Review: Looks fine + matches description
  Notes: Bumps the DYNAMIC_DELAY_MAX_NON_RANK_0_BLOCKS constant from 5 to 10 so that the delay is applied less frequently, fixes a few comments, and modifies some test cases.

• e3c408cd0 Consensus,Interface(consensus): push all ingress messages (#2233)
  Review: Looks fine + matches description
  Notes: Sets is_latency_sensitive to true on all ingress messages so that they get pushed immediately rather than having to first be advertised.

• c5e9e2af1 Consensus,Interface(consensus): reduce consensus bouncer function refresh period from 3 seconds to 1 second. (#1885)
  Review: Looks fine + matches description
  Notes: Reduces the consensus bouncer refresh period from 3 seconds to 1 second, allowing a lagging node to catch up by up to 10 blocks per second.

• 237990ccb Crypto,Interface(crypto): skip ingress_expiry check for anonymous queries and read_state requests (#1768)
  Review: Looks fine + matches description
  Notes: Hoists the call to validate_ingress_expiry out of validate_request_content and instead calls it within the various validate_request implementations, skipping it whenever the caller is anonymous.

• 161b12ce8 Crypto,Interface(crypto): Add support for derivation to ecdsa_secp256r1 crate (#1730)
  Review: Looks fine + matches description
  Notes: Adds support for derived subkeys to the ic-crypto-ecdsa-secp256r1 crate so that going forward the dependency can be removed on the internal threshold ECDSA implementation.

• d361dd692 Execution,Interface: Update cycles cost for compute (#2308)
  Review: Looks fine + matches description
  Notes: Increases the cycles costs of various compute actions.

• 74751adc0 Execution,Interface: Skip per-canister overhead if no instructions used (#2322)
  Review: Looks fine + matches description
  Notes: Avoids subtracting instruction_overhead_per_canister from the count of instructions remaining in the round for canisters that didn’t use any instructions.

• 71f3b769c Execution,Interface: [v3] Make subnet replica version of latest block available to execution environment (#2248)
  Review: Looks fine + matches description
  Notes: Passes the replica version of each block through to the execution environment which will make use of it in future commits.

• b29e83ac7 Execution,Interface: enable allowed_viewers feature for canister log visibility (#2244)
  Review: Looks fine + matches description
  Notes: Sets the allowed_viewers_feature field of the execution environment config to true.

• 3006ab867 Execution,Interface: Evict sandboxes based on their RSS (#2197)
  Review: Looks fine + matches description
  Notes: Evicts sandbox processes based on their resident set size (the amount of memory allocated to their process) rather than evicting a fixed number each time. This allows for more sandbox processes while still keeping a limit on their total memory usage.

• 39f67a628 Interface(crypto): Introduce master key ID variant for vetKD (#2108)
  Review: Looks fine + matches description
  Notes: Adds VetKd as a new subtype to the MasterPublicKeyId variant. This is the first step to implementing the vetKeys feature.

• 0515bc380 Interface,Message Routing: Introduce a minimum supported certification version. (#1502)
  Review: Looks fine + matches description
  Notes: Adds the MIN_SUPPORTED_CERTIFICATION_VERSION constant and filters out versions below it from the all_supported_versions response. This will allow the older versions to be phased out in the future.

• fa22c9fa9 Interface,Message Routing: Introduce msg_limit for application subnets in payload builder (#1798)
  Review: Looks fine + matches description
  Notes: Adds the MAX_STREAM_MESSAGES constant to the xnet payload builder and applies it when inducting messages to ensure the number of signals in the reverse stream will remain below the limit.

• ac6c9d1e5 Interface,Message Routing: Reduce the limit for messages in Stream from 50k to 10k. (#2185)
  Review: Looks fine + matches description
  Notes: Reduces the newly added MAX_STREAM_MESSAGES from 50k to 10k.

• 8c2e0c98e Interface,Networking(https-outcalls): re-enable H/2 support for outcalls (#2200)
  Review: Looks fine + matches description
  Notes: Updates the HttpsConnectorBuilder used by the HttpOutcalls adapter to use enable_all_versions rather than enable_http1, enabling it to support both HTTP1 and HTTP2. This was previously included in last week’s release but was reverted due to test flakiness which has now been sorted.

• 993fc8586 Interface,Node: initial draft of custom metric tool and its systemd timer (#1963)
  Review: Looks fine + matches description
  Notes: Introduces a new metrics_tool executable and a new metrics_tool.service service which runs the metrics tool every minute, exporting metrics to a file which can then be fed into the node_exporter via a textfile collector.

Bugfixes:

• f306fcb8c Consensus,Interface(recovery): Use metric for highest validated certification for determining node heights (#2332)
  Review: Looks fine + matches description
  Notes: During recovery, uses the latest certification height from the validated pool rather than the latest delivered certification.

• 3d590ec5a Consensus,Interface: remove the async locks from the orchestrator (#2325)
  Review: Looks fine + matches description
  Notes: Replaces usages of the async tokio::sync::RwLock with the synchronous sync::RwLock.

• e70f04d9f Consensus,Interface(recovery): Print example of expected ssh key format during recovery (#2187)
  Review: Looks fine + matches description
  Notes: During recovery or subnet splitting, when requesting an SSH key from the admin, an example is provided to show the expected format.

• 55297ccfd Consensus,Interface: update ic.json5 nns_urls references (#2192)
  Review: Looks fine + matches description
  Notes: Replaces 2 instances of the old nns_url name to nns_urls.

• ec12a4eea Interface,Networking(https-outcalls): Increase allowed header size limit for HTTP/2 requests (#2292)
  Review: Looks fine + matches description
  Notes: Bumps hyper-util to 0.1.10 to gain access to the new max_header_list_size method which is used to increase the header size limit to 52KiB.

• 218fdbcaa Interface,Networking: remove the idle future (#2285)
  Review: Looks fine + matches description
  Notes: Removes the idle future from the Bitcoin adapter since it was used to cover cases which almost never happen and added a fair amount of complexity.

• 52bc5275e Interface,Networking: call make_idle if the adapter is idle initially (#2278)
  Review: Looks fine + matches description
  Notes: Within the Bitcoin adapter, calls make_idle on the connection_manager and the blockchain_manager if the adapter is itself idle, this ensures the 2 managers don’t remain in non-idle states forever on subnets which don’t run the Bitcoin adapter.

• b56d5e1ba Interface,Networking: synchronize properly with tokio::Notify instead of sleeping (#2178)
  Review: Looks fine + matches description
  Notes: Calls adapter_state.active().await to wait for the adapter to become active, rather than sleeping 100ms.

• 582ce51a6 Interface,Node: Fix confusing logging of ipmitool (#2297)
  Review: Looks fine + matches description
  Notes: Avoids logging an error message in the case where the ipmitool returns an error but the MAC address is still successfully read during get_mac_address_from_ipmitool_output.

• 677aa6862 Owners(IDX): bump to thiserror v1.0.65 in Bazel (#2305)
  Review: Looks fine + matches description
  Notes: Bumps thiserror from 1.0.64 to 1.0.65.

• 7be43416d Owners(IDX): bump to typenum v1.17.0 in Bazel (#2296)
  Review: Looks fine + matches description
  Notes: Bumps typenum from 1.16.0 to 1.17.0.

• 53f97714f Owners(IDX): work around spurious rebuilds in rustix (#2287)
  Review: Looks fine + matches description
  Notes: Patches rustix to work around an issue which was causing builds to be non-deterministic.

• dac2f36f9 Node: Make regex for special address ranges more strict (#2315)
  Review: Looks fine + matches description
  Notes: Modifies the regex check for IPv6 addresses within check-network.sh to exclude ::1 and fe80....

• 361d09aeb Node: Improve prestorecon performance (#2218)
  Review: Looks fine + matches description
  Notes: Calls selabel_open once and shares the handle rather than calling it once per thread.

• 3753b4388 Node(setupos): update default ipv6 prefix and gw for setupos testnets (#1805)
  Review: Looks fine + matches description
  Notes: Updates the default IPv6 prefix and gateway used for SetupOS testnets.

Chores:

• 80dc25b52 Consensus,Interface(consensus): move notary related functions from consensus_utils crate to notary.rs in consensus crate (#2310)
  Review: Looks fine + matches description
  Notes: Moves notary related functions and constants out of the consensus_utils package and into notary.rs within the consensus package because this code is only used by the notary and so doesn’t need to live in the shared library.

• d3cfffac1 Consensus,Interface(consensus): Replace generic pool metrics in certification pool (#2312)
  Review: Looks fine + matches description
  Notes: Replaces usages of the generic PoolMetrics with a new implementation specifically for the certification pool to additionally track the height ranges for which there are certifications and certification shares.

• 4f7cebee1 Consensus,Interface: Don’t log default replica version in batch delivery (#2276)
  Review: Looks fine + matches description
  Notes: Grabs the replica version from the block rather than passing the value into deliver_batches, also removes the 2nd (now redundant) logging of the replica version now that the version is always taken from a single source.

• 10150fb34 Consensus,Interface(consensus): Rename persistent certification pool section for consistency (#2262)
  Review: Looks fine + matches description
  Notes: Renames the persistent_pool field of CertificationPoolImpl to validated.

• c5e6242f5 Crypto,Interface: remove unneeded clippy allows (#2326)
  Review: Looks fine + matches description
  Notes: Adds allow-expect-in-tests = true and allow-unwrap-in-tests = true to the clippy.toml and then removes a load of now redundant clippy allow statements.

• 4899bba2c Interface: upgrade tower-rs crates (#1773)
  Review: Looks fine + matches description
  Notes: Bumps tower from 0.4.13 to 0.5.1 and tower-http from 0.5.2 to 0.6.1.

• 54f0fae1c Interface: disallow async locks (RwLock) (#2144)
  Review: Looks fine + matches description
  Notes: Adds a rule to clippy.toml which disallows the async tokio::sync::RwLock type + adds some overrides where it is currently being used.

• c84916512 Interface,Message Routing: Keep extra in-memory states (#2061)
  Review: Looks fine + matches description
  Notes: Adds the extra_heights_to_keep arg to StateManager::remove_inmemory_states_below which can be used to avoid removing states at the heights specified.

• 619190192 Owners(IDX): bump rules_rust to 0.53.0 (#2307)
  Review: Looks fine + matches description
  Notes: Bumps rules_rust from 0.42.1 to 0.53.0.

• 35a25eaf6 Owners(IDX): proc-macro2: 1.0.88 → 1.0.89 & remove rules_rust.patch (#2224)
  Review: Looks fine + matches description
  Notes: Bumps proc-macro2 from 1.0.88 to 1.0.89.

• 7dede601c Node: add more logging for check-network.sh (#2302)
  Review: Looks fine + matches description
  Notes: Adds more logging to the check-network.sh script + reformats the error response in case of a setup error.

• 60ce9d351 Node: revert removal of old API BN certificates (#2209)
  Review: Looks fine + matches description
  Notes: Reverts the temporary hack to remove pre-existing certificates from the boundary nodes.

• 36cbd803b Node: Update Base Image Refs [2024-10-24-0808] (#2229)
  Review: Looks fine + matches description
  Notes: Updates the IC-OS base image references.

Refactoring:

• 6c4efce2e Interface,Message Routing: split load_checkpoint (#2210)
  Review: Looks fine + matches description
  Notes: Introduces the new CheckpointLoader struct containing instance methods to generate the various datasets needed to build a checkpoint and refactors load_checkpoint to use it.

---

Proposal 133901: Adopt

This proposal is the same as proposal 133900 except with the HASHES_IN_BLOCKS_FEATURE_ENABLED set to true.
I have run the build verification script successfully and so have voted to adopt this proposal.

---

Proposal 133902: Adopt

This proposal is the same as proposal 133900 except the images are built using Linux Kernel version 6.11.
I have run the build verification script successfully and so have voted to adopt this proposal.

[zenithcode]

Proposal: 133900

Summary:

1. The build hash in CDN, proposal and local build matches and is “e621d2e5b89fe727035e4870ff4f579c10c5cbf230af15fe2805d3dec76770df”.
2. Vote: Voting yes to adopt the proposal
3. Reason: The changes in this proposal looks good, and build has matches too. One change related to canister_creation_fee is still not implemented ( but noted that a followup PR / proposal will be there soon ).
   
   

Features:

• e7b16eab4 Consensus,Interface(consensus): adjust dynamic block maker delay (#2336)
  Notes: const DYNAMIC_DELAY_MAX_NON_RANK_0_BLOCKS has been increased from 5 to 10. With this, the dynamic block maker delay will be triggered less frequently.
  Review: Code changes look good and match release notes.
• e3c408cd0 Consensus,Interface(consensus): push all ingress messages (#2233)
  Notes: As per current behaviour only small size message blow 1024 bytes are pushed. All the large message rely on advert and then request to download, and hence add to latency. Is_latency_sensitive is now set to true, to push all the messages.
  Review: Code changes look good and match release notes.
• c5e9e2af1 Consensus,Interface(consensus): reduce consensus bouncer function refresh period from 3 seconds to 1 second. (#1885)
  Notes: The refresh period in Consensus_Bouncer has been changed to 1s from 3s. This will allow nodes to catch up more quickly ( 10 blocks per second theoretically ).
  Review: Code changes look good and match release notes.
• 237990ccb Crypto,Interface(crypto): skip ingress_expiry check for anonymous queries and read_state requests (#1768)
  Notes: ingress_expiry checks are now skipped for all anonymous query and read state requests, as per the specification #343
  Review: Code changes look good and match release notes.
• 161b12ce8 Crypto,Interface(crypto): Add support for derivation to ecdsa_secp256r1 crate (#1730)
  Notes: Added code changes to support derivation path handling. DerivationPath. DerivationIndex structures have been implemented allowing BIP32-style key derivation. Ckd, ckd_pub, derive_offset functions have been added to support the implementation.
  Review: Code changes look good and match release notes.
• d361dd692 Execution,Interface: Update cycles cost for compute (#2308)
  Notes: Significant increase in message base fee from 590K to 5M. Instruction fee from 0.4 to 1 cycle, and noted that the canister creation fee will be addressed in a separate PR. These changes are per the discussion on fees on the forum Evaluating Compute Pricing in Response to Increased Demand on the Internet Computer Protocol.
  Review: Code changes look good and match release notes.
• 74751adc0 Execution,Interface: Skip per-canister overhead if no instructions used (#2322)
  Notes: When total_instructios_used is zero, the code will skip the per-canister balance check overhead, and hence will improve the round capacity.
  Review: Code changes look good and match release notes.
• 71f3b769c Execution,Interface: [v3] Make subnet replica version of latest block available to execution environment (#2248)
  Notes: replica_version has been added to Batch to make it available to the execution environment. Various places in the code has changed to accommodate this including test cases too.
  Review: Code changes look good and match release notes.
• b29e83ac7 Execution,Interface: enable allowed_viewers feature for canister log visibility (#2244)
  Notes: The allowed_viewer_feature flag is now enabled for the canister log visibility.
  Review: Code changes look good and match release notes.
• 3006ab867 Execution,Interface: Evict sandboxes based on their RSS (#2197)
  Notes: Eviction for sandbox has been changed from percentage to based on their RSS. Eviction Count is now Set to 200 processes to balance eviction cost without overwhelming the system. Eviction Memory Threshold: Evict 1 GiB (1,024 MiB) of memory at a time. Default Memory Usage per Process: Assumes each sandbox process uses 50 MiB of memory, with updates managed asynchronously.
  Review: Code changes look good and match release notes.
• 39f67a628 Interface(crypto): Introduce master key ID variant for vetKD (#2108)
  Notes: This is one of many changes introduced to support VetKeys on ICP. This change introduces master key ID variant. pub struct VetKdKeyId has been added, and later made part of MasterPublicKeyId
  Review: Code changes look good and match release notes.
• 0515bc380 Interface,Message Routing: Introduce a minimum supported certification version. (#1502)
  Notes: This change is one of the step to phase out old certification versions. MIN_SUPPORTED_CERTIFICATION_VERSION constant has been added and set to V17.
  Review: Code changes look good and match release notes.
• fa22c9fa9 Interface,Message Routing: Introduce msg_limit for application subnets in payload builder (#1798)
  Notes: msg_limit hs been added to stream_slice to make sure it does not exceed a specified minimum. This is endured by bounding the stream slice to a max_message_limit upperbound and stream_begin as a lower bound.
  Review: Code changes look good and match release notes.
• ac6c9d1e5 Interface,Message Routing: Reduce the limit for messages in Stream from 50k to 10k. (#2185)
  Notes: Max stream message has been reduced to 10K from 50K as paer practical usage patterns.
  Review: Code changes look good and match release notes.
• 8c2e0c98e Interface,Networking(https-outcalls): re-enable H/2 support for outcalls (#2200)
  Notes: This change reintroduces http2 for in the builder.
  Review: Code changes look good and match release notes.
• 993fc8586 Interface,Node: initial draft of custom metric tool and its systemd timer (#1963)
  Notes: A custom metric_tool has been added to be exported to node_exporter through textfile collector. This tool collects specific metrics TLB shootdowns and export them in Prometheus format.
  Review: Code changes look good and match release notes.

Bugfixes:

• f306fcb8c Consensus,Interface(recovery): Use metric for highest validated certification for determining node heights (#2332)
  Notes: certification_share_height has been added to node_heights. This was a bug previously where wrong metric was used to calculate highest agreed upon state. certification_share_height is not taken into account in artifact_pool_certification_height_stat stat.
  Review: Code changes look good and match release notes.
• 3d590ec5a Consensus,Interface: remove the async locks from the orchestrator (#2325)
  Notes: Locks have been removed at various places in the orchestrator. Code changes show the removal of .await()
  Review: Code changes look good and match release notes.
• e70f04d9f Consensus,Interface(recovery): Print example of expected ssh key format during recovery (#2187)
  Notes: Changes to the console to add the expected ssh key format during recovery.
  Review: Code changes look good and match release notes.
• 55297ccfd Consensus,Interface: update ic.json5 nns_urls references (#2192)
  Notes: This change addresses some missed places of changing nns_url to nns_urls ( from previous proposals ).
  Review: Code changes look good and match release notes.
• ec12a4eea Interface,Networking(https-outcalls): Increase allowed header size limit for HTTP/2 requests (#2292)
  Notes: .http2_max_header_list_size(MAX_HEADER_LIST_SIZE) has been added to Client Builder to support calls with header size more than 16KB.
  Review: Code changes look good and match release notes.
• 218fdbcaa Interface,Networking: remove the idle future (#2285)
  Notes: The future that returns when the adapter becomes idle was implemented in a very complex way. The change has been removed.
  Review: Code changes look good and match release notes.
• 52bc5275e Interface,Networking: call make_idle if the adapter is idle initially (#2278)
  Notes: Enables the adapter to stay idle until it detects activity, minimizing resource usage when no new events occur.
  Reviews: Code changes look good and match release notes
• b56d5e1ba Interface,Networking: synchronize properly with tokio::Notify instead of sleeping (#2178)
  Notes: Code changes have been made to AdapterState where idle_seconds has been introduced to check for how long a adapter should wait before becoming idle. Two methods idle and active have been added that use the watch channel to monitor and manage the adapter.
  Review: Code changes look good and match release notes.
• 582ce51a6 Interface,Node: Fix confusing logging of ipmitool (#2297)
  Notes: A fix was made to remove confusing logging on successful retrieval of mac address.
  Review: Code changes look good and match release notes.
• 677aa6862 Owners(IDX): bump to thiserror v1.0.65 in Bazel (#2305)
  Notes: thiserror is bumped from 1.0.64 to 1.0.65
  Review: Code changes look good and match release notes.
• 7be43416d Owners(IDX): bump to typenum v1.17.0 in Bazel (#2296)
  Notes: typenum has been bumped to 1.17.
  Reviews: Code changes look good and match release notes.
• 53f97714f Owners(IDX): work around spurious rebuilds in rustix (#2287)
  Notes: rustix patches has been made for determinism.
  Reviews: Code changes look good and match release notes.
• dac2f36f9 Node: Make regex for special address ranges more strict (#2315)
  Notes: Changes have been made to regex to filter out special addresses more strict.
  Reviews: Code changes look good and match release notes.
• 361d09aeb Node: Improve prestorecon performance (#2218)
  Notes: selabel_open call has been moved out of the for loop to improve performance.
  Reviews: Code changes look good and match release notes.
• 3753b4388 Node(setupos): update default ipv6 prefix and gw for setupos testnets (#1805)
  Notes: A minor change in setupos gateway for testnets.
  Review: Code changes look good and match release notes.

Chores:

• 80dc25b52 Consensus,Interface(consensus): move notary related functions from consensus_utils crate to notary.rs in consensus crate (#2310)
  Notes: Its a simple change to move notary related changes from concensus_utils crate to notary.rs in consensus crate for maintainability.
  Review: Code changes look good and match release notes.
• d3cfffac1 Consensus,Interface(consensus): Replace generic pool metrics in certification pool (#2312)
  Notes :struct PerTypeMetrics has been added and later used in PoolMetrics struct. It helps measure max_height, min_height, count and enhances observability of pool states.
  Review: Code changes look good and match release notes.
• 4f7cebee1 Consensus,Interface: Don’t log default replica version in batch delivery (#2276)
  Notes: Instead of logging the default version, the actual version of the block being delivered is logged.
  Review: Code changes look good and match release notes.
• 10150fb34 Consensus,Interface(consensus): Rename persistent certification pool section for consistency (#2262)
  Notes: A minor renaming from persistent_pool to validated for more consistency.
  Review: Code changes look good and match release notes.
• c5e6242f5 Crypto,Interface: remove unneeded clippy allows (#2326)
  Notes: #[allow(clippy::needless_collect)] directive was used in many files and is now removed to allow expect and unwrap in tests.
  Review: Code changes look good and match release notes.
• 4899bba2c Interface: upgrade tower-rs crates (#1773)
  Notes: tower-rs has been upgraded to 0.5.1 along with some minor bumps to tower-http to 0.6.1
  Review: Code changes look good and match release notes.
• 54f0fae1c Interface: disallow async locks (RwLock) (#2144)
  Notes: #![allow(clippy::disallowed_types)] directive has been added to multiple files to suppress lint warnings for tokio::sync::Mutex and tokio::sync::RwLock.
  Review: Code changes look good and match release notes.
• c84916512 Interface,Message Routing: Keep extra in-memory states (#2061)
  Notes: The change has been made to make memory usage efficient by removing in-memory states more eagerly at checkpointed heights, while keeping some necessary states for CUP creation and validation. This is of the changes, and many more will follow. This particular change adds one more parameter to remove_inmemory_states_below() &BTreeSet::new() allowing some states to be retained.
  Review: Code changes look good and match release notes.
• 619190192 Owners(IDX): bump rules_rust to 0.53.0 (#2307)
  Notes: rules_rust has been bumped to 0.53.0
  Review: Code changes look good and match release notes.
• 35a25eaf6 Owners(IDX): proc-macro2: 1.0.88 → 1.0.89 & remove rules_rust.patch (#2224)
  Notes: proc-macro2-1.0.89 has the fix for build reproducibility issue and hence rules_rust.patch is now removed.
  Review: Code changes look good and match release notes.
• 7dede601c Node: add more logging for check-network.sh (#2302)
  Notes: Improvements have been made to eval_command_with_retries() command to capture additional information ‘ip -6 route show’ and ${dns_servers}".
  Review: Code changes look good and match release notes.
• 60ce9d351 Node: revert removal of old API BN certificates (#2209)
  Notes: It is a revert of fix: add a temporary measure to remove existing API BN certs by r-birkner · Pull Request #2146 · dfinity/ic · GitHub
  Review: Code changes look good and match release notes.
• 36cbd803b Node: Update Base Image Refs [2024-10-24-0808] (#2229)
  Notes: Updates base image references
  Review: Code changes look good and match release notes.

Refactoring:

• 6c4efce2e Interface,Message Routing: split load_checkpoint (#2210)
  Notes: It’s a refactoring and splits load_checkpoint function for readability and also enabling async loading of protobufs for validation.
  Review: Code changes look good and match release notes.

Proposal: 133901

Summary:

1. The build hash in CDN, proposal and local build matches and is “dfe619494d935c496e81ad717219b2cff56d296898ea309296b65863d42cc582”.
2. Vote: Voting yes to adopt the proposal
   
   

Features:

cc1319059 Interface,Networking(Consensus): Enable the hashes-in-blocks feature
Notes: HASHES_IN_BLOCKS_FEATURE_ENABLED is set to true.
Review: Code changes look good and match release notes.

Proposal: 133902

Summary:

1. The build hash in CDN, proposal and local build matches and is “155b99ea7ddbf3491404f5221b7240affd9287fa90b0eef76c36d6f94919e9dc”.
2. Vote: Voting yes to adopt the proposal
   
   

Other changes:

• f910b32ef Node: Squashed commit of the following: Try using 6.11 kernel. Use 6.11 base images
  Notes: base image reference updates and use of kernel 6.11
  Review: Code changes look good and match release notes.

[timk11]

Proposal 133900

Vote: Adopt

Reason: Build is successful, hashes match, commits match descriptions and the reasoning behind the changes is sound.

As this is my first review in this topic, some of the allocated time was spent setting up a server structure for verification (which took longer than expected due to some glitches) and bringing myself up to speed on the current state of the protocol. For this review I’ve focused selectively on consensus-related commits with the intention of broadening the focus for future reviews.

Review

Features:

[e7b16eab4]
Increases DYNAMIC_DELAY_MAX_NON_RANK_0_BLOCKS from 5 to 10, meaning that once 10 (up from 5) non-rank-0 blocks have been notarised within the previous 30 rounds, a dynamic delay of up to 10 seconds is triggered in which blocks from rank-0 block makers have priority, in order to combat a scenario in which malicious nodes are able to force their own blocks to be notarised. The threshold is being increased because the delay was being triggered a bit too often, leading to potential network slowing.

[e3c408cd0]
Changes the is_latency_sensitive flag from false to true, with the effect that all ingress messages are pushed to nodes in full, instead of only those messages smaller than 1024 bytes with adverts being used for larger messages, in order to reduce latency.

[c5e9e2af1]
Reduces the consensus bouncer function refresh period in order to allow faster catch-up for lagging nodes, which is necessary now that all subnets bar one have had their notarisation delay reduced to 0.3 seconds and can therefore can produce a significantly faster rate of blocks.

[39f67a628]
Adds VetKd variant to the MasterPublicKeyId enum as the first step in the implementation of the VetKeys encryption feature.

Bugfixes:

[f306fcb8c]
Adds field certification_share_height to NodeMetrics. Within the recovery get_node_metrics function, which fetches a response from nodes, the certification_height metric is now taken from the validated artifact pool (as is the new metric) instead of using the last certified height from the state manager.

[3d590ec5a]
Removes async / await from several lines in the orchestrator code + related minor changes.

[e70f04d9f]
Adds format and example to printed instructions to enter an SSH key.

[55297ccfd]
Updates 2 references from nns_url to nns_urls in keeping with an earlier update.

Chores:

[80dc25b52]
Moves get_adjusted_notary_delay and get_adjusted_notary_delay_from_settings functions from rs/consensus/utils/src/lib.rs to rs/consensus/src/consensus/notary.rs + related code changes.

[d3cfffac1]
Changes some of the metrics for the certification pool. In particular, adds “per type” metrics for the height of objects and number of artifacts in a certification pool.

[4f7cebee1]
Replaces current_replica_version with block.version() in deliver_batches function for the purpose of logging a replica version during batch delivery, and removes a redundant second log.

[10150fb34]
Renames certification_pool.persistent_pool to certification_pool.validated.

Proposal 133901

Vote: Adopt

Reason: Build is successful. Hashes match. Only one commit, matching description.

[cc1319059]
Switches the HASHES_IN_BLOCKS_FEATURE_ENABLED flag to true in order to implement the hashes-in-block feature. This feature adds only the hashes of ingress messages to blocks instead of the messages in full, with the aim of increasing throughput, and is explained in detail here.

Proposal 133902

Vote: Adopt

Reason: Build is successful. Hashes match. Only one commit, matching description.

[f910b32ef]
Upgrades Linux kernel version to 6.11 and installs the kernel. (The commit notes should specify “Linux kernel” but do not.)

[yuvika]

Proposal: 133900

Summary:

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits.
   
   

Commits Review

Features:

1. e7b16eab4
   Summary:
   Consensus, Interface(consensus): adjust dynamic block maker delay.
   Notes:
   Update the DYNAMIC_DELAY_MAX_NON_RANK_0_BLOCKS variable value from 5 to 10. The delay will occur less frequently as the threshold of maximum allowable non-rank-0 notarized blocks is increased.
   Review:
   The description matches the code changes.

2. e3c408cd0
   Summary:
   Consensus, Interface(consensus): push all ingress messages.
   Notes:
   Change the boolean is_latency_sensitive from false to true to enable the functionality wherein ingress messages are pushed directly reducing latency. This may increase the bandwidth needed. On the other hand earlier, only smaller ingress messages would be pushed directly and larger ones would have higher latency since they required both an advert and a request from a peer.
   Review:
   The description matches the code changes.

3. c5e9e2af1
   Summary:
   Consensus, Interface(consensus): reduce consensus bouncer function refresh period from 3 seconds to 1 second.
   Notes:
   Reduce refresh period from 3s to 1s. It will check the pool every 1s for unnecessary or expired ingress messages. In theory, it can process up to 10 blocks per second.
   Review:
   The description matches the code changes.

4. 237990ccb
   Summary:
   Crypto, Interface(crypto): skip ingress_expiry check for anonymous queries and read_state requests.
   Notes:
   Validate non-anonymous expired queries and read_state requests only and skip for anonymous ones.
   Review:
   The description matches the code changes.

5. 161b12ce8
   Summary: Crypto, Interface(crypto): Add support for derivation to ecdsa_secp256r1 crate.
   Notes:
   Add support for having derivation available in the 3 signature utility crates so that the dependency on the internal threshold ECDSA protocol implementation can be removed.
   Review:
   The description matches the code changes.

6. d361dd692
   Summary:
   Execution, Interface: Update cycles cost for compute.
   Notes:
   This adjusts the fees to better reflect the system load. The base message execution fee, the per instruction cycle fee, etc. were updated. UNIVERSAL_CANISTER_CYCLE_MARGIN was increased from 7M to 42M, update_message_execution_fee was increased from 590K to 5M and ten_update_instructions_execution_fee was increased from 4 to 10.
   Review:
   The description matches the code changes.

7. 74751adc0
   Summary:
   Execution, Interface: Skip per-canister overhead if no instructions used.
   Notes:
   Per canister total_instructions_used = 0 means insufficient cycle balance. Executions will fail due to insufficient cycle balance without additional overhead. Update tests.
   Review:
   The description matches the code changes.

8. 71f3b769c
   Summary:
   Execution, Interface: [v3] Make subnet replica version of latest block available to execution environment.
   Notes:
   Add ReplicaVersion to the execution environment. The new field is added to the Batch struct.
   Review:
   The description matches the code changes.

9. b29e83ac7
   Summary:
   Execution, Interface: enable allowed_viewers feature for canister log visibility.
   Notes:
   Enable the allowed_viewers flag for canister log visibility.
   Review:
   The description matches the code changes.

10. 3006ab867
    Summary:
    Execution, Interface: Evict sandboxes based on their RSS.
    Notes:
    Increase DEFAULT_MAX_SANDBOX_COUNT from 2k to 5k. Evict sandbox process on memory-based thresholds.
    Review:
    The description matches the code changes.

11. 39f67a628
    Summary:
    Interface(crypto): Introduce master key ID variant for vetKD.
    Notes:
    Introduce the vetKeys feature for end-to-end encryption on the IC. This is not complete yet. The scope of this change is to add a new variant, VetKd, to the MasterPublicKeyId.
    Review:
    The description matches the code changes.

12. 0515bc380
    Summary:
    Interface, Message Routing: Introduce a minimum supported certification version.
    Notes:
    Phase out old certificate versions. Update MIN_SUPPORTED_CERTIFICATION_VERSION to V17.
    Review:
    The description matches the code changes.

13. fa22c9fa9
    Summary:
    Interface, Message Routing: Introduce msg_limit for application subnets in payload builder.
    Notes:
    Introduce a message limit on the number of messages inducted in a stream slice. This limit is defined by slice.header.begin +
    MAX_STREAM_MESSAGES, where MAX_STREAM_MESSAGES=10k.
    Review:
    The description matches the code changes.

14. ac6c9d1e5
    Summary:
    Interface, Message Routing: Reduce the limit for messages in Stream from 50k to 10k.
    Notes:
    Decrease MAX_STREAM_MESSAGES to 10k from 50k which is more realistic and accounts for bottlenecks.
    Review:
    The description matches the code changes.

15. 8c2e0c98e
    Summary:
    Interface, Networking(https-outcalls): re-enable H/2 support for outcalls.
    Notes:
    Re-enable http/2 functionality, which was earlier causing a test failure. The issue was due to a bug in the test server, which is now fixed.
    Review:
    The description matches the code changes.

16. 993fc8586
    Summary:
    Interface, Node: initial draft of custom metric tool and its systemd timer.
    Notes:
    Introduce a performance monitoring tool for reporting custom metrics collected per minute. Metrics such as sum_tlb_shootdowns will be calculated and exported to Prometheus’s node_exporter.
    Review:
    The description matches the code changes.

Bugfixes:

1. f306fcb8c
   Summary:
   Consensus, Interface(recovery): Use metric for highest validated certification for determining node heights.
   Notes:
   During recovery, fetch the highest certification in the validated pool. Add a new field certification_share_height in NodeMetrics.
   Review:
   The description matches the code changes.

2. 3d590ec5a
   Summary:
   Consensus, Interface: remove the async locks from the orchestrator.
   Notes:
   Remove calls to await(), to get rid of async locks from orchestrator. And use synchronous RwLock instead.
   Review:
   The description matches the code changes.

3. e70f04d9f
   Summary:
   Consensus, Interface(recovery): Print example of expected ssh key format during recovery.
   Notes: Log an example of the expected ssh key format during recovery.
   Review:
   The description matches the code changes.

4. 55297ccfd
   Summary:
   Consensus, Interface: update ic.json5 nns_urls references.
   Notes:
   Rename nns_url to nns_urls in ic.json.template.
   Review:
   The description matches the code changes.

5. ec12a4eea
   Summary:
   Interface, Networking(https-outcalls): Increase allowed header size limit for HTTP/2 requests.
   Notes:
   Set MAX_HEADER_LIST_SIZE to = 52KiB, which is higher than the spec of 48KiB. This value was increased as responses to https outcalls with headers greater than 16KB failed.
   Review:
   The description matches the code changes.

6. 218fdbcaa
   Summary:
   Interface, Networking: remove the idle future.
   Notes:
   Revert the complex “idle” future due to complexity.
   Review:
   The description matches the code changes.

7. 52bc5275e
   Summary:
   Interface, Networking: call make_idle if the adapter is idle initially.
   Notes: To minimize resource usage, the adapter will stay idle until activity is detected.
   Review:
   The description matches the code changes.

8. b56d5e1ba
   Summary:
   Interface, Networking: synchronize properly with tokio::Notify instead of sleeping.
   Notes:
   Add a new field idle_seconds to the AdapterState struct which will contain information on how long the adapter should wait before becoming idle. Also, two new asynchronous methods were added, idle and active return when/if the adapter becomes/is idle and when/if the adapter becomes/is awake respectively.
   Review:
   The description matches the code changes.

9. 582ce51a6
   Summary:
   Interface, Node: Fix confusing logging of ipmitool.
   Notes:
   Fix the confusing logging of the ipmitool which exits with an error even if the necessary information (mac address) has been extracted.
   Review:
   The description matches the code changes.

10. 677aa6862
    Summary:
    Owners(IDX): bump to thiserror v1.0.65 in Bazel.
    Notes:
    Update thiserror version from 1.0.64 to 1.0.65.
    Review:
    The description matches the code changes.

11. 7be43416d
    Summary:
    Owners(IDX): bump to typenum v1.17.0 in Bazel.
    Notes:
    Update typenum version from v1.16.0 to v1.17.0.
    Review:
    The description matches the code changes.

12. 53f97714f
    Summary:
    Owners(IDX): work around spurious rebuilds in rustix.
    Notes:
    rustix patch to ensure determinism.
    Review:
    The description matches the code changes.

13. dac2f36f9
    Summary:
    Node: Make regex for special address ranges more strict.
    Notes:
    Update the regex rule for IPv6 addresses to filter out ::1 and fe80... but not ...fe80... and ...::1....
    Review:
    The description matches the code changes.

14. 361d09aeb
    Summary:
    Node: Improve prestorecon performance.
    Notes:
    Call selabel_open once and share a single selabel handle to improve performance in prestorecon.cc.
    Review:
    The description matches the code changes.

15. 3753b4388
    Summary:
    Node(setupos): update default ipv6 prefix and gw for setupos testnets.
    Notes:
    Update IPv6 network configuration,ipv6_prefix and ipv6_gateway from 2a00:fb01:400:200 to 2a00:fb01:400:44::1
    Review:
    The description matches the code changes.

Chores:

1. 80dc25b52
   Summary:
   Consensus, Interface(consensus): move notary related functions from consensus_utils crate to notary.rs in consensus crate.
   Notes:
   Moved notary-related functions from the consensus_utils crate directly into the notary.rs file within the consensus crate to reduce dependency on consensus_utils as these functions are only used by the notary.
   Review:
   The description matches the code changes.

2. d3cfffac1
   Summary:
   Consensus, Interface(consensus): Replace generic pool metrics in certification pool.
   Notes:
   Add a new struct PerTypeMetrics struct, which tracks the minimum, maximum, and count metrics for artifacts in a certification pool. Replace the usage of PoolMetrics with that of the updated one.
   Review:
   The description matches the code changes.

3. 4f7cebee1
   Summary:
   Consensus, Interface: Don’t log default replica version in batch delivery.
   Notes:
   Log the version of the block instead of passing the default version to deliver_batches and remove a redundant debug log in deliver_batches.
   Review:
   The description matches the code changes.

4. 10150fb34
   Summary:
   Consensus, Interface(consensus): Rename persistent certification pool section for consistency.
   Notes:
   Rename persistent_pool to validated in CertificationPoolImpl.
   Review:
   The description matches the code changes.

5. c5e6242f5
   Summary:
   Crypto, Interface: remove unneeded clippy allows.
   Notes:
   Remove directive #[allow(clippy::needless_collect)] from various files after allow-expect-in-tests = true and allow-unwrap-in-tests = true .
   Review:
   The description matches the code changes.

6. 4899bba2c
   Summary:
   Interface: upgrade tower-rs crates.
   Notes:
   Update tower version from 0.4.13 to 0.5.1 and tower-http version from 0.5.2 to 0.6.1.
   Review:
   The description matches the code changes.

7. 54f0fae1c
   Summary:
   Interface: disallow async locks (RwLock).
   Notes:
   Add a new rule #![allow(clippy::disallowed_types)] to clippy.toml to disallow the async tokio::sync::RwLock and tokio::sync::Mutex type.
   Review:
   The description matches the code changes.

8. c84916512
   Summary:
   Interface, Message Routing: Keep extra in-memory states.
   Notes:
   Pass an additional argument extra_heights_to_keep to remove_inmemory_states_below, this way states that are below the standard height threshold can be retained. The interface, APIs and tests in the state manager have been added. The goal is to improve the memory usage.
   Review:
   The description matches the code changes.

9. 619190192
   Summary:
   Owners(IDX): bump rules_rust to 0.53.0.
   Notes:
   Update rules_rust in WORKSPACE.bazel from v0.42.1 to v0.53.0.
   Review:
   The description matches the code changes.

10. 35a25eaf6
    Summary:
    Owners(IDX): proc-macro2: 1.0.88 → 1.0.89 & remove rules_rust.patch.
    Notes:
    Update the proc-macro2 version from 1.0.88 to 1.0.89 and corresponding checksum for verification.
    Review:
    The description matches the code changes.

11. 7dede601c
    Summary:
    Node: add more logging for check-network.sh.
    Notes:
    Improve logging in the check-network.sh script. Update eval_command_with_retries() method, which can get information such as ip6_output, ip6_route_output and dns_servers.
    Review:
    The description matches the code changes.

12. 60ce9d351
    Summary:
    Node: revert removal of old API BN certificates.
    Notes:
    Revert commit c6f645a.
    Review:
    The description matches the code changes.

13. 36cbd803b
    Summary:
    Node: Update Base Image Refs [2024-10-24-0808].
    Notes:
    Update base container image references.
    Review:
    The description matches the code changes.

Refactoring:

1. 6c4efce2e
   Summary:
   Interface, Message Routing: split load_checkpoint.
   Notes:
   Refactor the load_checkpoint method for readability and enable asynchronous loading of protobufs for validation.
   Review:
   The description matches the code changes.

Proposal 133901

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits
   
   

Commits Review

Features:

1. [cc1319059]
   Summary:
   Interface, Networking(Consensus): Enable the hashes-in-blocks feature.
   Notes:
   Update the HASHES_IN_BLOCKS_FEATURE_ENABLED variable to true. When this variable is true, all ingress messages will be stripped before sending them to peers. The blocks will be reconstructed on the receiver side by looking up the referenced ingress messages in the ingress pool or fetching missing ingress messages.
   Review:
   The description matches the code changes.

Proposal 133902

Summary

1. Vote: Adopt
2. Hash: Hashes match
3. Reasons to adopt: Builds fine + hashes match + release notes match the commits
   
   

Commits Review

Other Changes:

1. f910b32ef
   Summary:
   Node: Squashed commit of the following: Try using 6.11 kernel. Use 6.11 base images
   Notes:
   Update the base docker file, download and install Linux Kernel version 6.11 and use the new image with the latest changes.
   Review:
   The description matches the code changes.

[ipsita]

Proposal 133900

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Features

e7b16eab4
Summary: Adjust dynamic block marker delay
Notes
updates DYNAMIC_DELAY_MAX_NON_RANK_0_BLOCKS from 5 to 10 this means that the delay will still be triggered but less frequently now.
Review: Code changes matches the commit message.

e3c408cd0
Summary: Push all ingress messages
Notes
Before this change we only push ingress messages which have a size less than 1024 bytes this means for the messages which are more than 1024 bytes the node have the request to download it from the peer which adds latency in the range of 100’s of milliseconds. Now with this change all the message are pushed to the ingress pool which means that we save of latency but the trade off is that some messages received by the node may have been expired but this would be a very rare occurrence since now we actively remove expired messages from the ingress pool.
Review: Code changes matches the commit message.

c5e9e2af1
Summary: Reduce consensus bouncer function refresh period from 3 seconds to 1 second
Notes
Reduces the refresh period from 3 second to 1 seconds so that the node which is lagging behind can catchup quickly, in theory catching up-to 10 blocks per second.
Review: Code changes matches the commit message.

237990ccb
Summary: skip ingress_expiry check for anonymous queries and read_state requests
Notes
In this change ingress expiry check is skipped for anonymous messages queries and anonymous read_state requests.
Review: Code changes matches the commit message.

161b12ce8
Summary: Add support for derivation to ecdsa_secp256r1 crate
Notes
Implement derivation paths using DerivationIndex and DerivationPath for BIP32-style key generation. derive_subkey and derive_subkey_with_chain_code methods in PrivateKey and PublicKey, hierarchical key derivation is streamlined. The ckd and ckd_pub functions support SLIP-10 compatible key generation along the path.
Review: Code changes matches the commit message.

d361dd692
Summary: Update cycles cost for compute
Notes
Updates various cycle fees
Increases UNIVERSAL_CANISTER_CYCLE_MARGIN from 7M to 42M
Increases update_message_execution_fee from 590K cycles to 5M cycles.
Increase instruction execution fee from 0.4 cycles to 1 cycle per instruction
Review: Code changes matches the commit message.

74751adc0
Summary: Skip per-canister overhead if no instructions used
Notes
Skips canister execution if there are not enough cycles without additional overhead, this does not impact round capacity.
Review: Code changes matches the commit message.

71f3b769c
Summary: Make subnet replica version of latest block available to execution environment
Notes
The replica_version has been integrated into Batch, making it accessible within the execution environment. Numerous code sections, including test cases, have been updated to support this addition.
Review: Code changes matches the commit message.

b29e83ac7
Summary: Enable visibility control for canister logs with allowed_viewers
Notes: The allowed_viewers flag has been introduced to manage visibility settings for canister logs.
Review: Code changes match the commit message.

3006ab867
Summary: Sandbox eviction based on RSS memory usage
Notes: The sandbox eviction policy has shifted from a percentage-based method to using Resident Set Size (RSS) for memory-based evictions. The limit is set to handle 200 processes at a time, with 1 GiB of memory being freed per eviction cycle. Each sandbox process is assumed to use about 50 MiB, with updates handled asynchronously.
Review: Code changes match the commit message.

39f67a628
Summary: Add master key ID variant for VetKD support
Notes: This update introduces a master key ID variant to support VetKeys on ICP, adding the VetKdKeyId struct, which integrates with MasterPublicKeyId.
Review: Code changes match the commit message.

0515bc380
Summary: Introduce minimum supported certification version
Notes: To phase out outdated certification versions, a new constant MIN_SUPPORTED_CERTIFICATION_VERSION has been set to V17.
Review: Code changes match the commit message.

fa22c9fa9
Summary: Add msg_limit to application subnet payload builder
Notes: The addition of msg_limit ensures message limits are enforced within stream_slice, with both upper and lower bounds applied for message control.
Review: Code changes match the commit message.

ac6c9d1e5
Summary: Adjust stream message limit from 50k to 10k
Notes: The maximum allowable messages in streams has been reduced to 10,000, reflecting typical usage patterns.
Review: Code changes match the commit message.

8c2e0c98e
Summary: Re-enable HTTP/2 for HTTPS outcalls
Notes: HTTP/2 support has been restored in the builder, improving networking efficiency.
Review: Code changes match the commit message.

993fc8586
Summary: Initial draft of custom metrics tool with systemd timer
Notes: A custom metrics tool has been added to export specific metrics, such as TLB shootdowns, to node_exporter in a Prometheus-compatible format.
Review: Code changes match the commit message.

Bugfixes

f306fcb8c
Summary: Utilize correct metric for determining node heights in certification
Notes: Added certification_share_height to node_heights to correct a previous issue where the wrong metric was used in calculating the highest validated certification state. This metric is now included in artifact_pool_certification_height_stat.
Review: Code changes match the commit message.

3d590ec5a
Summary: Remove async locks from orchestrator
Notes: Async locks have been removed in various sections of the orchestrator, reflected by the elimination of .await() calls in the code.
Review: Code changes match the commit message.

e70f04d9f
Summary: Add example of expected SSH key format for recovery
Notes: Console output has been modified to display the expected SSH key format during recovery for improved clarity.
Review: Code changes match the commit message.

55297ccfd
Summary: Update ic.json5 to use nns_urls consistently
Notes: Adjusted references from nns_url to nns_urls in ic.json5, addressing missed instances from earlier changes.
Review: Code changes match the commit message.

ec12a4eea
Summary: Increase HTTP/2 request header size limit
Notes: http2_max_header_list_size(MAX_HEADER_LIST_SIZE) has been added to the Client Builder to accommodate requests with headers exceeding 16KB.
Review: Code changes match the commit message.

218fdbcaa
Summary: Simplify idle future handling in networking
Notes: The idle future implementation, which was overly complex, has been removed for improved code simplicity and readability.
Review: Code changes match the commit message.

52bc5275e
Summary: Enable adapter to remain idle until activity is detected
Notes: Modified the adapter to stay in an idle state until it detects activity, which conserves resources when no events are occurring.
Review: Code changes match the commit message.

b56d5e1ba
Summary: Synchronize adapter state with tokio::Notify
Notes: AdapterState has been updated to introduce idle_seconds, determining how long the adapter should wait before transitioning to idle. Two methods, idle and active, use a watch channel to manage the adapter state.
Review: Code changes match the commit message.

582ce51a6
Summary: Clarify logging for ipmitool in node
Notes: Adjusted logging for ipmitool to remove potentially confusing messages when successfully retrieving a MAC address.
Review: Code changes match the commit message.

677aa6862
Summary: Update thiserror to version 1.0.65
Notes: Updated thiserror dependency from version 1.0.64 to 1.0.65 in Bazel.
Review: Code changes match the commit message.

7be43416d
Summary: Bump typenum to version 1.17
Notes: Upgraded the typenum library to version 1.17 in Bazel.
Review: Code changes match the commit message.

53f97714f
Summary: Resolve rebuild issues in rustix
Notes: Applied patches to rustix to improve build determinism and prevent spurious rebuilds.
Review: Code changes match the commit message.

dac2f36f9
Summary: Strengthen regex for special address range filtering
Notes: Enhanced regex patterns to more strictly filter out special address ranges.
Review: Code changes match the commit message.

361d09aeb
Summary: Optimize prestorecon performance
Notes: Moved the selabel_open call outside of the loop to boost performance.
Review: Code changes match the commit message.

3753b4388
Summary: Update IPv6 prefix and gateway for setupos testnets
Notes: Made a minor adjustment to the default IPv6 gateway settings for setupos test networks.
Review: Code changes match the commit message.

Chores

80dc25b52
Summary: Move notary-related functions to notary.rs in consensus crate
Notes: Simplifies maintainability by relocating notary-related functions from the consensus_utils crate to the notary.rs file within the consensus crate.
Review: Code changes match the commit message.

d3cfffac1
Summary: Replace generic pool metrics in certification pool
Notes: Introduces PerTypeMetrics to improve PoolMetrics by tracking max and min heights, count, and providing enhanced observability of pool states.
Review: Code changes match the commit message.

4f7cebee1
Summary: Log actual replica version instead of default in batch delivery
Notes: Adjusts logging to display the specific version of the block being delivered rather than the default version.
Review: Code changes match the commit message.

10150fb34
Summary: Rename certification pool section for consistency
Notes: Minor renaming of persistent_pool to validated to enhance naming consistency.
Review: Code changes match the commit message.

c5e6242f5
Summary: Remove unnecessary clippy allowances
Notes: Removed the #[allow(clippy::needless_collect)] directive, allowing expect and unwrap within tests.
Review: Code changes match the commit message.

4899bba2c
Summary: Upgrade tower-rs dependencies
Notes: Updated tower-rs to version 0.5.1, along with a minor bump for tower-http to version 0.6.1.
Review: Code changes match the commit message.

54f0fae1c
Summary: Disallow async locks
Notes: Added #![allow(clippy::disallowed_types)] to multiple files to suppress warnings related to tokio::sync::Mutex and tokio::sync::RwLock.
Review: Code changes match the commit message.

c84916512
Summary: Improve memory management by keeping essential in-memory states
Notes: Adjusts in-memory state handling to improve memory usage by removing states more aggressively at checkpointed heights while retaining necessary states for CUP creation and validation. Adds an additional parameter to remove_inmemory_states_below() and BTreeSet::new() to support this functionality.
Review: Code changes match the commit message.

619190192
Summary: Bump rules_rust to version 0.53.0
Notes: Updated the rules_rust dependency to version 0.53.0.
Review: Code changes match the commit message.

35a25eaf6
Summary: Update proc-macro2 and remove patch
Notes: Upgraded proc-macro2 to version 1.0.89, which resolved a build reproducibility issue, allowing removal of rules_rust.patch.
Review: Code changes match the commit message.

7dede601c
Summary: Enhance logging in check-network.sh
Notes: Added improvements to eval_command_with_retries() to capture additional information such as ip -6 route show and ${dns_servers} for troubleshooting.
Review: Code changes match the commit message.

60ce9d351
Summary: Revert removal of older API BN certificates
Notes: Reverts a previous fix that temporarily removed older API Boundary Node certificates.
Review: Code changes match the commit message.

36cbd803b
Summary: Update base image references
Notes: Refreshed base image references to the latest versions.
Review: Code changes match the commit message.

Refactoring

6c4efce2e
Summary: Refactor load_checkpoint for readability and async support
Notes: Refactored the load_checkpoint function to improve readability and facilitate asynchronous loading of protobufs for validation purposes.
Review: Code changes match the commit message.

Proposal 133901

Summary

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

Features

cc1319059
Summary: Enable the hashes-in-blocks feature
Notes: The HASHES_IN_BLOCKS_FEATURE_ENABLED flag has been set to true, enabling the hashes-in-blocks functionality.
Review: Code changes match the commit message.

Proposal 133902

1. Vote: Adopt
2. Hash: All the hashes match
3. Reasons to adopt: The release notes match the commits and the code changes. Builds successfully and all the hashes match

f910b32ef
Summary: Update to kernel 6.11 and use corresponding base images
Notes: Updated base image references and use 6.11 kernel version.
Review: Code changes match the commit message.

[ZackDS]

Hey @DRE-Team any inside on Proposals 133938 and 133939 ? Thanks.
Guess it’s safe to assume at this point that 133938 is a duplicate for 133901 both being (commit cc13190) release-2024-10-31_03-09-hashes-in-blocks while the 133939 is the one that was originally intended to be released on Friday but some issues with the tool delayed it ?

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133950.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-base-hotfix-snapshots (72a6598aaa193edc965e0860da731cc5af7c89e0)

This release is based on changes since release-2024-10-31_03-09-base (88227422ae6e3bfc6c74a5216309a3b86a93744b).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.
Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/72a6598aaa193edc965e0860da731cc5af7c89e0/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 72a6598aaa193edc965e0860da731cc5af7c89e0 --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133951.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-hashes-in-blocks-hotfix-snapshots (c6847128f3a872e0e084b2920bfcd21f881c69fa)

This release is based on changes since release-2024-10-31_03-09-hashes-in-blocks (cc1319059ee8283cf96481109f98d0b14b967859).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.
Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/c6847128f3a872e0e084b2920bfcd21f881c69fa/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c c6847128f3a872e0e084b2920bfcd21f881c69fa --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133952.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-ubuntu20.04-hotfix-snapshots (0111c20983290bd4a338ad22855e176803e21842)

This release is based on changes since release-2024-10-31_03-09-ubuntu20.04 (51f6f4e4ab7fa2a8ad4cf573e04fc2686e14fa57).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.
Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/0111c20983290bd4a338ad22855e176803e21842/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 0111c20983290bd4a338ad22855e176803e21842 --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133953.

Here is a summary of the changes since the last release:

Release Notes for release-2024-10-31_03-09-6.11-kernel-hotfix-snapshots (1eeb4d74deb00bd52739cbd6f37ce1dc72e0c76e)

This release is based on changes since release-2024-10-31_03-09-6.11-kernel (f910b32efbd32183962b74464b1044b900a58a5b).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.
Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/1eeb4d74deb00bd52739cbd6f37ce1dc72e0c76e/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 1eeb4d74deb00bd52739cbd6f37ce1dc72e0c76e --guestos

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

While not required for this NNS proposal, as we are only electing a new GuestOS version here, you have the option to verify the build reproducibility of the HostOS by passing --hostos to the script above instead of --guestos, or the SetupOS by passing --setupos.