Proposal to elect new release rc--2024-09-26_01-31

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133142.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-26_01-31-base (35153c7cb7b9d1da60472ca7e94c693e418f87bd)

This release is based on changes since release-2024-09-19_01-31-base (0441f40482386397f7c688bf508ddd901ca6c1b7).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Features:

• dbce2fa6d Consensus,Interface(consensus): Increase block maker delay when there have been too many non-rank-0 blocks notarized (#1531)
• a8464ac47 Execution,Interface: Execute subnet messages on aborted canisters (#1378)
• 9b242019f Execution,Interface: Propagate hook execution status to SystemState (#667)
• b36319f9e Execution,Interface: Capture backtrace in syscalls (#1505)
• a96b75d28 Execution,Interface: Canister Backtrace on Trap (#1449)
• bfd6fa8fb Execution,Interface: Validate initial wasm memory size for Wasm64 (#1534)
• da884ed19 Execution,Interface,Message Routing: Callback expiration priority queue (#1532)
• 2259be58d Execution,Interface,Networking: Enable canister snapshots (#919)
• 7f27f9e34 Interface(PocketIC): bitcoin integration (#1491)
• a9e76c402 Interface,Message Routing: maybe_parallel_map util function (#1376)
• 220baf8e1 Interface,Networking(call-v3): Return a certificate for duplicate requests that are already executed (#1523)
• b2ce10e4a Interface,Networking: Introduce metrics to the block stripper/assembler (#1488)
• 6ab95d4e5 Owners(IDX): Bump bazel to 7.0.1 (#1578)
• cc5e5060d Node: Upgrade HostOS base image to 24.04 (#1587)
• 26a62d038 Node: Update SetupOS to 24.04 (#1537)

Bugfixes:

• ff4d43607 Consensus,Interface(consensus): Correct several artifact bounds and update docs (#1074)
• 1914efda5 Consensus,Interface(consensus): Include equivocation proofs in artifacts returned by get_all_validated (#1579)
• b383408d6 Execution,Interface: fix a metric for counting actually executed canisters per round (#1596)
• ec89de506 Interface,Message Routing: Fix a race condition in StateManager tests (#1673)
• ee5a50001 Interface,Networking: start the BTC adapter only by accepting a config (#1584)
• 8d630c57d Interface,Node: Fix long-standing typo (#1602)

Performance improvements:

• 146430974 Interface,Message Routing: Defragment correct state (#1683)

Chores:

• 9e9f3653d Consensus,Interface(consensus): Add info log when producing equivocation (#1623)
• 974ec76f5 Consensus,Interface(consensus): move get_block_maker_delay function from consensus_utils crate to consensus crate (#1527)
• a368e8f26 Execution,Interface: make allowed viewers feature flag configurable and enable it for ic-starter (#1598)
• 66389f30e Interface,Networking: refine the exposed public interface of the adapters and start them in consistent way (#1622)
• 905909681 Interface,Networking: hide some structs and functions from the BTC adapter (#1617)

Refactoring:

• 09e7929d5 Interface(nervous-system): Remove ic-sns-governance and ic-sns-init’s test_feature configurations (#1606)

---

Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/35153c7cb7b9d1da60472ca7e94c693e418f87bd/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 35153c7cb7b9d1da60472ca7e94c693e418f87bd

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133143.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-26_01-31-ubuntu24 (d101161c54df27f514902d22958c77aae9ce8035)

This release is based on changes since release-2024-09-26_01-31-base (35153c7cb7b9d1da60472ca7e94c693e418f87bd).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

• d101161c5 Node: Upgrade GuestOS to 24.04

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/d101161c54df27f514902d22958c77aae9ce8035/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c d101161c54df27f514902d22958c77aae9ce8035

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

[DRE-Team]

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 133144.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-26_01-31-no-canister-snapshots (c43a4880199c00135c8415957851e823b3fb769e)

This release is based on changes since release-2024-09-26_01-31-base (35153c7cb7b9d1da60472ca7e94c693e418f87bd).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

• c43a48801 Execution,Interface,Networking: Revert “feat: Enable canister snapshots”

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/c43a4880199c00135c8415957851e823b3fb769e/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c c43a4880199c00135c8415957851e823b3fb769e

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

[Lorimer]

@DRE-Team

7f27f9e34 Interface(PocketIC): bitcoin integration (#1491)

I don’t think this is a GuestOS change. Presumably it was picked up by bazel due to the modification to rs/consensus/src/consensus/payload_builder.rs

Similarly I don’t think the HostOS and SetupOS commits are actually actioned by this proposal →

@DRE-Team

• cc5e5060d Node: Upgrade HostOS base image to 24.04 (#1587)
• 26a62d038 Node: Update SetupOS to 24.04 (#1537)

---

Additionally, I came across this commit and would’ve expected this to have been included in the proposal summary. Is it not considered a GuestOS change? → a438bb77e - perf: Defragment memory during checkpointing (#1620)

Having checked I see that this commit was intentionally removed from the proposal summary (’ Excluded by authors’). @Luka are you able to elaborate?

[hpeebles]

I have reviewed all commits listed in this proposal and in my opinion they all look fine, I have also run the build verification script which completed successfully, so I have voted to adopt the proposal.

Full review:

Features:

dbce2fa6d Consensus,Interface(consensus): Increase block maker delay when there have been too many non-rank-0 blocks notarized (#1531)
Review: Looks fine + matches description
Notes: If too many non-zero rank blocks have been notarized recently, then a delay is applied for non-zero rank blocks to ensure more zero rank blocks get finalized.

a8464ac47 Execution,Interface: Execute subnet messages on aborted canisters (#1378)
Review: Looks fine + matches description
Notes: When a checkpoint is reached, multi block executions that have not yet completed are aborted. This commit makes it such that certain management canister methods which modify state are disallowed if a canister is still processing an aborted update request.

9b242019f Execution,Interface: Propagate hook execution status to SystemState (#667)
Review: Looks fine + matches description
Notes: Adds the update_on_low_wasm_memory_hook_status to the SystemState and updates it each time the canister requests more memory from the replica.

b36319f9e Execution,Interface: Capture backtrace in syscalls (#1505)
Review: Looks fine + matches description
Notes: Adds the backtrace details to the response if a call to the SystemApi results in a trap.

a96b75d28 Execution,Interface: Canister Backtrace on Trap (#1449)
Review: Looks fine + matches description
Notes: Adds the backtrace field to HypervisorError::Trapped and populates it on receipt of a wasmtime error from a canister.

bfd6fa8fb Execution,Interface: Validate initial wasm memory size for Wasm64 (#1534)
Review: Looks fine + matches description
Notes: Adds a step when validating a wasm to check that the initial heap size does not exceed the limit imposed by the protocol.

da884ed19 Execution,Interface,Message Routing: Callback expiration priority queue (#1532)
Review: Looks fine + matches description
Notes: Adds the unexpired_callbacks queue to CallContextManager which holds all pending callbackIds ordered by their expiry date. This allows for the new (currently unused) expire_callbacks function which returns the list of callbackIds which has expired.

2259be58d Execution,Interface,Networking: Enable canister snapshots (#919)
Review: Looks fine + matches description
Notes: Sets the canister_snapshots flag to enabled by default.

7f27f9e34 Interface(PocketIC): bitcoin integration (#1491)
Review: Looks fine + matches description
Notes: Wires up the Bitcoin adapter within PocketIC, allowing PocketIC to handle Bitcoin requests.

a9e76c402 Interface,Message Routing: maybe_parallel_map util function (#1376)
Review: Looks fine + matches description
Notes: Removes duplicated code by adding the maybe_parallel_map util function which runs a function in parallel if a thread pool is provided else runs sequentially.

220baf8e1 Interface,Networking(call-v3): Return a certificate for duplicate requests that are already executed (#1523)
Review: Looks fine + matches description
Notes: Immediately returns the previous response from the new V3 call end point if an incoming request is a duplicate of one which has already been processed.

b2ce10e4a Interface,Networking: Introduce metrics to the block stripper/assembler (#1488)
Review: Looks fine + matches description
Notes: Adds various metrics to the new block stripper/assembler.

6ab95d4e5 Owners(IDX): Bump bazel to 7.0.1 (#1578)
Review: Looks fine + matches description
Notes: Bumps Bazel from 6.5.0 to 7.0.1.

cc5e5060d Node: Upgrade HostOS base image to 24.04 (#1587)
Review: Looks fine + matches description
Notes: Bumps HostOS from using Ubuntu 20.04 to using 24.04.

26a62d038 Node: Update SetupOS to 24.04 (#1537)
Review: Looks fine + matches description
Notes: Updates the SetupOS image references to use the new images built using Ubuntu 24.04 + a few other minor changes to SetupOS.

Bugfixes:

ff4d43607 Consensus,Interface(consensus): Correct several artifact bounds and update docs (#1074)
Review: Looks fine + matches description
Notes: Fixes an off by one error in one of the consensus calculations + updates documentation.

1914efda5 Consensus,Interface(consensus): Include equivocation proofs in artifacts returned by get_all_validated (#1579)
Review: Looks fine + matches description
Notes: Includes equivocation proofs when iterating the validated artifacts in the consensus pool.

b383408d6 Execution,Interface: fix a metric for counting actually executed canisters per round (#1596)
Review: Looks fine + matches description
Notes: Fixes the executed_canisters_per_round metric to only track canisters that were actually executed rather than those that were scheduled since some may not have actually been executed due to round instruction limits. Also switches the metric from a gauge to a histogram.

ec89de506 Interface,Message Routing: Fix a race condition in StateManager tests (#1673)
Review: Looks fine + matches description
Notes: Fixes a test to wait for changes to be flushed to disk before running the checks.

ee5a50001 Interface,Networking: start the BTC adapter only by accepting a config (#1584)
Review: Looks fine + matches description
Notes: Refactors the BTC adapter, exposing a new start_server function which takes arguments that allow it to be used by PocketIC.

8d630c57d Interface,Node: Fix long-standing typo (#1602)
Review: Looks fine + matches description
Notes: Literally just fixes a typo of “an” to “a”.

Performance improvements:

146430974 Interface,Message Routing: Defragment correct state (#1683)
Review: Looks fine + matches description
Notes: Fixes the canister_states reference which gets defragmented after creating each checkpoint.

Chores:

9e9f3653d Consensus,Interface(consensus): Add info log when producing equivocation (#1623)
Review: Looks fine + matches description
Notes: Logs a warning containing the full details if ever an equivocation proof is detected.

974ec76f5 Consensus,Interface(consensus): move get_block_maker_delay function from consensus_utils crate to consensus crate (#1527)
Review: Looks fine + matches description
Notes: Just moves the location of the get_block_maker_delay function.

a368e8f26 Execution,Interface: make allowed viewers feature flag configurable and enable it for ic-starter (#1598)
Review: Looks fine + matches description
Notes: Adds allowed_viewers_feature to the execution environment config so that it can be updated easily rather than using the hard coded ALLOWED_VIEWERS_ENABLED value.

66389f30e Interface,Networking: refine the exposed public interface of the adapters and start them in consistent way (#1622)
Review: Looks fine + matches description
Notes: Makes the HTTP adapter expose a start_server function whose signature matches the start_server function which was just added to the BTC adapter + also performs some minor refactoring.

905909681 Interface,Networking: hide some structs and functions from the BTC adapter (#1617)
Review: Looks fine + matches description
Notes: Just modifies the visibility of a few BTC adapter types from public to private.

Refactoring:

09e7929d5 Interface(nervous-system): Remove ic-sns-governance and ic-sns-init’s test_feature configurations (#1606)
Review: Looks fine + matches description
Notes: Determines if in test mode within the SNS governance canister package, rather than the SNS governance library, meaning the --test_feature flag can be removed from the SNS governance library and can also be removed from the packages that depend on the library.

I have also successfully run the build verification scripts for d101161c54df27f514902d22958c77aae9ce8035 and c43a4880199c00135c8415957851e823b3fb769e so have voted to adopt their proposals too.

[cyberowl]

proposal - 133142

Vote: ADOPT

Hash: MATCH

URLS: MATCH

Feedback: NONE

Features:

[dbce2fa6d]
Malicious peers could delay notarization of honest blocks and push their own blocks to be finalized first; the solution is to give extra time for rank-0 (honest) blocks to be notarized before allowing too many non-rank-0 blocks to proceed.

[a8464ac47]
Removes the tracking of long-running canisters and instead using each canister’s state to directly determine if operations like pausing or aborting execution are allowed. Additionally, new safety checks are added to prevent certain operations, such as altering task queues, for aborted canisters while allowing safer actions like querying their status.

[9b242019f]
OnLowWasmMemoryHookStatus status across various components, including the canister_state, sandbox_safe_system_state, which tracks the execution of hooks when Wasm memory is low.

ExecutionMemoryType enum allows the system to distinguish between WasmMemory and StableMemory during memory operations, enabling targeted tracking and management of these memory types during execution, particularly in relation to the OnLowWasmMemoryHookStatus.

[b36319f9e]
Refactor HypervisorError::CalledTrap and HypervisorError::Trapped variants to include a backtrace field, capturing backtrace details for improved debugging.

[a96b75d28]
Backtrace functionality in the handling of HypervisorError, allowing for detailed reporting of backtraces in error cases.

HypervisorError::Trapped variant to capture and include a canister’s backtrace when it traps, provided the canister_backtrace feature is enabled and permissions allow.

rustc-demangle crate to assist in demangling Rust function names for improved error logging.

Handling and parsing of the name_section in WebAssembly (Wasm) modules, allowing for the storage and encoding of function, type, memory, global, table, local, and label names.

[bfd6fa8fb]
validate_initial_wasm_memory_size validates that the initial size of 64-bit Wasm memory declared in a module does not exceed the maximum allowed size, returning an error if the declared size is too large.

[da884ed19]
Priority queue in CallContextManager to track and expire best-effort callbacks based on deadlines, alongside new functions for handling expired callbacks and ensuring their persistence.

[2259be58d]
Enable canister_snapshots

[7f27f9e34]
ic-btc-adapter-client and ic-btc-consensus, have been added, and functionality to interact with a Bitcoin testnet has been implemented.

PocketIC to integrate with Bitcoin.
Simulate interactions with Bitcoin by adding the ability to configure a Bitcoin subnet and connect to a bitcoind process. This includes handling Bitcoin-related tasks like deploying the Bitcoin canister, configuring it to use the regtest network, and supporting integration with the IC Bitcoin API.

[a9e76c402]
Replace parallel_map with maybe_parallel_map, which allows operations to be executed either in parallel using a thread pool or sequentially if no thread pool is provided.

[220baf8e1]
Update call_sync_v3 to check if an ingress message is already in the certified state, allowing the handler to return a certificate early without resubmitting the message to the ingress pool. This optimization is achieved by adding the tree_and_certificate_for_message and parsed_message_status functions for state validation and message status parsing, preventing duplicate submissions.

[b2ce10e4a]
Metrics tracking for ingress message handling, including counts and timing related to ingress messages fetched from peers and the ingress pool.
More detailed metrics for download durations and ingress message sourcing.

[6ab95d4e5]
Update of the .bazelversion from version 6.5.0 to 7.0.1

[cc5e5060d]
Upgrade ubuntu

[26a62d038]
Enable unprivileged users to use ping, and updates to Docker base images and configurations.

Fix a bug with ipmitool in MAC address generation by handling command output errors more gracefully.

Bugfixes:

[ff4d43607]
Off-by-one error and inaccuracies in calculating upper bounds for finalizations due to confusion over inclusive ranges

Updated the worst-case scenario, adjusting the parameter l, and refining the artifact count calculations

Tightening the bounds on pool sizes and adjusting how block_proposals, notarizations

[1914efda5]
equivocation_proof in get_all_validated

[b383408d6]
Fix metric executed_canisters_per_round. Captures executed canister ids and renames var from executed to active_canisters

[ec89de506]
flush_deallocation_channel to fix tests

[ee5a50001]
Cleaned up how server is started in main.
start_server includes conditional logic to start a metrics server based on systemd management
Passes runtime handles directly

[8d630c57d]
Fix typo.

Performance improvements:

[146430974]
Moves and reallocates the canisters, defragmenting the memory layout of the canister list.

The previous commit aimed to defragment the memory layout of canisters but incorrectly targeted a non-fragmented state; the current change corrects this by applying the defragmentation to the actual fragmented state, thus achieving the desired performance improvements.

Chores:

[9e9f3653d]
Equivocation Proof log

[974ec76f5]
Moving functions around to remove libs

[a368e8f26]
Enable flag allowed_viewers_feature

[66389f30e]
Starting the server has been moved to main and func to start has been moved to lib.
start_metrics_grpc method and its related configurations have been removed from lib.rs and reintegrated in a different manner in main.rs

[905909681]
BTC adapter has package-private visibility (no pub keyword), and the ChannelError enum has no variants.

Refactoring:

[09e7929d5]
Remove the --test_feature targets across various components
Update the build process by centralizing test configurations to specific canisters

proposal - 133143

Vote: ADOPT

Hash: MATCH

URLS: MATCH

Feedback: NONE

[d101161c5]
Addition of the systemd-machine-id-setup --commit command to the relabel-machine-id scripts

Adjustments in various SELinux policies, such as permissions for filebeat and handling cryptographic directories.

Corrections to user account settings and service configurations within the guestos environment

proposal - 133144

Vote: ADOPT

Hash: MATCH

URLS: MATCH

Feedback: NONE

[c43a48801]
Disable canister_snapshots

[cyberowl]

I believe that checkpoint commit relates to the following:
fix: Defragment correct state (#1683) · dfinity/ic@1464309 · GitHub.
However I believe that commit you shared conflicts with commit 146430974.

[Lorimer]

Thanks @cyberowl, that’s useful context. I haven’t gone through all of the commits yet.

One line from the original commit was reverted, and another two lines adjusted. That doesn’t mean that the original commit didn’t still make changes to the GuestOS that are indeed present in this release (yet not present in the proposal summary).

I’m still not clear on the rationale for omitting that commit from the proposal summary change log. It seems to be based on a judgement call about the significance of the changes that were made, but that surely should not come down to the person constructing the proposal summary (as a matter of principle). You could use the same argument to justify hiding any number of changes.

[Lorimer]

TLDR: I’m voting to reject all three proposals. Not because I’m concerned about the actual content of the changes (I anticipate being alone in this, and do not intend on blocking these proposals). Instead I’m rejecting on prinicpal, for the sake of raising awareness and hopefully further discussion.

I think it’s critical that the GuestOS change log in the proposal summary does not intentionally or accidentally hide commits that do indeed alter GuestOS behaviour. I think proposals that do this should be seen as taboo, as a matter of principle (regardless of the scope of the changes in question). Allowing this to be normalised means building up dangerous precedent that can be used by bad actors in the future who may succeed in obfuscating malicious changes. I strongly believe that IC OS proposal summary change logs should be validated as a comprehensive list of everthing that modifies GuestOS behaviour. It’s my contention that proposals that do not meet this requirement should be rejected.

As far as I can see, this is the case for release 35153c7cb7b9d1da60472ca7e94c693e418f87bd. Given that the other two releases are based on that release, I see this as applying to them too.

---

133142

Build successful and hashes generated on my machine match (CDN and local build), and the GuestOS hash matches the proposal payload.

There are 87 commits since the previous release, 28 of which are referenced in this proposal. There are 45 files that have been modified both by commits referenced in this proposal as well as commits that weren’t. Browsing through these revealed one commit that was omitted from the proposal summary, but should have been included. In addition, the PocketIC commit is not a GuestOS change and ideally wouldn’t have been included in this proposal summary. Raised questions about these here.

All commits appear to match their commit messages well and seem reasonable. If you're interested in my comments to this effect for every commit, then please expand.

Features:

• dbce2fa6d Consensus,Interface(consensus): Increase block maker delay when there have been too many non-rank-0 blocks notarized (#1531)
  • This looks like a great change, particularly in light of the recent subnet config updates to significantly reduce notarisation delay. It’s similar to the sort of change I had in mind here →

@Lorimer

I am curious as to why this parameter is a fixed value rather than adaptive (adaptive to network conditions that ebb and flow - incrementing when network conditions are bad, and decrementing when better performance can be achieved)

• a8464ac47 Execution,Interface: Execute subnet messages on aborted canisters (#1378)

  • The function can_execute_msg is refactored to can_execute_subnet_msg with additional logic to handle different states of canisters (paused, aborted). Ensures that only safe subnet messages are executed on aborted canisters.

• 9b242019f Execution,Interface: Propagate hook execution status to SystemState (#667)

  • Introduces OnLowWasmMemoryHookStatus to track the status of the hook. Adds logic to update and check the hook status based on memory conditions. Updates protobuf definitions to include the new status. Adds tests to verify the new functionality.

• b36319f9e Execution,Interface: Capture backtrace in syscalls (#1505)

  • Modifies error handling to include backtraces in HypervisorError. Updates tests to check for backtrace inclusion.

• a96b75d28 Execution,Interface: Canister Backtrace on Trap (#1449)

  • Adds backtrace capture to HypervisorError::Trapped. Updates tests to verify backtrace capture.

• bfd6fa8fb Execution,Interface: Validate initial wasm memory size for Wasm64 (#1534)

  • Adds validation logic to check initial wasm memory size. Updates tests to verify the new validation.

• da884ed19 Execution,Interface,Message Routing: Callback expiration priority queue (#1532)

  • Again, the commit message accurately describes the changes made in the code. It mentions the implementation of a priority queue for callback expiration times, a function to return newly expired callbacks, and persistence for this queue.

• 2259be58d Execution,Interface,Networking: Enable canister snapshots (#919)

  • Does what it says on the tin. Proposal 133144 provides a rollback for this commit, just in case.

• 7f27f9e34 Interface(PocketIC): bitcoin integration (#1491)

  • Discussed here

• a9e76c402 Interface,Message Routing: maybe_parallel_map util function (#1376)

  • Introduces a new utility function maybe_parallel_map to handle both parallel and sequential mapping based on the presence of a thread pool. The changes seem to be a straightforward refactor to reduce code duplication.

• 220baf8e1 Interface,Networking(call-v3): Return a certificate for duplicate requests that are already executed (#1523)

  • Added logic to check if a message is already in the certified state before submitting it to the ingress pool

• b2ce10e4a Interface,Networking: Introduce metrics to the block stripper/assembler (#1488)

  • Introduces various metrics to track the performance and behavior of the block stripper/assembler. Metrics include counts of ingress messages, download durations, and block assembly durations.

• 6ab95d4e5 Owners(IDX): Bump bazel to 7.0.1 (#1578)

  • The commit message accurately describes the changes made, including the upgrade to Bazel 7.0.1 and the necessary adjustments to rules_haskell and default options.

• cc5e5060d Node: Upgrade HostOS base image to 24.04 (#1587)

• 26a62d038 Node: Update SetupOS to 24.04 (#1537)

  • Mentioned here

Bugfixes:

• ff4d43607 Consensus,Interface(consensus): Correct several artifact bounds and update docs (#1074)
  • Corrects the bounds for finalizations and update the related documentation
• 1914efda5 Consensus,Interface(consensus): Include equivocation proofs in artifacts returned by get_all_validated (#1579)
  • Ensures that equivocation proofs are included in the artifacts returned by get_all_validated, as described in the commit message.
• b383408d6 Execution,Interface: fix a metric for counting actually executed canisters per round (#1596)
  • Refines the logic for counting executed canisters. Switches the metric from a gauge to a histogram.
• ec89de506 Interface,Message Routing: Fix a race condition in StateManager tests (#1673)
  • Adds a flush_deallocation_channel method to ensure that deallocation requests are processed before assertions in tests
• ee5a50001 Interface,Networking: start the BTC adapter only by accepting a config (#1584)
  • The changes include refactoring the initialisation process and ensuring the metrics server starts correctly when managed by systemd.
• 8d630c57d Interface,Node: Fix long-standing typo (#1602)
  • A simple correction of a comment typo from “an partition” to “a partition”. I hope this bug fix was thoroughly tested

Performance improvements:

• 146430974 Interface,Message Routing: Defragment correct state (#1683 )
  • The previous commit (a438bb7) aimed to defragment the memory layout but did not target the correct state. This fix ensures that the intended performance improvements are achieved by defragmenting the correct state. I’m not sure why a438bb7 wasn’t actually referenced in this change log though (it should have been).

Chores:

• 9e9f3653d Consensus,Interface(consensus): Add info log when producing equivocation (#1623)
  • Adds a log entry when an equivocation is detected
• 974ec76f5 Consensus,Interface(consensus): move get_block_maker_delay function from consensus_utils crate to consensus crate (#1527)
  • Moves the get_block_maker_delay function to the consensus crate and remove unnecessary dependencies
• a368e8f26 Execution,Interface: make allowed viewers feature flag configurable and enable it for ic-starter (#1598)
  • Adds a feature flag for allowed viewers in the execution environment config. Enables the feature flag in ic-starter
• 66389f30e Interface,Networking: refine the exposed public interface of the adapters and start them in consistent way (#1622)
  • The changes include modifications to the build files and the main initialisation logic
• 905909681 Interface,Networking: hide some structs and functions from the BTC adapter (#1617)
  • Hides certain structs and functions from the BTC adapter, making them private

Refactoring:

• 09e7929d5 Interface(nervous-system): Remove ic-sns-governance and ic-sns-init’s test_feature configurations (#1606)
  • The commit message is very informative and aligns with the change. 19 files refactored, primarily Bazel build files (BUILD.bazel) and some Rust source files

I’ve also validated the unelection component of this proposal below.

There currently appear to be 13 blessed replica versions registered, 6 of which would be unelected by this proposal. These unelected versions are not running on any subnets, nor any unassigned nodes, so appears safe to unelect. Expand for details.

I’ve listed blessed replica versions below, ordered by elected date, and crossed out the versions that would be unelected.

• 36c1976, elected 2024-09-09 (proposal 132375), UNELECTION PROPOSED, running on 0 subnets
• 843e71b, elected 2024-09-09 (proposal 132376), UNELECTION PROPOSED, running on 0 subnets
• 3318f74, elected 2024-09-11 (proposal 132412), UNELECTION PROPOSED, running on 0 subnets
• 290fd2a, elected 2024-09-11 (proposal 132413), UNELECTION PROPOSED, running on 0 subnets
• 5d1beac, elected 2024-09-15 (proposal 132500), UNELECTION PROPOSED, running on 0 subnets
• afe1a18, elected 2024-09-16 (proposal 132481), running on 0 subnets
• 1799735, elected 2024-09-16 (proposal 132482), running on 0 subnets
• c180069, elected 2024-09-16 (proposal 132507), UNELECTION PROPOSED, running on 0 subnets
• c664899, elected 2024-09-18 (proposal 132547), running on 1 subnets and all unassigned nodes (since proposal 133093)
• cacf86a, elected 2024-09-18 (proposal 132548), running on 0 subnets
• 0441f40, elected 2024-09-23 (proposal 133061), running on 0 subnets
• 7f6a81f, elected 2024-09-23 (proposal 133062), running on 27 subnets
• c87abf7, elected 2024-09-23 (proposal 133063), running on 9 subnets

---

133143

Build successful and hashes generated on my machine match (CDN and local build), and the GuestOS hash matches the proposal payload.

@DRE-Team

d101161c5 Node: Upgrade GuestOS to 24.04

This proposal is largely the same as 133142 (above), except that this proposal also upgrades GuestOS to Ubuntu 24.04. This is a significant upgrade that affects multiple components, including shell scripts, systemd services, SELinux policies, Dockerfile configurations, and minor adjustments in Rust code. The implication would be that this commit ensures compatibility and/or improved functionality with the new OS version. But there are numerous aspects about this commit that I’m unclear about, such as the removal of cleanup commands that were in place to avoid indeterministic builds.

If I knew Eero Kelly’s handle on this forum, I’d ask if he could provide a bit more context and reasoning for each of the changes in that commit. @DRE-Team is this something you could request?

---

133144

Build successful and hashes generated on my machine match (CDN and local build), and the GuestOS hash matches the proposal payload.

@DRE-Team

c43a48801 Execution,Interface,Networking: Revert “feat: Enable canister snapshots”

This proposal is largely the same as 133142 (above), except that this proposal also disables canister_snapshots (a compile-time flag requiring a separate build).

[ipsita]

Proposal 133142

All the hashes matches. voting to adopt.

Features

[dbce2fa6d]
Increase the block maker delay for high rank block makers in case malicious nodes manage to delay notarization for rank 0 blocks. This increase the change that rank 0 blocks are notarized.

[a8464ac47]
Aborted canisters have already taken messages from input queue and stored them in task queue. Subnet messages that would pause canister execution or alter task queue are not allowed.

[9b242019f]
Check hook condition and propagate hook status to SystemState. Hook condition is checked whenever addition execution memory is requested. Hook status can be condition not executed, ready for execution, executed.

[b36319f9e]
When a syscall results in a trap we backtrace and return the reject message.

[a96b75d28]
When a canister traps a new options field is introduced in Hypervisor::Trapped error variant to include back trace.
This field will be None if:
A. canister_backtrace feature is not implemented.
B. Canister does not include a name.
C. Calling canister does not have permission to view the trace.

[bfd6fa8fb]
Checks if the initial size of wasm64 memory is not greater than allowed.

[da884ed19]
Implements a priority queue of best-effort to return newly expired callbacks in CallContextManager function

[2259be58d]
canister_snapshots feature flag is enabled.

[7f27f9e34]
Bitcoin integration in PocketIC.

[a9e76c402]
Created a function maybe_parallel_map to remove duplicate code.

[220baf8e1]
Checks if the message is already known. Since if the message is already known it returns the certificate without re-submitting the message to ingress pool.

[b2ce10e4a]
Added different metrics to the new block stripper and assembler.

[6ab95d4e5]
updates bazelversion from 6.5.0 to 7.0.1. Also updates image urls

[cc5e5060d]
update ubuntu host os from 20.04 to 24.04

[26a62d038]
Update SetupOS to 24.04

Bugfixes

[ff4d43607]
updates worst case scenario by adjusting parameter l and refining artifact count.

[1914efda5]
Include equivocation proofs in artifacts returned by get_all_validated

[b383408d6]
Improves the tracking to executed canisters by refining the counting logic and switching the associated metric to histogram.

[ec89de506]
Added code state_manager.flush_deallocation_channel(); to flush deallocation channel since remove_states_below sends request to deallocation thread.

[ee5a50001]
Enables addition of BTC adapter to pocket ic

[8d630c57d]
Fixed type an to a

Performance improvements:

[146430974]
Makes sure we actually defragment the correct state, specially the once which have been fragmented in previous rounds.

Chores:

[9e9f3653d]
Added info log when producing equivocation, this would be helpful when getting paged.

[974ec76f5]
Moves get_block_maker_delay function from consensus_utils crate to consensus crate. Also removes unneeded dependencies.

[a368e8f26]
Added feature flag allowed_viewers_feature configurable in execution config. The feature flag is enabled in implementation and testing but disabled in prod.

[66389f30e]
Refines the public exposed interface of adapters and start them in consistent way

[905909681]
Hide some structs and functions from the BTC adapter

Refactoring:

[09e7929d5]
Removed the --test_feature target across various components and updated the build process by centralizing the test configs to specific canisters.

Proposal 133143

All the hashes matches. voting to adopt.

[d101161c5]
Upgrades GuestOS to 24.04

Proposal 133144

All the hashes matches. voting to adopt.

[c43a48801]
canister_snapshots feature flag is disabled.

[yuvika]

Proposal 133142

Voting to adopt.

local, CDN and the hash in the payload matches.

Features

• [dbce2fa6d]: Increased block maker delay for higher-ranked nodes to prevent potential delays caused by malicious nodes trying to disrupt rank 0 block notarization.
• [a8464ac47]: Improved handling of aborted canisters by disallowing subnet messages that could pause canister execution or modify the task queue.
• [9b242019f]: Added a mechanism to check hook conditions during memory requests and update the SystemState based on whether the condition is pending, ready, or executed.
• [b36319f9e]: Enhanced trap handling for syscalls by including a backtrace and sending a corresponding reject message.
• [a96b75d28]: Introduced a new backtrace field in the Hypervisor::Trapped error variant for better error tracing, conditional on feature implementation, canister naming, and permission.
• [bfd6fa8fb]: Validated that the initial size of wasm64 memory doesn’t exceed the permissible limits.
• [da884ed19]: Implemented a priority queue to handle expired callbacks efficiently in the CallContextManager.
• [2259be58d]: Enabled the canister_snapshots feature flag.
• [7f27f9e34]: Added Bitcoin integration support within PocketIC.
• [a9e76c402]: Simplified code by creating a maybe_parallel_map function to eliminate redundancy.
• [220baf8e1]: Prevented duplicate message submissions by returning the certificate if the message is already recognized.
• [b2ce10e4a]: Enhanced the block stripper and assembler with additional metrics.
• [6ab95d4e5]: Updated Bazel version to 7.0.1 and refreshed image URLs.
• [cc5e5060d], [26a62d038]: Upgraded host OS from Ubuntu 20.04 to 24.04 for improved support.

Bugfixes

• [ff4d43607]: Adjusted parameters to refine worst-case scenarios and artifact counts.
• [1914efda5]: Added equivocation proofs in artifacts returned by get_all_validated.
• [b383408d6]: Enhanced tracking of executed canisters by updating metrics and using histograms.
• [ec89de506]: Fixed deallocation issues by flushing channels in state_manager.
• [ee5a50001]: Enabled addition of the BTC adapter in PocketIC.
• [8d630c57d]: Corrected minor typo from an to a.

Performance Improvements

• [146430974]: Ensured correct state defragmentation, particularly for states fragmented in earlier iterations.

Chores

• [9e9f3653d]: Added logs for equivocation events to assist with monitoring.
• [974ec76f5]: Moved get_block_maker_delay to the consensus crate and cleaned up dependencies.
• [a368e8f26]: Introduced a configurable allowed_viewers_feature flag, enabled for testing but disabled in production.
• [66389f30e]: Standardized the interface and initialization of adapters.
• [905909681]: Restricted visibility of certain structs and functions in the BTC adapter.

Refactoring

• [09e7929d5]: Centralized test configurations across components and removed outdated test targets.

Proposal 133143

Voting to adopt.

local, CDN and the hash in the payload matches.

• [d101161c5]: Updated GuestOS to Ubuntu 24.04.

Proposal 133144

Voting to adopt.

local, CDN and the hash in the payload matches.

• [c43a48801]: Disabled the canister_snapshots feature flag.

[zenithcode]

Proposal: 133142

Summary of Review:

1. Hash Match: Verified that the CDN hash, locally built hash and the hash in payload of the proposal matches and is be4060bb40804834cce0e5a8ccdb08a63289ccc10f57c55e9e11938fc82d79e1
2. Voting YES on the proposal

Features:
Dbce2fa6d

I verified the change in consensus where block maker delay is increased when there are too many non-rank-0 blocks. The addition of function count_non_rank_0_blocks and modifications in get_block_maker_delay address these changes.

A8464ac47

Dts_aborted_execution_does_not_block_subnet_messages function and couple of other changes in can_execute_subnet_msg addresses the change Execute subnet messages on aborted canisters.

9b242019f

I verified that this commit and release notes match the code changes and the logic
In the case with memory_allocation
min(memory_allocation - used_stable_memory, wasm_memory_limit) - used_wasm_memory
And Without memory allocation
wasm_memory_limit - used_wasm_memory

Was addressed in function update_on_low_wasm_memory_hook_status around line 1269

B36319f9e

This commit addresses capture of backtrace in syscalls. Functions add_backtrace has addressed it.

A96b75d28

Certain modifications here done to function trap_code_to_hypervisor_error and other changes match the release note Canister Backtrace on Trap.

bfd6fa8fb

The commit has necessary changes for additional validation for initial size for WASM64 memories.
const WASM_PAGE_SIZE: u32 = wasmtime_environ::Memory::DEFAULT_PAGE_SIZE;
And function validate_initial_wasm_memory_size addresses this.

Da884ed19

Various modifications in the code bases such as checking unexpired_callbacks is subset of all best-effort callbacks, function calculate_callback_deadlines() implements a priority queue of best-effort callback expiration times in CallContextManager.

2259be58d

This PR enables canister_snapshots: FlagStatus::Enabled,

But this is later also disabled in proposal https://dashboard.internetcomputer.org/proposal/133144
May be it will follow a bit more modifications.

7f27f9e34

Verified the bitcoin integration in pocket IC packages. Various places the code has been added to support this integration. Around 20 files have been changed maily addition to pocket-ic folder in packages.

A9e76c402

This commit addresses the addition of maybe_parallel_map() and removal of parallel_map(). This function is added and used at various places. This matches Some(thread_pool) => parallel_map(thread_pool, items, f), based on this condition None => items.map(|x| f(&x)).collect::<Vec>().

220baf8e1

Currently the replica node time out if a duplicate messageis submited. This commit addresses this problem by returning a certificate for duplicate requests that are already executed.

B2ce10e4a

This PR is adding a bit more metrics like IngressMessageSource and IngressSenderMetrics to the block stripper/assembler.

6ab95d4e5
Verified that the code has changes to bump bazel from 6.5.0 to 7.0.1

Cc5e5060d
Verified that the docker file has the necessary changes to upgrade the host OS from 20.04 to 24.04

26a62d038

Not sure about this change, was not able to verify how the code changes in feat: [NODE-1443] Update SetupOS to 24.04 by Bownairo · Pull Request #1537 · dfinity/ic · GitHub are related to Setup OS upgrade to 24.04 ?
The Diff shows some other changes.

May be there is a glitch in the PR.
But verified that Setup OS is also upgraded to 24.04 as part of feat: [NODE-1443] Update SetupOS to 24.04 by Bownairo · Pull Request #1537 · dfinity/ic · GitHub which is merged to master.

Bugfixes:

Verified the bug fixes addressed in commits
Ff4d43607, 1914efda5, b383408d6, ec89de506, ee5a50001, 8d630c57d
The release notes matches the code changes done.

Performance improvements:

146430974

Verified there are changes around defragmenting correct state. The code was changed to consider all state rather than just the checkpointed state.

Chores:

9e9f3653d

Verified an level “warn” logging has been added for EquivocationProof

974ec76f5

Refactoring of get_block_maker_delay verified from concensus_util to concensus.

A368e8f26

Verified that allowed_viewers_feature is now configurable.

66389f30e

Several adapters including metrics_server and btc_adapter are now started in a consistent way.

905909681

Certain struct and functions are made private from public.
Command, ProcessBitcoinNetworkMessageError, ChannelError, Channel are among them.

Refactoring:

09e7929d5

Verified some refactoring was done around ic-sns-governance and ic-sns-init’s test_feature configurations

Proposal - 133143

Summary of Review:

1. Hash Match: Verified that the CDN hash, locally built hash and the hash in payload of the proposal matches and is ed77eae148f076571219450bbcfba16e3dcd86aa51595d3e48f02e87294de39d
2. Voting YES on the proposal.

D101161c5

The changes Look good in various networking services.

However, these are supporting changes to the previously changed base image to 24.04.

Proposal - 133144

Summary of Review:

1. Hash Match: Verified that the CDN hash, locally built hash and the hash in payload of the proposal matches and is aa69a874d5405a76dcf9cb7401d603bd902465a9408d0c80707ce964b53a0668
2. Voting YES on the proposal.

c43a48801 Execution,Interface,Networking: Revert “feat: Enable canister snapshots”

Verified that The Enabled canister snapshot is disabled now which was enabled in proposal 133142.

May be it will be enabled later after it bit more modifications and improvements.

Code changes verify release notes.

[ilbert]

Proposal 133142

Hashes match.
ADOPTED.
Review:

I’ve reviewed all the commits listed in the proposal, for the Execution and Runtime layers:

a8464ac47:
Updates and renames the can_execute_msg function to use the canister_states directly. The function now checks if the canister is aborted by looking at the task_queue of the canister’s system_state.
If the message does not have an effective canister id, if the canister does not have any effective state or if the message does not have any invoked method, the function returns true.
If the canister is paused, the function returns false, while if the canister is aborted, the function returns false when the invoked management canister’s method is DeleteCanister, StopCanister or LoadCanisterSnapshot.
If the invoked management canister’s method is InstallCode or InstallChunkedCode, the function returns true only if the canister is not aborted and doesn’t have any ongoing long install code execution.
For all the other invoked management canister’s methods, the function returns true.
As a consequence of the updates above, removes the long_running_canister_ids argument from the SchedulerImpl struct’s drain_subnet_queues method.
Updates the inner_round method of the SchedulerImpl to calculate the drain_subnet_queues’s ongoing_long_install_code input without mapping the canister_states to the old long_running_canister_ids.
Added a comment on GitHub.

9b242019f:
Adds the on_low_wasm_memory_hook_status to the SystemState struct, to track the status of the execution of the low_on_wasm_memory hook. Adds the set and get methods to the SystemState to set and get the hook status. By default, the hook state is set to ConditionNotSatisfied variant in the SystemState.
Adds the stable_memory_usage and wasm_memory_usage to the MemoryUsage struct, together with the add_execution_memory method. This method is invoked in the allocate_execution_memory method and is used to track the stable and heap memory used.
The hook status is set to Ready in the update_on_low_wasm_memory_hook_status of the SandboxSafeSystemState, following the logic from the IC interface specification.
This is part of the work to enable the canister_on_low_wasm_memory hook.

b36319f9e:
Adds the canister_backtrace flag to the StoreData. Its value is set to the value of the corresponding WasmtimeEmbedder’s config feature flag.
Adds the backtrace field to the HypervisorError::CalledTrap enum variant. This enables to track the backtrace also when calling ic0.trap, if it is enabled.

a96b75d28:
Adds the backtrace field to the HypervisorError::Trapped enum variant.
Adds the demangle and convert_backtrace helper functions to print the backtrace in the canister logs.

bfd6fa8fb:
Adds the validate_initial_wasm_memory_size function, which is invoked in the validate_wasm_binary and checks if the declared size is lower than the allowed size, and returns an error otherwise.

da884ed19:
Adds the unexpired_callbacks field to the CallContextManager struct, which keeps track of all the non-expired best-effort callbacks. This field is still not used in the codebase, except for adding and removing callback ids from it inside the register_callback and unregister_callback methods.
Adds the expire_callbacks method to the CallContextManager struct, which removes and returns the expired callbacks from the unexpired_callbacks set. This method is still not used in the codebase.
This commit is needed to add the field to the serialized CallContextManager.

2259be58d:
Enables canister snapshots via the dedicated feature flag.

b383408d6:
Refactors the executed_canisters_per_round metric to become an histogram to avoid data loss between scrapes and to only count canisters that have used instructions.
Added a on GitHub.

a368e8f26:
Adds the allowed_viewers_feature field to the execution environment’s Config struct and uses it instead of the removed ALLOWED_VIEWERS_ENABLED constant. The flag is set to disabled by default.

All reviewed commits match their description.

---

Proposal 133143

Hashes match.
ADOPTED.
Review:
This release is the same as the one in 133142 with the addition of the commit d101161c5, which matches its description.

---

Proposal 133144

Hashes match.
ADOPTED.
Review:
This release is the same as the one in 133142 with the addition of the commit c43a48801, which matches its description.

[ZackDS]

Proposal 133142

I built the replica without errors and verified hashes and listed commits match, voted to adopt it.

Bugfixes:
ff4d43607 Consensus,Interface(consensus):
Fix for an off-by-one error in the upper bound for finalizations and updates docs. Mainly the maximum height span to number of rounds for deriving bounds in a worst case scenario.

1914efda5 Consensus,Interface(consensus):
Adds equivocation proofs in artifacts returned by the get_all_validated function in the consensus pool.

b383408d6 Execution,Interface:
Improved ccounting of the actually executed canisters per round, by switching executed_canisters_per_round from a gauge to a histogram.

ec89de506 Interface,Message Routing:
Adds flush_deallocation_channel function used to send a deallocation request to the deallocation_sender channel and then wait for a notification indicating that the deallocation has been processed, to StatemMnager tests in order to wait before any checks are performed.

ee5a50001 Interface,Networking:
Refactor of start_server to include waiting for a config for metrics server only be started if it is managed by systemd.

8d630c57d Interface,Node:
Simple typo it was an instead of a.

Performance improvements:
146430974 Interface,Message Routing:
Fixes the code that effectively takes ownership of the canister_states field to provide cleaner memory layout for faster defragmentation.

Chores:
9e9f3653d Consensus,Interface(consensus):
Within the check if the hash of the proposal.content is different from the hash of the existing_metadata.content, if there’s a mismatch, it creates an EquivocationProof and now with the current change it logs a warning message.

974ec76f5 Consensus,Interface(consensus):
The get_block_maker_delay and the is_time_to_make_block function is moved from consensus_utils crate to consensus crate.
This get_block_maker_delay function calculates the delay for a block maker based on the rank and notarization delay settings.
While is_time_to_make_block function determines whether it’s time for a replica to create a new block based on the current time, block maker delay, and round information.

a368e8f26 Execution,Interface:
The allowed_viewers feature flag is made configurable and enabaled by default for canister log visibility to be used by DFX.

66389f30e Interface,Networking: refine the exposed public interface of the adapters and start them in consistent way (#1622)
The start_server function responsible for starting the HttpsOutcallsService server, which handles HTTP requests for canister interactions and the spawn_grpc_server function that creates a gRPC server for handling HTTP outcalls were refactored and started in more consistent way.

905909681 Interface,Networking:
Sets the visibility of some BTC adapter structs and types to private.

Refactoring:
09e7929d5 Interface(nervous-system):
Removes SNS Governance and SNS Init’s --test-feature configs and also from the listed crates, while the conditional check that enables test features based on the test feature flag is now in the governance canister, this determines if it is in test mode .

Proposal 133143

Hash matches, voted to adopt.

Proposal 133144

Hash matches, adopted.

[stefan.schneider]

@Lorimer

One line from the original commit was reverted, and another two lines adjusted. That doesn’t mean that the original commit didn’t still make changes to the GuestOS that are indeed present in this release (yet not present in the proposal summary).

The TLDR of those two changes is that the first (the omitted) one had a bug that I quickly fixed with a follow-up commit, ending up in the same weekly release. They are both relevant, and it shouldn’t have been omitted. Sorry for the confusion.

[Zane]

Proposal 133142

Hashes and code match. Voted to adopt.

[a8464ac47] Modified logic to determine whether a subnet message can be executed. Previously a set of canister ids with long running messages was computed, can_execute_msg would return true if the target canister wasn’t executing a long running message already or in case of InstallCode requests if there wasn’t already one being run. drain_subnet_queues has been modified and no longer processes the set of long running canisters, while can_execute_msg has been renamed to can_execute_subnet_msg to better reflect what it does, now if the canister has no effective state or ids or if the message is of Response type it will return true unconditionally, otherwise it checks the task at the front of the canister’s task queue to determine whether the canister has been aborted or is currently paused. Paused canister can’t execute messages, InstallCode and InstallChunkedCode requests are only allowed if the canister is not aborted and there is no ongoing long install code execution and DeleteCanister, StopCanister and LoadCanisterSnapshot requests are only allowed if the canister is aborted, other message types return true.

[9b242019f] Added new field on_low_wasm_memory_hook_status to SystemState and CanisterStateBits structs. MemoryUsage has been extended with stable_memory_usage and wasm_memory_usage fields, these are updated by allocate_execution_memory and then used to determine whether the hook status should be changed.

[b36319f9e] Added canister_backtrace flag to StoreData, which is initialized with the same value as the one set in the EmbeddersConfig. Added support for backtraces to CalledTrap variant of HypervisorError enum.

Adds the backtrace field to the HypervisorError::Trapped enum variant.
Adds the demangle and convert_backtrace helper functions to print the backtrace in the canister logs.

[a96b75d28] Extend Trapped variant of HypervisorError enum with optional backtrace field, this will contain the canister backtrace if they are enabled and the caller has permission to view them.

[bfd6fa8fb] Added validate_initial_wasm_memory_size function to validate initial wasm memory size when running in wasm64, it verifies the declared size doesn’t exceed the maximum allowed size and returns an error if it does.

[da884ed19] Added unexpired_callbacks field to CallContextManager struct, this is a set of callbacks to best effort messages ordered by deadline. A new method has also been added called expire_callbacks, though still not used it allows to remove all expired callbacks from the set.

[2259be58d] Enable canister snapshots by default.

[a9e76c402] Reduced code duplication by creating maybe_parallel_map function, which uses parallel_map when the passed optional thread pool is not None, otherwise it uses a sequential map.

[b383408d6] Chanaged executed_canisters_per_round type to Histogram instead of a IntGaugege and modifed execute_canisters_in_inner_round to also return a set of canisters ids that were executed in the round.

[146430974] Defragment replicated state instead of the checkpointed one.

[a368e8f26] Add allowed_viewers_feature feature flag to execution environment config and use it in place of an hardcoded constant.

Proposal 133143

Hashes and code match. Voted to adopt.

Same code changes as the one proposed with 133142 but update Ubuntu version used by GuestOS.

Proposal 133144

Hashes and code match. Voted to adopt.

Same code changes as the one proposed with 133142 but with snapshot disabled for system subnets.

[Luka]

@Lorimer

@DRE-Team

7f27f9e34 Interface(PocketIC): bitcoin integration (#1491)

I don’t think this is a GuestOS change. Presumably it was picked up by bazel due to the modification to rs/consensus/src/consensus/payload_builder.rs

agreed, will add this to scope exclusions

@Lorimer

Similarly I don’t think the HostOS and SetupOS commits are actually actioned by this proposal →

SetupOS we intentionally include because Node Providers normally use same versions when setting up their nodes as we elect in these proposals. See here: Node Deployment Guide - Internet Computer Wiki

And SetupOS depends on HostOS (“//ic-os/setupos/envs/prod:copy_hostos_img” → “//ic-os/hostos/envs/prod:disk-img.tar.zst”)

@Lorimer

Additionally, I came across this commit and would’ve expected this to have been included in the proposal summary. Is it not considered a GuestOS change? → a438bb77e - perf: Defragment memory during checkpointing (#1620)

Having checked I see that this commit was intentionally removed from the proposal summary (’ Excluded by authors’). @Luka are you able to elaborate?

Checking why was this excluded, will get back to you.

[Lorimer]

@Luka

agreed, will add this to scope exclusions

Thanks @Luka

@Luka

And SetupOS depends on HostOS (“//ic-os/setupos/envs/prod:copy_hostos_img” → “//ic-os/hostos/envs/prod:disk-img.tar.zst”)

But HostOS is not modified by this proposal (there’s a separate proposal topic for that). Technically neither is SetupOS (my understanding is SetupOS is not under NNS control).

@Luka

Checking why was this excluded, will get back to you.

Thanks, @stefan.schneider kindly commented about this above.

How does the ‘excluded by authors’ mechanism work (is it based on some sort of commit message convention)? Or is it a manual step when composing the proposal?

[free]

There’s a manual step involved, where we can move commits from one section to another (e.g. from features to chores).

It was I who intentionally (and apparently incorrectly) removed the commit in question. As Stefan said above, this was an inaccurate fix, quickly followed by the right one. And I just wanted to keep the noise out of the release notes, so I only kept the latter.

Lesson learned, will not do this in the future.

[Luka]

@Lorimer

How does the ‘excluded by authors’ mechanism work (is it based on some sort of commit message convention)? Or is it a manual step when composing the proposal?

It’s manually excluded just before publication. We give developers the ability to review the notes and alter whatever is necessary.

@Lorimer

But HostOS is not modified by this proposal (there’s a separate proposal topic for that). Technically neither is SetupOS (my understanding is SetupOS is not under NNS control).

I hear you. But I guess this is just a drawback of current implementations. I think it’s better to let the community know about these changes when it’s perfectly possible that a new node will be deployed and used in a subnet with these changes rather than hiding it.