Proposal to elect new release rc--2024-09-06_01-30

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 132375.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-06_01-30-base (36c1976526fc488d40a89000f81d9f4897d2324e)

This release is based on changes since release-2024-08-29_01-30-base (35bfcadd0f2a474057e42393917b8b3ac269627a).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Features:

  • 8a79f6e96 Consensus,Interface: Stop relaying IDKG complaints & openings (#1283)
  • a9236c98b Execution,Interface: Add more metrics for canister snapshots usage (#1266)
  • 0784ef66c Execution,Interface: Canister Backtrace Flag (#1291)
  • 5ff84f5b8 Execution,Interface: Doc links for CanisterManager errors (#1218)
  • 13ee679f5 Execution,Interface: Add Doc Links to Contract Violation Errors (#1211)
  • 66c25e664 Execution,Interface(system-api): add call_cycles_add128_up_to (#1158)
  • 54fcf35dd Execution,Interface: Add doc links to Validation Errors (#1188)
  • e26cf5157 Execution,Interface,Message Routing: Factor input scheduling into separate struct (#1253)
  • 0ca139ca3 Interface(sns-cli): Add sns list command (#1200)
  • b9dbfea28 Interface,Networking: Implement the ArtifactAssembler for DownloadStrippedArtifactMessage (#771)
  • 3cd7ee343 Interface,Node: Add cpu field to deployment.json (#1199)

Bugfixes:

  • 0b3c42296 Execution,Interface: Revert “feat(system-api): add call_cycles_add128_up_to (#1158)” (#1319)
  • 83c0f0ab7 Execution,Interface(snapshots): Compute the correct subnet available memory after snapshot operations (#1259)
  • bd471209f Execution,Interface(execution): Canister snapshots should include Wasm globals (#1250)
  • 3824835dc Execution,Interface,Message Routing: Fix brown paper bag bug in InputSchedule (#1361)
  • e0b6f8aff Interface(nervous_system): Fix a bug with advancing buf (#1314)
  • 468b3a23e Interface,Networking: caller of canister http composite query transform (#1337)
  • e5654570d Interface,Networking: propagate panics from the Shutdown struct and remove the panic hooks (#1135)
  • 86b5a804f Owners(deps): drop release_max_level_debug from slog features (#1327)
  • ab306b022 Node: Fix missing log function (#1256)

Performance improvements:

Chores:

  • bb66a260f Consensus,Execution,Interface: Update query stats log message to be less confusing in PocketIC (#1322)
  • f3a6cf88d Consensus,Interface: use HTTPS to fetch CUP (#408)
  • 3b688978c Consensus,Interface: Remove allow result_large_err clippy lint (#1226)
  • 17286a12d Consensus,Interface: Avoid cloning NiDKG transcripts (#1219)
  • 622fb34d2 Consensus,Interface,Networking: don’t bound the bouncer API to the Identifiable artifact trait (#1255)
  • 2a14ccf91 Execution,Interface: Refactor canister snapshot charges (#1119)
  • 4b657a11e Execution,Interface,Message Routing: Drop ResponsePlaceholder (#1348)
  • a860eec16 Execution,Interface,Message Routing: Remove the unused session nonce (#1288)
  • 4d09678d2 Interface: sort rust derive traits (#1241)
  • 69784766b Interface,Message Routing: simplify the xnet endpoint (#1247)
  • 6e947d1a4 Interface,Networking: remove uni streams from transport (#1201)
  • 98423a211 Interface,Node: Improve partition_tools (#1209)
  • f72f9e395 Owners: update quinn (#1276)
  • 18e3b9ba3 Node: Update Base Image Refs [2024-09-05-0807] (#1341)
  • 13787acc0 Node(IDX): move artifacts to new folder (#1326)
  • 2fae0ae29 Node: update node deployment path verification warning (#1243)
  • 574e00d70 Node: Extract logging functionality (#1202)
  • 1f3fa2e1b Node: Update Base Image Refs [2024-08-29-0808] (#1212)

Refactoring:

  • cd9216dda Consensus,Interface: Unify and group together the creation of the bouncers and assemblers (#1306)
  • fdb9d22f9 Consensus,Interface: Move the ingress bounce to the ingress manager crate (#1301)
  • ffdc6f1b4 Execution,Interface,Message Routing: Renaming and minor cleanup before new CanisterQueues (#1345)
  • 6f6469518 Execution,Interface,Message Routing: Redo the ValidateEq annotations for CanisterQueues (#1338)
  • 644de3a7b Interface: merge the constants from ic_prep and ic_constants (#1329)
  • cee511467 Interface(nervous_system): Remove most usages of hotkey_principal (#870)
  • 14e933e70 Interface,Networking: add more documentation about design decisions in the quic transport (#1221)

Full list of changes (including the ones that are not relevant to GuestOS) can be found on GitHub.

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/36c1976526fc488d40a89000f81d9f4897d2324e/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 36c1976526fc488d40a89000f81d9f4897d2324e

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

5 Likes

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 132376.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-06_01-30-canister-snapshots (843e71b994e1bf28edf80ad7746a7355259de91a)

This release is based on changes since release-2024-09-06_01-30-base (36c1976526fc488d40a89000f81d9f4897d2324e).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Features:

  • 843e71b99 Execution,Interface,Networking: Enable canister snapshots

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/843e71b994e1bf28edf80ad7746a7355259de91a/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 843e71b994e1bf28edf80ad7746a7355259de91a

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

2 Likes

Thanks for this release DFINITY!

This commit is particularly interesting →

@luc-blaeser how is backwards compatibility maintained (can older snapshots without exported_globals still be loaded without issues)?


Update: No worries, I did some reading about protobuf and repeated fields, which default to empty (in other words they’re optional) :+1:

1 Like

I think it would be much clearer if the reversion were displayed underneath the commit being reverted, i.e.

(or just removed from the change log altogether if it’s an exact reversion)

This would save reviewers time, reviewing something that they later realise has been removed from the release.

2 Likes

Hi @Lorimer,
Thank you for analyzing the code changes in the proposal!
Indeed, the fix should be backwards compatible. If a snapshot happened to work before just because it did not mutate globals, then it will still work. Otherwise, the snapshot was already broken and should not be restored. This PR solves this issue, however, only for new snapshots taken after the fix.
A good way to analyze whether a snaphot was affected is to look at the Wasm binary code and check for the presence of global.set.

3 Likes

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 132412.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-06_01-30-base-hotfix-https-outcalls (3318f746113e7695ae0b904be5a48820884eb6d8)

This release is based on changes since release-2024-09-06_01-30-base (36c1976526fc488d40a89000f81d9f4897d2324e).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

  • 3318f7461 Interface,Networking: fix: revert the old error string when outcall response is bigger than the limit

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/3318f746113e7695ae0b904be5a48820884eb6d8/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 3318f746113e7695ae0b904be5a48820884eb6d8

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

3 Likes

Hello there!

We are happy to announce that voting is now open for a new IC release.
The NNS proposal is here: IC NNS Proposal 132413.

Here is a summary of the changes since the last release:

Release Notes for release-2024-09-06_01-30-canister-snapshots-hotfix-https-outcalls (290fd2ae93a02e1ab579895c806161e42b6acf2b)

This release is based on changes since release-2024-09-06_01-30-canister-snapshots (843e71b994e1bf28edf80ad7746a7355259de91a).

Please note that some commits may be excluded from this release if they’re not relevant, or not modifying the GuestOS image. Additionally, descriptions of some changes might have been slightly modified to fit the release notes format.

To see a full list of commits added since last release, compare the revisions on GitHub.

Other changes:

  • 290fd2ae9 Interface,Networking: fix: revert the old error string when outcall response is bigger than the limit

IC-OS Verification

To build and verify the IC-OS disk image, run:

# From https://github.com/dfinity/ic#verifying-releases
sudo apt-get install -y curl && curl --proto '=https' --tlsv1.2 -sSLO https://raw.githubusercontent.com/dfinity/ic/290fd2ae93a02e1ab579895c806161e42b6acf2b/ci/tools/repro-check.sh && chmod +x repro-check.sh && ./repro-check.sh -c 290fd2ae93a02e1ab579895c806161e42b6acf2b

The two SHA256 sums printed above from a) the downloaded CDN image and b) the locally built image, must be identical, and must match the SHA256 from the payload of the NNS proposal.

2 Likes

There’s very little information provided with this ‘hotfix’,

  • What’s the issue?
  • How does this hotfix fix it?
  • Which commit does this revert? The commit message is ‘revert’ (that’s it :neutral_face:)

fix: revert the old error string when outcall response is bigger than the limit’

  • The commit summary ^ in the release notes indicates that the fix is to change the error string… There’s a dependency on the exact contents of this string somewhere? Really?
  • This commit has numerous other changes such as changing dependency versions

@DRE-Team Please could someone from DFINITY provide some clarity? Thanks.

Reasonable ask. I’ve forwarded the request to the networking team, who have the most domain knowledge and can provide the most relevant information.

2 Likes

In the meantime, the only information I have is:
The HTTP outcalls after this particular commit started returning a different error message (as a string), in case the canister-provided message limit is too small for the actually returned data. Some canisters, including ckETH were relying on the actual string value of the error message to understand that the provided message limit is too small, and then they would bump up the response message size limit and retry. Ugly hack around the API to keep the costs of http outcalls small. The http outcalls price goes up a lot if they immediately provide the max message size limit, so they opt for this iterative approach, even though it’s an ugly hack.
The issue is that this breaks conversions for ckETH, EVM RPC and likely multiple community projects. So the safest thing is to roll back the change.

4 Likes

@sat’s description is correct.

This commit has numerous other changes such as changing dependency versions

This is because the orginal change that was reverted was a version upgrade.

Do you have any other questions?

1 Like

Thanks @sat and @tim1 for the quick responses. Sorry for my delay, I’m just catching up on this over my lunch break.


All makes sense, thanks for elaborating. It would be good if commit messages (particularly for important commits like this) could be more descriptive in the future.

As a side note, this sounds like a bug in various canisters, as opposed to a bug in the recent IC OS releases (but I understand the need for a quick resolution). Are there going to be steps taken to address this type of issue, so that error messages can be safely adjusted in the future without fear of breaking systems that have taken a hacky dependency?

2 Likes

It would be good if commit messages could be more descriptive in the future.

I agree we need to improve this!

This sounds like a bug in various canisters

Yes relying on error strings, that are intended for humans, is very fragile. But the fact that people rely on this points to an issue in the current HTTPS outcalls API, specifically that the current API makes it very hard to pay only for what you use (you need to specify the expected response size in advance with max_response_bytes). As a follow-up, we will revisit the API and see how to improve it so that people don’t need to do hacky workarounds anymore.

5 Likes

We’re preparing a new IC release. The changelog will be announced soon.

2 Likes

We’re preparing a new IC release. The changelog will be announced soon.

2 Likes