Staged rollout of Boundary Node security upgrades that might impact dApps - action may be required

Developers
1

What is happening?

The boundary nodes are being updated with some potentially breaking changes to address security vulnerabilities:

  1. /api calls will only be allowed on the base domains:
    • icp-api.io
    • ic0.app
    • icp0.io
  2. Calls to /_/raw will require certification
  3. requests by crawlers for SEO will require certification

Since some changes are potentially breaking we will do the rollout in stages giving you the opportunity to test your dapps before the changes are rolled out to production.

Change details

/api calls

Boundary nodes receive /api requests to domains like:

  • mainnet.dfinity.network
  • boundary.ic0.app
  • <canister_id>.ic0.app

For more details see the https interface spec.

In order to fix a security vulnerability, the /api/ calls will only be allowed on the base domains:

  • icp-api.io
  • ic0.app
  • icp0.io

What does it mean?

In the first phase, /api requests to a hostname other than the base domains will be redirected to icp-api.io.

For example:

boundary.ic0.app/api/v2/<canister_id>/query will be redirected to icp-api.io/api/v2/<canister_id>/query. In the second phase, /api requests are only allowed on the base domains.

Recent versions of agent-js and agent-rs already rewrite /api/* requests to subdomains of the base domains to the base domain itself. (e.g., <canister_id>.ic0.app → ic0.app) and no changes are required.

Who is affected?

If you are using older libraries or have implemented your own you might have /api requests going to other hostnames, those requests will get redirected and eventually rejected. If you use newer libraries, this will not affect you.

Timeline

Thursday June 22nd - We have reserved 1 boundary node at [2604:6800:258:1:5000:98ff:fe7d:347b], [162.247.129.233] with the updated configuration. You can use dnsmasq or configure your /etc/hosts file (or windows equivalent) to point to it if you want to test your application. Note that you will need to point the following URLs (possibly more if you are using custom domains):

  • icp0.app
  • <canister_id>.ic0.app
  • icp0.io
  • <canister_id>.icp0.io
  • icp-api.io

During this period, that preview boundary node will track the regular boundary node releases. ie if there is a production deployment during this time, this particular boundary node will be patched and upgraded at the same time.

Wednesday July 5th 5pm UTC - For a duration of 24 hours we will update the production boundary nodes to accept /api requests only on the base domain names. /api requests to other hostnames will be redirected. After the 24 hours have elapsed we will rollback the changes. The idea is to rollback faster in case of widespread outage and to give you time to push fixes before we make the changes permanent.

Wednesday July 12th at 5pm UTC - For a duration of 24 hours we will update the production boundary nodes to accept /api requests only on the base domain names. /api requests to other hostnames will be redirected. After the 24 hours have elapsed we will rollback the changes. The idea is to rollback faster in case of widespread outage and to give you time to push fixes before we make the changes permanent.

Wednesday July 19th at 5pm UTC - Production boundary nodes will only serve /api requests on the base domain names hostname, and redirect /api requests received for other hostnames.

Wednesday July 26th at 5pm UTC - Boundary nodes will stop accepting /api requests to hostnames other than the base domain names.

Please use this forum thread to reach out with any questions or issues you encounter.

*** July 7th 2023 - Edited with the new timeline in response to issues from July 5th***

9 Likes
2

Hello. Can you please clarify what this will mean for canisters that host assets that aren’t certified? For example, most NFT canisters allow for viewing of NFT images through <canister_id>.raw.ic0.app. Will that still be possible after this change is implemented?

2 Likes
3

Hi @LightningLad91 -

First some context:

/_/raw/ and <canisterid>.raw.ic0.app are different things:

  • /_/raw is a special path used by some projects to serve a dapp specific service worker
  • <canisterid>.raw.ic0.app is a way to get the decoded http response directly from the boundary node.

As far as <canisterid>.raw.ic0.app - we don’t expect that behavior to change. You can verify it for yourself by setting an entry like this in your /etc/hosts file:

# replace <canisterid> with your actual canister id
162.247.129.233     <canisterid>.raw.ic0.app

And the behavior should remain the same.

3 Likes
4

Thank you. Really appreciate the explanation and I am relieved that those collections will keep working.

1 Like
5

Attention agent devs @quint, @neeboo, @rdobrik, @Gekctek, @levi, @ccyanxyz

4 Likes
6

What is the minimal agent-js version to be safe?

1 Like
7

Hi, could you clarify that what should we do with the third one “requests by crawlers for SEO will require certification”? What changes do we have to make with the SEO crawlers?

1 Like
8

Will this affect us if we are using an Alternate(fork) of icxproxy?

1 Like
9

bump ^^

Would be nice to know what we need to be at in order to ensure there is no breaking change in our code

10

I would suggest using at least: Release v0.15.4 · dfinity/agent-js · GitHub

However, looking at the code it looks the rewrite from *.ic0.app to ic0.app has been in agent-js since this commit: feat: add url rewrite to `HttpAgent` (#516) · dfinity/agent-js@7b4333f · GitHub which is quite a while ago (Dec 2021)

2 Likes
11

Thanks @raymondk! This seems to translate to v0.10.3

12

It’s hard for me to say without knowing exactly how you are using your fork of icx-proxy.
The most important part IMO is that you are making /api requests to one of the base domains. For example you shouldn’t be making /api calls to <canisterid>.ic0.app or boundary.dfinity.network

Hope that answers your question - but I’m happy to discuss it live if you want to go into your specific use case.

13

Yes - but more recent is always better :wink:

1 Like
14

hi @cyaolong -

For some context - search engine crawlers go through a special path where they will use icx-proxy on the boundary node to translate api responses into HTML - very similar to how you would be accessing resource over raw (ie canisterid.raw.ic0.app)

In general:

  • if you are using raw and that’s all you care about, nothing should change for you.
  • if you are using the regular endpoint and you do not have a custom service worker then your assets are already being certified and you should be good to go.

Folks who might run into issues are likely to be those who are using a custom service worker that doesn’t enforce certification.

Hope that helps

1 Like
15

Our fork is here: GitHub - ORIGYN-SA/icx-proxy: Fork of ICx proxy that runs perpetualOS the basic usage is in the readme.

Looks like we are passing in the replica uri upon config: icx-proxy/src/main.rs at f6bdce44189a56c9a33d071b41049d2d06e78b34 · ORIGYN-SA/icx-proxy · GitHub

Looks like we are using in the setup. Sounds like we need to update that?

https://ic0.app

in

icx-proxy --replica "https://ic0.app" --redis-url "redis://tf-icx-proxy-redis-cluster-dev-us-east-1.tvmdlr.ng.0001.use1.cache.amazonaws.com:6379" --phonebook-id "ngrpb-5qaaa-aaaaj-adz7a-cai"
16

Hi @skilesare

Yes, you should use one of the three base domains. Ideally, you use icp-api.io as it is dedicated to the API endpoints, whereas the others also serve “browser requests” (i.e., the service worker). Due to that, icp-api.io is less likely to be added to a domain blocklist like the one of Spamhaus.

1 Like
17

I’ve noticed refreshing any page on my dapp with a URL that has a path after the apps entry point is now simply failing, I’ve noticed the same when refreshing my browser after a search on Kinic, here’s my error (I presume the route after the base domain is the problem here):

Timestamp: Tue, 27 Jun 2023 14:29:01 GMT

Request: {
“method”: “GET”,
“url”: “Internet Computer Loading”,
“origin”: “https://6t32g-3yaaa-aaaao-ai56a-cai.icp0.io”
}

Response: {
“status”: 500,
“statusText”: “Response verification failed”,
“body”: “Response verification failed”
}

Unless I haven’t understood correctly does this mean we can no longer reload specific pages on our dapp without the base address as our entry point? Navigation through my dapp works without problem but reloading assets for a specific page fails…

18

@Fortaite this is related to a different issue:

1 Like
19

Thank you, I’ll take a look.

20

so during the time deploy on ic is not working?