Let's talk about boundary nodes as a high-risk point of failure for the IC

I want to bring some attention to boundary nodes and the risks that they bring to the IC today since I feel like this subject does not receive enough attention.

Background
Query calls are executed by a single replica. IC’s trust model is that some replicas may be compromised (but not many). A query call may hit a malicious replica that could fake any response. To protects against this, we make use of certification. Currently, certification to secure query calls is implemented in canisters and is optional.

Problem
As far as I understand, boundary nodes provide service workers to legacy clients, i.e your web browser, so that they can verify certificates in query responses. Boundary nodes do not go through consensus so when a client installs a service worker, it is provided by a single boundary node.

The service worker acts as a source of truth when verifying query responses, so this is a critical point of failure in my opinion. A malicious party could run a boundary node and modify canister responses to make it seem like they reached consensus on something that they in fact did not. Apparently, it does not seem that a solution to this is near (source).

Scale
Query calls make up more than 90% of the IC’s traffic (source). Not all of these query calls are made through a service worker provided by a boundary node. However, this suggests that a high percentage of the calls on the IC is either a) not verified at all, or b) verified using certified variables with a service worked provided by a boundary node as a source of truth.

Recommended discussions to participate in
Long Term R&D: Decentralized CA and DNS (proposal)
Long Term R&D: Boundary Nodes (proposal)

Is there a reason why I do not see much discussion regarding this? Does anyone know of any progress to solve this problem?

5 Likes

I think the consensus (heh) so far has been that some flows can take a bad “query” as long as the critical parts of your app are done through update calls.

For example, say you display a user’s balance, and your js app verifies that a user has enough balance to purchase some monkey thing. You’re pretty safe getting the balance with a query, as long as you verify the balance through an update (either in the purchase flow, or before). Worst case scenario, the user has a mild panic attack :slight_smile:

1 Like

Always appreciate another point of view, and I can agree with that thought process when it comes to transactions of tokens a.k.a Bitcoin-level of functionality.

However in the context of a “world computer” or a “web3 blockchain”, I don’t think this makes much sense. If IC is supposed to be an alternative to centralized cloud services for e.g hosting websites, then query calls will have to be the most common type of call since going through consensus with update calls are too slow. We already see this in the stats of IC’s current traffic. Malicious responses to queries can cause all sorts of problems such as censorship, misinformation, and malware (which in turn can steal funds) to name a few.

I am extremely bullish on IC and an IC-dapp dev myself, but I don’t think we should look away from a problem like this.

1 Like

Yes, this is indeed a serious concern, especially with many apps already serving web apps from *.raw.ic0.app, against the constant warnings issued by @nomeata and others. I believe there’s a post somewhere documenting custom domains + a custom agent, that could work for validating whatever ic0.app serves, but then again you’re forced to “trust” some other 3rd party that serves the code for the agent itself…

There was also some talk about self signed ssl on-chain after some multisig is enabled, so that could possibly be another way of adding trust to the system.

2 Likes

Right, even if you utilize the service worker for certification you have the problem of either having to trust a potentially malicious boundary node or some 3rd party.

Interesting, do you have a link?

Hmmm. I remember it was somewhere in one of the ECDSA threads, but my google fu is failing me right now. I found a mention here, in what appears to be a transcript of a dfinity presentation:

ECDSA threshold signature technology, in simple terms, is to allow each canister to sign data without storing the private key, and this signature can be verified by the public key, and each canister can get a unique public key. This is in the same line as the Chain Key technology we have implemented, and it has a wide range of applications. For example, canister can directly initiate a Bitcoin or Ethereum transaction and sign it.

This means that what originally had to be done in a private environment to give the private key to the program can now be done in a decentralized environment. It can also be used in the issuance of SSL certificates, DNS custom domain names, and so on.

1 Like

Interesting. I would love to learn the technical details of anything that may solve this because the more I think about this problem, the more critical it seems.

Hi Gabe, what is the difference between boundary node and replicas? Is it probably same node?

How do you incentive boundary nodes?

Replicas run IC smart contracts, boundary nodes serve requests.

Thanks. Could they be same server? Or it has to be 2 nodes?
How do you incentive boundary nodes?

Did you find an answer to this? This is very interesting and I would also like to know.

Wouldn’t you just incentivize them the same way you incentivize any other node?

Likely, but I would like to have it confirmed. I am not well acquainted with the technical details of the disbursement system. Perhaps it is not as easy to prove your work as a boundary node compared to a replica node that participates in the consensus.

Can you chime in on this @yotam?

Query calls get you fast responses but indeed have no security guarantees. In our canister development guidelines the recommendation is to either use certified variables or update calls (or a quick query call followed by a confirming update call). DFINITY is exploring the option of query certification, where the user can dynamically select query calls and get the response certified. This will be more costly than standard query calls but faster than update calls since they do not have to go through consensus as update calls do since they do not change state on the IC.

Wrt trust in the boundary node, DFINITY is looking at developing a web extension to give an alternative to trusting the boundary node to deliver the right service worker. Another forum thread has some discussion.

Interesting, but I assume that the service worker is still needed at the client in order to validate the certificates?

A web extension is definitely useful for more tech-savvy people but as I argue in the thread you referred to, it takes away a lot from the simplicity of the IC if we were to rely on it. Today, my mother or even grandmother could access my dapp on IC. If we require a web extension (in order to be safe), then that would not be the case. My point is that, while it certainly is nice to have, I do not believe that a web extension is the solution to the problem in the scope of mass adoption.

2 Likes

There is a lot to unpack in this thread, and there are many good observations on parts that are ongoing work within DFINITY, at different levels of the process.

  • Security of queries: Currently, queries are answered by a single replica and responses are not signed. That leaves them vulnerable to attacks by either a replica or even a boundary node. In the future, responses will have to be signed. We have been working on this issue internally, and will present our suggestion soon.
  • Boundary nodes are currently not rewarded in the same way that regular nodes are. This will change in the future, so that there is an incentive for more node providers to operate a boundary node.
  • The distribution of the service worker is indeed a security-critical part of the system. As @JensGroth wrote, we will provide a web extension as one short-term stop gap for tech savvy users. We are also working toward broader solutions such as using signed HTTP exchanges (SXG), but that’s still a bit further down the road.
5 Likes

Can you provide some clarity? Current information on NP remuneration models at IC Wiki does not reveal such a split

Sure! Boundary nodes are right now not represented as actual nodes in ICP – they do not appear at all. At the moment, all boundary nodes are actually run by the DFINITY foundation. This is obviously not a long-term setup; it will change later this year, and boundary nodes will then also be considered part of the protocol and rewarded similarly to other nodes.

4 Likes

A malicious party could run a boundary node and modify canister responses to make it seem like they reached consensus on something that they in fact did not

Hello, thank you for bringing up such an important issue!

I’m working on the Trust team in Dfinity and this is the exact problem that we are trying to address. Our approach to this essentially splits users into two groups, convenience focused users and security conscious users.

For security conscious users we have an e2e trustless solution in the form of a local desktop proxy: Try Out the IC HTTP Proxy: A Leap Towards Decentralized HTTP. This is still in a PoC phase but it is actively being worked on to get this to a more production ready state.

We are also planning to continue investigation into more alternatives, a mobile counterpart to the desktop proxy, web view wrappers, self hosted gateways and native browser integrations. We are not sure which of these options will be successful at this point, more work is needed before we can say for sure.

Browser based solutions (aside from a native integration) such as the service worker and browser plugins have proven to be cumbersome and unable to provide a truly trustless solution. So we will continue to pursue more options outside of the browser.

Now for the other group, the convenience focused group. We do not have a short term solution for them that will provide an e2e trustless connection. A native browser integration would solve that, but that likely won’t happen for a long time since we are dependent upon external parties to collaborate with us on that front. So for the foreseeable future, convenience will come at the cost of some decentralization.

Decentralized API boundary nodes can help us with this though: Boundary Node Roadmap. We won’t be able to fully decentralize the HTTP Gateway, but we can at least federate it. Community hosted HTTP Gateways will be able to provide alternate routes to the same canisters. We will then look at finding ways to verify the behavior of these gateways.

4 Likes