New proposal types for firewall configuration

The Internet Computer node machines use a firewall to protect themselves from network attacks. Currently, all node machines use exactly the same firewall configuration, which is set in the registry via a proposal (propose-to-set-firewall-config). This firewall configuration is provided as a single string that defines the ruleset in nftables format (which is the format used by the firewall running on the guest-OS), and a list of IPv6 prefixes that is then injected into the former string. The list is updated with every new datacenter that is joining the IC.

This process is problematic for multiple reasons: first, it assumes that all IC nodes will always use nftables as their firewall. This might not be true in the future and we would like to decouple the IC protocol from specific implementation details. Second, there is no way of setting specific firewall rules to certain nodes, or certain subnets, etc. Third, the list of IP prefixes needs to be constantly updated with the onboarding of new data centers.

For these reasons we have modified the process and are introducing three new proposal types. The new proposals allow to specify the logic of the firewall rules, independent of the target platform that enforces them. These are the three new proposals:

  • propose-to-add-firewall-rules(scope, rules, positions, hash)

    • scope: scope where these rules should be applied. This can be one of:
      global - apply to all nodes (in the future also to boundary nodes)
      replica_nodes - apply to all (core) node machines
      subnet(subnet_id) - apply to all nodes that are assigned to the given subnet_id
      node(node_id) - apply to a specific node with the given node_id
    • rules: a list of rules to be added
    • positions: the positions where these rules should be added (lower position = higher priority)
    • hash: a SHA-256 of the expected ruleset for the given scope, after addition of the new rules. This is used to verify that the result ruleset is as expected by the caller.
  • propose-to-remove-firewall-rules(scope, positions, hash)
    The parameters for this proposal are similar to the ones above, where positions here are the positions to remove the rules from.

  • propose-to-update-firewall-rules(scope, rules, positions, hash)
    Again, the parameters are similar, and the positions indicate where to replace existing rules with the new given ones.

In addition to the above proposals, we are modifying the code that generates the firewall configuration on node machines to translate the rules from the logical representation in which they are stored in the registry to the target firewall platform (e.g., nftables). In addition, it will automatically whitelist all nodes that are listed in the registry to talk to all other nodes on the ports used by the IC protocol. This code will be part of the upcoming releases in the next few weeks.

Meanwhile, we plan to use the new proposal types soon, in order to prepare the firewall rules in the new format and put them into the registry.

I will be happy to provide more information for anyone interested, and to answer any questions.

Yotam

11 Likes

Hi @yotam,

Can you give a brief background on the reasoning behind the latest Update firewall rules proposal? (with more description than listed in the proposal)

Wait, I thought a key pitch for the IC and blockchain technology is that firewalls are no longer required or needed? If firewalls are protecting the node machines and data and will continue to do so, then why would we need blockchain tech to replace the whole IT stack? I’m confused here - how are the IC firewalls different from other firewalls, and shouldn’t DFINITY stop using the “without firewalls” hype phrase when talking about the IC?

1 Like

The nodes machines aren’t the IC, they are just regular computers.

Don’t buy into Dfinity’s marketing too much, they also say you don’t need a DBMS on the IC for that matter and while it is technically true, in practice you very much need one for any non trivial dApp.

2 Likes

@JaMarco, Where are these “regular computers” described in the Internet Computer documentation? All I see are “nodes in subnets” and “boundary nodes”, which are all nodes in the IC managed by the NNS via “node providers”. Why would the word “node” also refer to a computer not in the IC network when “node” always refers to a computer in a network? If not the IC network, then which network do these “regular computers” belong to? I’m probably missing something obvious, but now I’m even more confused.

Hi @justmythoughts , the update firewall rules proposal simply updates a set of existing rules, as explained in this post.

The firewall rules are stored in the registry as an ordered list per scope. A scope is the scope in which firewall rules are applied, this can be a single node, a subnet, or all replica nodes.

The proposal you linked updates one rule in the “all replica nodes” scope. If one wants to verify it, they can fetch the existing rules of that scope (using ic-admin: ic-admin --nns-url https://nns.ic0.app/ get-firewall-rules replica_nodes) and compare the rule at position 0 (the one that is being updated) to the proposed updated rule.

W.r.t the other recent questions on why there is a firewall, the claim for the IC was always that devs don’t need to worry about firewalls. But we do use firewalls as part of the framework. We are working on making the rulesets more permissive and minimize them to the bare minimum necessary for security, but we never claimed, to my knowledge, that we do not use firewalls on the node machines.

4 Likes

@yotam, Dom has said in multiple videos that firewalls are not needed for the IC, and that the IC could theoretically replace almost all current spending on the cybersecurity industry for those that migrate from Web2. I could probably dig up a couple of clips to that effect if someone insists, since it would take some time to watch his lengthy videos again. However, that should not really be necessary. The below images are straight from the “Internet Computer Infographic” web page here, from which I highlighted the key text in yellow: Internet Computer Content Validation Bootstrap. I’m frankly a bit disappointed in the overselling of an “open Internet” that “doesn’t need a firewall”, since the reality appears to be that some walls are definitely still required.


1 Like

@Sabr I am not a marketing person so I am not responsible for any if these publications and so I am not trying to explain them.
To my understanding, the point has always been that canisters do not need firewalls, among other legacy IT services you might have had to maintain were you running on the cloud.
The IC itself runs on node machines which are servers in data centers. Nothing changed at this physical layer of the stack, it is just “hidden” from the developers.
In my opinion, this does not contradict at all any of these publications you posted above.

4 Likes

I’m sorry, but there most certainly is an unequivocal contradiction, since the official marketing makes it clear that the “tamperproof code” hosted by the IC blockchain (i.e., the canisters) “does not need to be protected by a firewall”. Moreover both “systems and services don’t need a firewall”, where “systems” implies the NNS or SNSs, and “services” clearly implies the canisters. These marketing statements are categorically false based on this thread conceding that a firewall is indeed necessary.

Moreover, even Web2 applications (analogous to canisters) are not individually protected by any firewalls. Only the network that they are on is protected. So to make an artificial distinction between the IC (network) and its canisters (applications) is not even a logical distinction in comparison to Web2.

What this thread is telling me is that the IC is similar to (though admittedly not the same as) any other walled enclave / network on the Internet. In effect, the Internet Identity is not just a common identity for dApps, but also a login ID and password to a walled-in network. In fact, it will likely also be a login ID and password to many walled-in networks / enclaves within the IC too. It won’t make a lot of sense in the future to have only one IC firewall to keep out hundreds of millions of anonymous individuals when hundreds of millions of Internet Identity users are already behind the main IC firewall.

I’m gradually coming to the conclusion that public subnets should indeed be the norm rather than the exception. DFINITY is trying to conform the IC to a Web2 world by creating private enclaves behind walls, rather than building for a much more decentralized Web3 world run by DAOs, where full data transparency of all DAO data is the gold standard. Whenever private organizational data is maintained, an elite technocracy must also be maintained to protect that data. Ultimately, of course, that will just perpetuate centralized organizations and a Web2 world.

I would really like to hear from someone at DFINITY as to why these firewalls are really necessary in a blockchain infrastructure. I would also like to know how this firewall will remain protective or very relevant once hundreds of millions of hypothetical users with Internet Identities get behind that firewall in the future.

1 Like

I’d like to hear more too however I think the meaning is your software itself will not need a firewall protecting it because it’s tamper-proof. In the picture above you posted there is an example of hackers modifying server code and changing it, which firewall systems need to catch, but a tamper-proof smart contract piece of software doesn’t need that protection.

That’s my understanding but again I’d like to hear more as this is something I mention frequently when trying to explain advantages of building on the IC.

1 Like

This in specific should be specified in the advertising. I would say it is misleading at the very least. False advertisements could be on the dramatic side. It seems like another “you don’t need a bridge”, and we def have a bridge.

On the lower level (for someone newer to the industry) what does this mean for the IC overall? Does this mean that if a malicious actor passed the firewalls, they could indeed “hack” the IC? Making the tamperproof the same misleading ad style? If not, can you see why individuals would be skeptical when clearly there are A LOT of miss leading advertisements?I mean is this their marketing strategy? Rely on hypotheticals or “in theory” and not relay the “in practice methods”. If so, touche, I see what you all did there lol

1 Like

Not just on the cloud but for companies that run with everything on site - even smaller ones. I’ve worked for companies where everything was hosted on site and that is a complete headache and costs a lot of money.

Building on the IC would take time to develop but would eliminate the need for much of what we had to maintain including the firewall - so in that case eliminating the need for a WAF would be accurate to me.

1 Like

But can an off the shelf solution account for all types of use? There is a reason why there is no standard for firewalls and that is everyone requires one that fits their specific needs.

Tbf the firewall claim sounds like “you don’t need DBs” all over again: true at first glance but crambles once you actually start building. Eventually we’ll go full circle and have to reinvent the wheel.

I heard Dfinity and by Dfinity I mean Dom make the same claim for backups and cybersecurity expenses too and you very much need both on the IC. Just cause the state is replicated, it doesn’t mean it can’t get corrupted and must be rolled back and security audit are required, even more so for immutable or financial dApps.

3 Likes

@Sabr, please provide links to the video.
Given your fierce commitment to blockchain openness, you may have been wishful thinking while watching the video.

Sure, here’s one per the link below, where Dominic states at the 23:25 mark, “When you build on the Internet Computer, you don’t need firewalls to protect the system”. That’s the direct quote, no wishful thinking required. Moreover, it is not just the blockchain or smart contract being referred to here, but the whole IC “system”, including the node servers and subnets hosting the canisters. I recall this sweeping claim about no firewalls being required made elsewhere too. He also directly implies in the context of his slide that this would completely disrupt the $172 billion / yr. currently spent on cybersecurity costs. This is again misleading if cybersecurity investments and maintenance are still required on the IC like in Web2.

Finally, if servers and subnets need firewalls on the IC, then that means they also need offsite backup and restore functionality, security compliance audits, etc., which could imply some very significant cybersecurity costs that won’t all go away in Web3 vs. Web2. Perhaps worst of all, these firewalls imply a deeply trusted technocracy with privileged access to secure and maintain the walled-in network, which runs counter to the whole blockchain ethos. Perhaps this ideal blockchain ethos is indeed “wishful thinking” that can never be achieved. If so, then don’t claim that it has already been achieved!

Hi folks, thank you all for the discussion. I must say it is also at a very timely moment. Internally we are indeed discussing removing the firewall rules for the public HTTPS endpoint. Firewall rules are great way for risk management for certain adversary scenarios that’s why they exist today.

I am wondering in your opinion, excluding the PR and marketing aspects of using firewall rules, what will be the value added to the community by removing the firewall rules and making replicas open to everyone (not only the boundary nodes).

2 Likes

@rumenov, thanks for replying. Perhaps start by explaining what purpose the existing “firewalls” serve, since they could be materially different from what everyone currently understands firewalls to be under Web2. For example, if their primary purpose is just to protect against bot attacks like Distributed Denial of Service (DDoS), which could overwhelm the system or eat up node computing resources without using cycles, then that is a different category in my view. If that is all it is, then it is actually quite easy to change the marketing message to something factually correct while still being quite compelling – e.g., “no firewalls against humans” or “firewalls only against nonhumans”, instead of “does not need any firewalls”.

In fact, the related marketing message could be even more compelling than it is now, since it’s not just about “no firewalls” anymore. It is also about restoring an open Internet to humanity, largely free from the bot armies that currently plague existing social media platforms, which are obviously designed only for humans, not bots (hence the word “social”). For example, some pundits estimate that >60-70% of all social media posts are now from bots, which is outrageous.

Internet Identities combined with Proof of Personhood/Humanity, along with other countermeasures (like minimalist firewall features on boundary nodes targeting only nonhuman bots), could all play a part in the larger vision to restore an open Internet to humanity. And all of these could be implemented without violating the core blockchain ethos. They could also be implemented without creating a technocracy of trusted elites who control which humans get inside which IC walls, and which humans must remain outside.

In short, walls should not exist for real humans in an “open Internet”. So both the “fire” and the “walls” should preferably be only against nonhumans, i.e., the bot armies. The blockchain network should, in theory, be able to protect the rest. That said, I can understand if more traditional firewalls are still being used for the time being until all the necessary blockchain functionality is developed in the stack. However, there should be a viable plan to get to that true “blockchain singularity” vision where firewalls against real humans are no longer required. If such a viable vision exists (I personally don’t know), then present it as such. There should be no need to present it in a misleading way, or as if it has already happened.

1 Like

We can agree to disagree about the marketing materials and their reflection of the reality. Since I am not a marketing person, the best I can do (and have already done) is to provide this feedback to the marketing team.

But on the technical aspect I can provide more information. There are two layers here: one is the question of whether any firewall rules are needed. The other is whether the existing rules are good enough, or can we reduce and minimize them.

The first question is rather philosophical and as you can see there is already a debate here in this thread. Unfortunately I am on holidays so can’t find much time to elaborate now, but we have been quite cautious at the launch of the IC and decided to use firewalls to protect against network DoS attacks on the nodes. This is an easy and basic protection mechanism. It surely does not provide 100% protection and any other solution is way more complex. Unfortunately, other solutions such as scrubbing services cannot be directly applied to the IC as they are heavily centralized. We have a working group for this topic as we do need to protect IC nodes against network attacks.

Again, I am putting aside the question of whether this is in line with the marketing material. My understanding of this marketing was that it’s about apps running on the IC and not for the IC itself but I understand the confusion it creates. I would also like to point out that we at DFINITY never hid the fact that we are using a firewall on the nodes.

For the second question, as @rumenov pointed out, we are in active discussions on reducing and minimizing the rulesets. We are considering opening the API port on the nodes so anyone could directly access nodes. Also, the current list of rules is unnecessarily broad. It has been used since launch with adaptations as the network evolved. However, with the new feature that is described in the original post in this thread, nodes automatically manage most of the needed firewall entries based on the information in the registry. We are in the process of removing unnecessary entries. Unfortunately not all the required information is available in the registry, for example the addresses of boundary nodes. But (a) it is planned to have these in the registry as part of making boundary nodes NNS managed and (b) if we decide to open the API port for everyone we will not need to whitelist the boundary nodes separately.

I hope this information sheds more light on why the IC does use firewalls underneath. I am happy to discuss more technical issues, but cannot, unfortunately, provide answers on marketing questions.

3 Likes

No. Because the IC nodes are replicated you do not need backups. Security compliance audits would be based on the node provider’s policy. These are all not directly related.

I’d like to emphasize that the firewalls are there mainly to prevent volumetric DoS attacks and not to protect against exploits.