As mentioned in other places on the forum, we have made great progress with the recovery and stability of Trusted Execution Environment (TEE) enabled nodes. Following months of successfully running individual TEE nodes in all roles (unassigned, replica, and API BN), we are now ready for the next step: creating a dedicated TEE-enabled test subnet.
Because TEE nodes offer a higher level of hardware-backed security, we propose using 7 nodes for this subnet instead of the usual 13.
For now, this subnet is for test purposes only to gather operational experience. It will initially be authorized-only; no deployments will be permitted at this stage. Once we have sufficient experience, we will propose opening it up and/or create more TEE-enabled subnets.
For more background on TEEs on the Internet Computer, check out the Learn Hub.
We will submit the proposal for this subnet shortly. Happy voting!
ICP moving to TEEs is super exciting and IMO a huge boost to credibility. Looking forward to being able to store my secrets in my canisters
Does moving from 13 to 7 nodes make HTTP outcalls easier to forge? Or do they actually become harder to forge because the HTTPS certificate is checked in the GuestOS, which is now much harder / impossible to mess with? I guess the host could only block the outcall by IP then?
So maybe the only downside of 7 nodes is it might be slightly easier to take down a canister
Fantastic news! Congrats on all of the hard work coming together. I plan to review the proposed subnet configuration tonight or tomorrow before voting.
Vote: ADOPT
Summary: Congratulations to DFINITY and the node providers who have TEE enabled nodes for reaching the milestone of creating the first TEE-enabled test subnet. I’m happy to be here voting on this milestone. I wish you all continued success with developing and deploying this new capability.
You may wish to follow the Wenzel known neuron if you are looking for a reliable, credible, sensible Followee for every proposal topic. Visit codegov.org to learn more about my voting philosophies and the merger plan for the Wenzel, Synapse, and CodeGov known neurons.
Adds SEV-en nodes to create the 1st TEE enabled subnet for testing purposes and not open for public.
Kinda feel like this sort of proposals shouldn’t even be up for debate. Testing and R&D should just be executed by Dfinity period.
All 7 nodes are Healthy unassigned.
You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.
CO.DELTA
We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:
Look: We observe the details and context of NNS proposals.
Test: We test and verify the claims made by those proposals.
Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.
Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.
Again I think this is missing the point of the proposal. It’s a brand new 7 node test subnet creation. It does not fall under the existing requirements and or suggestions.
I ran some analysis software I wrote for comparing subnet configuration at a distance (with colours representing distinct values, to highlight commonalities and differences). The proposed new subnet is the first row below. It’s mostly consistent with many other application subnets, but (as @aligatorr89 has also highlighted above) the SEV feature is not set. Other subnets have this set to false, and this proposed subnet doesn’t specify it (rather than setting it to true, as one might expect for this subnet).
I also noticed that the max canister count is set to 0 (which means unspecified). This is actually the same for many other new subnets. I don’t think this is correct (for this or the other relatively new subnets that doesn’t specify a cap).
I don’t think you should be discouraging proper reviews, whatever the context. Also, I think you’re wrong. Mistakes made now could easily slip under the radar later, under the assumption that proper due diligence had already been performed when creating the subnet.
TLDR: Before adopting this new subnet the IC Target Topology should be updated by a motion that sets the requirements for this new type of subnet. At the moment there is nothing against which the parameters of the subnet can be validated.
@aligatorr89 has done a great job highlighting some unexpected parameter inconsistencies above. My analysis tooling has also flagged issues.
There will be 0 DFINITY-controlled nodes in this subnet if this proposal executes, complicating disaster recovery in the event of a subnet stall. The presence of a DFINITY node is a rule is built into the mandated IC Target Topology. My understanding is that available DFINITY nodes are all Gen1 (precluding them as an option).
Decentralisation Stats
Subnet node distance stats (distance between any 2 nodes in the subnet) →
Smallest Distance
Average Distance
Largest Distance
PROPOSED
1354.571 km
10055.187 km
16831.036 km
Subnet characteristic counts →
Continents
Countries
Data Centers
Owners
Node Providers
Node Operator
PROPOSED
4
7
7
7
7
7
Largest number of nodes with the same characteristic (e.g. continent, country, data center, etc.) →
The above subnet information is illustrated below, followed by a node reference table:
Map Description
Red marker represents a removed node (transparent center for overlap visibility)
Green marker represents an added node
Blue marker represents an unchanged node
Highlighted patches represent the country the above nodes sit within (red if the country is removed, green if added, otherwise grey)
Light grey markers with yellow borders are examples of unassigned nodes that would be viable candidates for joining the subnet according to formal decentralisation coefficients (so this proposal can be viewed in the context of alternative solutions that are not being used)
You may wish to follow the CO.DELTA known neuron if you found this analysis helpful.
CO.DELTA △
We’re a verifiably decentralised collective who review IC deltas (changes applied by NNS proposals). We follow a common code:
Look: We observe the details and context of NNS proposals
Test: We test and verify the claims made by those proposals
Automate: We automate as much as possible by building increasingly sophisticated tools that streamline and strengthen the reviewal process.
Every vote cast by CO.DELTA is the result of consensus among diligent, skilled and experienced team members acting independently. The CO.DELTA neuron follows the vote of D-QUORUM on NNS topics that the CO.DELTA team does not handle directly. You can therefore follow CO.DELTA on all topics and rely on the highest quality of vote.
Note that this analysis involved data provided by the IC-API, which is not open source. I’m in the process of switching over to more verifiable sources of this sort of information for future proposal reviews. See here for related discussion.
@rbirkner, out of interest, given that there are more subnets than we needed right now (and no signs of them being needed) why not appropriate one of the existing subnets that are still private, and have nothing deployed to them yet?
As of this commit which was included in replica version b0a37d0119a5df1dad84e50dc8717b77978d8f04, the semantics of max_number_of_canisters being set to 0 have changed: instead of meaning no limit, the replica’s default value, i.e 120_000, is used.
At a first thought, a TEE-enabled subnet doesn’t really change anything in terms of HTTP outcalls. Whenever a canister makes an HTTP outcall, each replica makes the same outcall and as part of that checks the HTTPS certificate when establishing the session (independent of whether it is a TEE-enabled node or not). There is no opportunity for a MitM.
Blocking the HTTP outcall is always possible (independent of whether it is a TEE-enabled node or not) as the node provider controls the network. However, as you mention, it is easier to block enough outcalls as there are fewer replicas in the subnet.
Regarding disaster recovery:
The DFINITY node in a subnet won’t be needed for SEV subnets. The point of SEV is to completely lock out the node provider/operator. We have added more tooling to enable recovery even in this case.
There are now three levels of disaster recovery:
If the orchestrator is running, we can rely on the newly added proposal called SetSubnetOperationalLevel. This proposal allows to halt a subnet, set a read_only SSH key on all nodes in that subnet and set a recovery SSH key on some node (replacing the need for a DFINITY node).
If the orchestrator is not running, we have added two ways to get the node into a state where the orchestrator is running again and we can rely on step 1 above. The two methods to do that are (we have described both in more detail on the TEE page on learnhub):
Manual rollback initiated by the node provider.
Booting into an alternative GuestOS.
And a separate reason: as of today, DFINITY has no Gen2 nodes on mainnet, which would be required to have an SEV-enabled node.
not needed for TEE subnet
