We are working on this. Will let you know how we get on.
6 Likes
I did a small change and got BtreeMap to work with canister stable storage: GitHub - victorcastro89/Ic-stable-storage: Rust Stable Storage Implementation
As soon as I have time I want to segment the stable storage so I can create multiple instances of btreeMap with different Max key and Value sizes so I can optimize the storage.
Do you guys thik is a good Idea to stoge images with it? 500Kbytes node size will allow 2Mb image be searched by four Keys in the Btreemap.
9 Likes
Sounds like a good idea, the only limitation I can see is that once you allocate the stable storage memory it seems you can’t deallocate and so you may want to use the write conservatively.
2 Likes
Hey folks, I am the author of StableBTreeMap. I was out the past two weeks so my apologies for being a bit late to the party.
Yes, it is absolutely possible to use StableBTreeMap within a canister as has already been demonstrated by @ovictor, and it was initially designed with that intent in mind.
Currently, we are prioritizing getting the bitcoin integration working over making the StableBTreeMap usable for canisters, but given the interest, I can create a small demo repo of how to use the StableBTreeMap within a canister for those who are interested.
There are some caveats here that are worth noting with using StableBTreeMap within a canister:
-
We didn’t do any in-canister profiling to measure how many instructions are required to access an element in
StableBTreeMapvs a standardBTreeMap.StableBTreeMapis expected to consume more instructions, but we haven’t yet looked into how much more compared a standardBTreeMap. Depending on your use-case, this may be a factor to consider. -
With one single stable memory, the only way to have multiple instance of
StableBTreeMapis to allocate, say,Xmemory pages to the firstStableBTreeMap,Ymemory pages to the secondStableBTreeMap, etc. This leads to developers having to allocate potentially memory than they really need, and it does put some constraints in terms of how big yourStableBTreeMapcan grow. This caveat will be addressed once the IC supports multiple memories.
There are also some changes to the API that I plan to do (e.g. support for generics), that will make the data structure easier to work with and closer to a standard BTreeMap in terms of its API semantics.
I’ll aim to have a demo repo to share with you folks by end of next week.
Update: I created some examples that you can find in this thread.
23 Likes
Please excuse the rudimentary questions in advance. But can the ecdsa threshold signature of a BTC product key be performed on two subnets at the same time? Or is the BTC subnet the main one and the NNS the sub one, so that if the BTC subnet fails for some reason, the NNS can take over?
Threshold ECDSA deployment with the production key that will be hosted on two large subnets on IC mainnet for public consumption (NNS as key backup subnet, another large subnet as signing subnet)
No need to apologize for your questions, it’s a legit one. 
If I understand your question correctly, you are asking whether two ECDSA signing subnets can produce ECDSA signatures at the same time, correct?
In the current setup, we envision that we have a single active ECDSA signing subnet, which will be a 34-node subnet (for the production release) that is still to be created. Let’s call it A for “active” subnet. The NNS will act as a key backup subnet. Let’s call it B for “backup” subnet. Under normal operation, all signing requests will be routed to the signing subnet A. B just passively holds the ECDSA private key for reasons of backup (key availability).
Let’s assume that one or a few nodes in subnet A fail, which is the likely failure scenario. Then A can still continue producing threshold ECDSA signatures as our protocol can handle failed nodes. Assume the case of A failing completely. Then we would currently likely not switch signing to B (the NNS) as threshold ECDSA is very heavy on CPU and this could negatively affect the NNS. We would either repair A or deploy a new signing subnet A’ and recover the threshold ECDSA private key from B (NNS) to A or A’.
The approach we have is largely compatible with having multiple active signing subnets, if this is required in the future for scaling out. In order to deploy this functionality, we would mainly need to implement deterministic load balancing on the message routing layer to load balance the signing requests to multiple signing subnets.
Hope that answers your question about multiple signing subnets.
6 Likes
Thank you for your very clear explanation.
One more question.
The NNS will act as a key backup subnet.
Backup means that you have the private key in its complete, undistributed form even if it is encrypted. If that is correct, is there any danger of it being lost or rewritten? If the best security for this INTEGRATION is to not have it read as a private key with secret dispersion by chain key technology, wouldn’t it be dangerous to have the private key somewhere in its complete form, whether it is a backup or in NNS? I am sure you have prepared considerable security, but what is the basis for being okay with that?
No, any threshold ECDSA key is always available in secret-shared form across all replicas of the respective subnet. Threshold ECDSA keys are never available in reconstructed form, not during generation, signing, or moving to another subnet.
When a threshold ECDSA key is available in secret-shared form on one subnet, it can be re-shared to another subnet. That is done by computing ECDSA dealings encrypted for the target replicas, providing those to the target replicas, and computing the secret shares from those.
For our production deployment, we will first generate the t-ECDSA key on the NNS subnet and then re-share it to a 34-node subnet that is generated from 34 newly-provisioned nodes as part of this. Then the signing capability is enabled only on the new subnet. That gives us the desired target situation of one active signing subnet and one “backup” subnet. On each of the subnets, they same key will be available using a different secret-shared representation.
Hope that clarifies the backup question you clears up your doubts w.r.t. security of the envisioned scheme for key management.
10 Likes
Great!
I understood very well. Amazing.
Sorry for the random comment. I just wanted to say that I think NIDKG and threshold cryptography is the coolest thing about the Internet Computer. So powerful. Probably under-appreciated by most in the broader community.
8 Likes
Speaking of which I saw an old video the other day with founders of other L1s speaking on why they “stayed away” from the core cryptography that powers DKG/TS, saying that it is unproven/requires weaker security assumptions. I would like to know what their updated opinion on that topic is. Proof of Stake Panel Discussion - Silicon Valley Ethereum Meetup - YouTube
1 Like
Zcash uses a trusted setup for its cryptography.
Ethereum is using BLS for their Beacon Chain afaik. Seems they have come round to the idea of using BLS.
3 Likes
So there is a risk of losing keys forever no matter how small the chances of it happening are
They are not really talking about DKG / threshold signatures, but rather about BLS signatures and the bilinear pairings with gap Diffie–Hellman assumptions underlying it. As Dom pointed out in the video, BLS and pairings stem from 2001 (https://www.iacr.org/archive/asiacrypt2001/22480516.pdf), and already in 2016 bilinear pairings were widely used in cryptography. Today (as @Maxfinity also points) out Ethereum’s beacon chain will use BLS signatures and many other blockchain projects do too.
2 Likes
I think we’d have major issues across the board if the system subnet (NNS) had a catastrophic failure like that.
Of course NNS is the single point of failure for the whole computer. My question was if it was possible or not.
Also can someone please tell me how many nodes of the subnets have to be malicious to do any harm in this context. And, when is DFINITY planning to finish developing the Periodic node shuffling feature
2 Likes
The IC is generally built on the assumption of 2/3 of the nodes being legitimate. From what I gathered in discussions a malicious amount of >1/3 but <2/3 can only stall the subnet, to be actively destructive there would have to be >2/3 malicious nodes. For comparison: in BTC, the threshold for both (stalling and destructive action) is at 1/2.
2 Likes
Understood. I can’t think of a single system that has managed to 100% eliminate the risk of losing a key. From a key management perspective I’m pretty impressed with what they’re doing. It seems pretty resilient and will only get better with time.
1 Like
Is this the same for the ECDSA subnet? IIRC there was a community conversation last year where one of the cryptographers at Dfinity explained that only 1/3 malicious nodes were needed, and that is why the subnet was going to have a node count equal to that of the system subnet.
Edit: It was Victor Shoup; here is a link to his presentation. There may be more explanation during the Q&A at the end but I couldn’t find it. Community Conversations | Threshold ECDSA Signing - YouTube
Edit 2: I just realized there is an updated ECDSA Threshold video from last month. In this one it sounds like the threshold is higher than was originally planned. I would definitely appreciate any clarification you can provide @dieter.sommer Community Conversations | Threshold ECDSA - YouTube
Also, thank you for uploading the latest video. I think these are wonderful resources for those of us who are more visual learners.
2 Likes
DFINITY is focussing on delivering SNS and BTC integration at the moment. Afaik, no design or implementation plan for automatic periodic reshuffling of nodes exists yet.
1 Like