Long Term R&D: Integration with the Ethereum Network

Maybe I’m behind on the Bitcoin integration architecture. In early designs I know that each replica was connected to Bitcoin and each subnet would come to consensus on the relevant state of the Bitcoin blockchain. Has this architecture changed? Is there one Bitcoin subnet now? Or does each subnet have its own Bitcoin canister?

2 Likes

Hi @dieter.sommer, Roman asks an interesting question which I also had.

If we can build ckETH using Phase 1, once Phase 2 is complete will there be a migration of the old ckETH to the new and improved ckETH? Or will it somehow replace it? How does this work?

4 Likes

I had expected Ethereum integration to be a light client or full node built client method rather than a JSON RPC API method. Are there any plans to implement these client methods in the future?

1 Like

This is an excellent question and I cannot give you a definitive answer here. In my opinion it is the community that would need to decide whether a chETH canister running in production would be shifted over from the Phase-1 to the Phase-2 implementation. The amount of ETH in such canister is probably a guiding light for this as well as how many cloud providers we have queried for each ETH balance check. Regardless of this, implementation of ckETH and the like can already be started soon, there will be a testnet phase etc., so the proposed approach can in any case help cut time to market for this.

Would like to hear what your opinions on your own questions would be.

This would be nice, however, the use of HTTP outcalls curently prevents the use of queries. The responses of HTTP outcalls inherently need to go through consensus.

The light client API of Ethereum is currently still worked on heavily and also the trust model of a light client is weaker than that of a full client which we are building on in Phase 2. A full Ethereum node built on chain currently exceeds what a single canister can do and thus would be too large of an engineering effort. Thus, both of those approaches are not suitable for a reasonably fast go to market. Note, however, that there is a bounty out regarding a design for an on-chain Ethereum light client. We think that the light client approach might be interesting once the light client APIs are settled and thus would like to have this explored.
@hokosugi Can you share what your concerns are with the approach proposed for Phase 2? It’s completely trustless and probably the most scalable design.

An Ethereum subnet is a subnet that gets enabled w.r.t. Ethereum integration, i.e., runs Ethereum nodes next to IC nodes. This is something we want to have on one or a few subnets, but not on every subnet. The overhead of an Ethereum subnet is quite considerable. I understand your concerns that you now need to trust multiple subnets, but multi-subnet dapps are one of the guiding principles behind the IC in my opinion. But to make it clear, you could enable Ethereum integration on many subnets if you really wanted to.

9 Likes

Thanks for the thoughtful response.

I am pretty sure most people (including myself) would like the Phase 1 ckETH to be migrated over to the Phase 2 ckETH. From a user perspective, it would be great to abstract away as much as possible the transition from Phase 1 to Phase 2.

We want to be able to announce an Eth integration now and then the tech improves over the course of the year. It will be fairly disruptive to have multiple types of ckETH and people will get confused between Phase 1 and Phase 2, particularly newcomers to the IC.

Is it possible to just have all the stuff happen in the background?

8 Likes

Thank you a lot for answering @dieter.sommer :pray:

My opinion will be unpopular, without a doubt.

But to facilitate your reading and save your time, here is my question at the end of my post : “Could you tell us if you see some advantages to bypass phase 1 and start with phase 2, taking the time, and after this only implement CkETH following the same order as for CkBTC ? Or for you, there is none dilemma and you are 100% to start with phase 1 ?”

Now, my opinion :

I would prefer that the phase 2 become the phase 1 and that only once this phase accomplished, CkETH be built upon this native integration. This, in order to follow the same founded logic behind BTC integration and CkBTC development. And currently, I don’t see any good reason to not follow the same order.

Even if it was possible to launch a first CkETH before Phase 2, and after this, « refresh » the CkETH by building it again relying on the phase 2 development, it :

  1. could look disordered and make people wondering « why did they reverse the order in the first place ? » ;
  2. could look inconsistent with BTC/CkBTC order launch ;
  3. would add complexity to the whole process of integration and cost much more resources to Dfinity than if they just had kept the same order that for BTC/CkBTC.

I feel like we put the cart before the horse here.

The BTC integration is a marvel and the CkBTC a promising success built upon it. Dfinity took the time yes, but they did well, because it has been launched once only, following a well justified order : technically and marketingly justified and understandable by everyone, even the non dev public (very important). Given this success, why don’t we just respect the same order even if it takes more time ?

In my humble opinion, here is why, but if I am wrong, please tell me.

People have been waiting for BTC integration for a long time. During that whole time, Dfinity have been heavily pressured and even harassed. We were in a fuddy environment, IC was attacked everywhere by everyone and you were being daily bullied by some hysterical people on Twitter and here. My point is I am afraid that this past is being determining Dfinity about the ETH integration and CkETH timeline and by consequence their order.

Maybe I am wrong, but I feel as if the ETH integration had been very quickly announced after the BTC integration in Twitter messages. People were even surprised by this proximity. We were told « it will be quickly out (in march), we could do it very soon, let’s do it all together. Help us people, you are welcome. Take it, this integration will be yours ».

My whole point is in these announcements : I am afraid that Dfinity have been like traumatized by the « accusations » of being slow, being centralizing the chain (by owning the process of the integration), being prioritizing BTC integration rather than doing other stuff presumed more important things by some fellows, etc.

Personally I have disagreed with each one of these accusations, and we are blessed that Dfinity prioritized the BTC integration and that they took the proper time to sort it out in due time and in due shape (even if I noted the acceleration in the last days, maybe to reassure the market and reward the patience of people). When I read that other devs could have done this while Dfinity should have done other stuff, we are forced to recognize that none dev did and maybe no one would have done it. We will never know.

Today, I see a sort of acceleration to integrate ETH in a way or another, and the fastest possible, so much that CkETH would come firstly and the native integration after only. At least, I would be like @dfisher in favor of finally having a proper CkETH developed with the same logic that CkBTC’s, so by not keeping the first version of CkETH which would be launched before the native integration. But ideally, i would be to start with phase 2 bypassing phase 1 and being patient to do things orderly and properly. How could we justify that a first CkETH would have been launched before being properly launched, « proper » meaning « following the same logic that CkBTC : waiting for the native BTC integration for well founded reasons ».

This is why I would prefer that Dfinity keep the same state of mind that they had for BTC integration, because it is clearly a success. Yes it took time, but now it is done and properly done. I feel uncomfortable with the narrative of the ETH launch, being divided in phases looking like starting with the end and finishing with the beginning. Etc. And I am afraid that it creates confusion, complexity and confusion, notably to the newcomers and the haters.

I would prefer that the native ETH integration come first, and once sorted out, that the CkETH is built upon it.

We have plenty of time before the next BTC halving. The bear market is still here. And anyway, Dfinity should stay the master of the clocks (« Rester maßtre des horloges ») by not being in a hurry, not being dictated the rythme and the order. This integration is too important to be « improvised ». We will have plenty of chains/tokens to integrate to let non Dfinity dev to operate their magic, but the first two ones are too important to improvise whereas a crowned success happened under Dfinity lead.

To summary : this integration should be driven EXACTLY as BTC’s. We should see the enthusiasm about BTC integration and CkETH to do exactly the same thing with ETH. There is no rush. The only rush is to do things properly. As a researcher myself (other domain), the main strength of IC is that its team is constituted of searchers, listening to the time of the research, not the contingencies of the markets. And it is for the best. Some doers will bring another and complementary spirit to IC, but Dfinity must stay as they are, keeping their searchers state of mind, doing things taking the necessary time rather than listening the time that the investors can psychologically handle.

Could you tell us if you see some advantages to start with phase 2, taking the time, and after this implement CkETH ? Or for you, there is none dilemma and you are 100% to start with phase 1 ?

I genuinely probably miss a point, so don’t hesitate to tell for which technical reasons you don’t proceed just like with BTC/CkBTC. But if it is only political or « marketingal », you should be in warrior mode like you did, for our greatest happiness.

6 Likes

To be clear, I think we should do Phase 1 quickly and then Phase 2 later, but for ckETH I think the user shouldn’t notice the difference when the migration happens (if that’s possible).

3 Likes

Would this not be possible in phase 2? Does phase 2 include a full Ethereum archival node? What type of node will phase 2 enable on the IC?

I would think this is very possible. Basically phase 1 has certain security properties and trade-offs, and phase 2 has most likely better security properties and trade-offs. If the community wants to build and maintain ckETH it should be a seamless integration, and behind-the-scenes phase 2 will simply increase the security.

If ckETH is an ICRC1 then the interface shouldn’t ever have to change, assuming the NNS or some other mechanism can upgrade the ckETH canister(s).

That being said, I have issues with the NNS having write access to these canisters, but perhaps that’s another topic for another time.

I think I’ve got the general idea correct here, but @dieter.sommer please correct me if I got anything wrong.

4 Likes

I don’t think that we are targeting an archival node, but probably a full node that prunes transaction history regularly.

On queries: AFAIK, we are currently thinking to use the HTTPS outcalls implementation internally, but each replica will route the request to its co-located Ethereum node. So queries wouldn’t be directly possible, but you could in principle cache (some part of the) ETH state in a canister to make it available as a query (at least on the same subnet for composite queries).

4 Likes

Does phase 1 complete?
Any Update?

We are increasingly living in a cross-chain world. As such, we should aim to not just integrate with the Ethereum network, but design this integration from the ground up to target all possible EVM-compatible networks. It’s not just enough to provide an Ethereum integration, there’s Arbitrum, Polygon, Optimisim, Coinbase’s L2, Binance Smart Chain, etc. Most DeFi activity is taking place beyond the confines of Ethereum. And once the Ethereum integration is complete, the community will naturally ask for further integrations. From this perspective, it would be great to use HTTP outcalls to accommodate as many different networks as possible. Let’s be driven by pragmatic business decisions.

Before we start building this on the IC, we should review the design decisions of other oracle networks, for instance ChainLink. With ChainLink deployments, you set up your own network of nodes, each node runs something similar to a docker container and has access to its own private state, e.g. can privately access its own API keys.

From this perspective, it is easy to see what is missing for both HTTP outcalls and the ETH integration, we need a way to direct different API keys and/or state to different nodes. As a user I should be able to set API key 1 to a subset of nodes, API Key 2 to another subset etc. With this feature cross-chain integrations like Omnic, used for bridging tokens, become secure; without it, I predict we will have major outages and DOS attacks on bridged tokens in the future. Currently, our scheme is not decentralised as a single node provider can DOS most HTTP outcalls that use an API key.

@dieter.sommer @Manu @benji @domwoe

10 Likes

@Roman, @dfisher, @lastmjs, all: Thanks a lot for your thoughts, and taking the time to write them down! Apologies for my late response, I am still recovering from a virus infection.

@Roman: My interpretation of your thoughts, at a high level, is that we should do the Ethereum integration diligently, do things properly, and not unnecessarily rush the integration. In other words, repeat the good things of the BTC integration. That’s the high-level semantics I get out of your arguments, abstracting from the concrete ckETH discussion. Please correct me if this interpretation is wrong. I could not agree more to this interpretation actually!

Now let’s discuss the proposal of Phase 1 and Phase 2 of the Ethereum integration. Phase 1 is thought of as an instrument to quickly make a first implementation of the API available to everyone in the community who wants to use it. The integration builds on HTTPS outcalls to one or more Ethereum cloud services like Infura, or QuickNode, thus has clearly a weaker trust model as Phase 2, but still is a solid implementation of an integration — just not trustless. For Phase 1, much of what is needed is there already in the form of the ic-web3 library by Rocklabs, which is already open source. The idea for Phase 1 now is to issue a grant to Rocklabs to retrofit the library to offer the same API, or one the is very close to the one for the Phase-2 integration. People will be able to build already on the API of Phase 1 early on. From this perspective, Phase 1 is analogous to the developer preview of the Bitcoin integration which was launched early on with the API close to the final one so that people could already build on it. The Bitcoin developer preview, however, did not do a real integration yet, so was less powerful than a Phase 1 of the Ethereum integration, which gives people more options on what they can do with it.

Your forum post was focussing on ckETH as application case for Ethereum integration. In my opinion, ckETH is only one use case, granted, a very important one, but there are many more: ckUSDC, ckUSDT, NFTs etc. One of your main assertions is that we should not launch ckETH before Phase 2 is complete. Thus, we should not transition ckETH from a weaker trust model of the integration (Phase 1) to a stronger one (Phase 2), but only launch with the strong one when Phase 2 is available. This definitely has some very valid points, e.g., avoid the reduced trustworthiness of the ckETH state acquired through Phase 1 integration vs Phase 2. From this perspective, your argument makes a lot of sense and ckETH’s strategy needs further discussion.

Continuing the ckETH discussion, a ckETH implementation will not be ready tomorrow: Someone needs to build an ERC-20-based minter, ideally customizable to any ERC-20 token, the code needs to go through a thorough security review, needs to run in beta phase on an Ethereum testnet for a while, and only thereafter can be launched for Ethereum Mainnet. Also, the question is whether DFINITY should build the ckETH minter canister, as it did for ckBTC, or the community should do it? That’s also something that is not yet clear, at least to me. Thus, for me, the ckETH question is not even the most important and most pressing one, it is one use case of many and its own timing dictates in many ways when it can be launched in production. Now to the concrete question: If ready, should it launch before Phase 2 is ready? I don’t know, this is actually a really tough question. Having a clean trust model without a migration to Phase 2 vs. a shorter time to market and our DEXs being able to use it earlier. An alternative is to have a preliminary chain-key Ethereum implementation that can bridge the gap towards the final launch. We had this for Bitcoin also in the form of icBTC by ICLighthouse.

My thinking goes into a direction that a community-launched chain-keyed Eth, lets call it icETH for now, based on Phase 1 might be a great idea. That is, Phase 1 is built and launched quickly so that everyone can start integrating their dapps with Ethereum. The API will stay (almost) the same for the Phase 2 launch. Then a first iteration of chain-key Ethereum in the form of icETH can be started with as well. And dapps can start integrating with icETH already. All of this helps reduce the time to market for the Phase 2 and Phase-2-based ckETH without severe drawbacks. The decisions are left to the individual community members on whether to adopt an early version of chain-key Ethereum as soon as possible and build upon it or whether to wait for the Phase 2 and a final ckETH to be ready. In any case, Phase 1 is an accelerator that does not cost us much, but helps a lot in getting the final API out so that our community can start building upon it. So it’s in many ways the most decentralized approach where everyone can do what they deem best for their projects. No one imposes anything on anyone. All we do now is to work with the community to enable this flexibility and shortening of the time to market.

@Roman, @dfisher, @lastmjs, all, I would really appreciate your feedback on this line of thinking.

I agree to that in that we should repeat everything that was good from the Bitcoin integration, not unnecessarily rush, and do things properly. But not be slower than we could be while doing things properly. Also let’s improve it where possible. E.g., offer people a means to start coding their dapp integrations as soon as we can with a close-to-final interface and also giving people a choice whether to run something in beta or production before Phase 2 is ready. I think there is no drawback of running any dapp, also ckETH, in beta with Phase 1. Also note that Phase 1 is not some insecure mess, but uses well-vetted cloud service providers as integration points to Ethereum. This will give us a lot of advancement in terms of time to market. For this reason, I don’t see a dilemma to start with Phase 1 — it’s a powerful accelerator for the community to build dapps using Ethereum. The question whether to launch the “official” ckETH with Phase 1 is a specific one that needs lots of thought. But considering that building ckETH also takes time, this might not even be a relevant question as ckETH needs time to be built, time to be reviewed, and time to be run in beta with an Ethereum testnet — it might not be ready that long before Phase 2 is ready. Considering this, I would see doing Phase 1 as outlined to be only an advantage to the community.

So, yes, DFINITY needs to take its time to build Phase 2 and ckETH, but we should support the community to start building their dapps as soon as possible. Whether individual dapps are launched in production with Phase 1 or only Phase 2 is up to the discretion of each project — we definitely don’t want to impose anything on anyone here.

@dfisher, I hope this has been answered with the above response. Just note that Phase 2 has started in parallel to the Phase-1 efforts. Phase 2 is not delayed or otherwise impaired by Phase-1 efforts — on the contrary, it benefits from the thinking w.r.t. the concrete API we want to offer.

@jordan, I fully agree that this is technically possible without doubt. @Roman’s main concern is that Phase 1 and Phase 2 have different trust models and switching the main ckETH implementation from one to the other is, at least theoretically, not very clean. It’s definitely something that deserves further discussion. We are currently pursuing Phase 1 and Phase 2 in parallel, according to the approach outlined above.

And, yes, ckETH should definitely be an ICRC1!

I think you have got the general idea correct here.

@lastmjs: For context, you are referring to queries to the Ethereum interface here. In theory, this is possible as an IC node could just query its collocated Ethereum node in Phase 2. However, XNet queries are not available yet, so there are things that would need to be built to have this. And, you are right, query support for Ethereum would be really great, but there are currently still some blockers for it.

@ all: I hope that my argument for doing a Phase 1 now can appeal to everyone, as its adoption is left to the community in a completely decentralized way. We only help the community getting Phase 1 built. This does not dilute our efforts for Phase 2 and it helps a lot in sharpening the final API for Phase 2 as we need to think about it very concretely. Also having dapps build on Phase 1 will help detect any issues, e.g., with the API, early on and reduce delays and time to market for Phase 2.

Looking forward to hearing your opinions!

7 Likes

Indeed, we are targeting a pruned full node comprising a consensus and execution client as of now to be collocated to each IC replica on one subnet. An archival node would not be of much additional value for most of the user cases we see currently and would consume an excessive amount or resources of the IC machines.

Exactly, the idea is to use HTTP outcalls to localhost, which would then be directed to the respective collocated Ethereum full node that sits on the same physical machine as the IC replica.

The blockchain world is indeed increasingly becoming a X-chain world! There are dozens of chains, and a handful of chains you mention that are most interesting. Now the first question is: Can we do a native integration with all of them within a year’s time frame? Probably not. The second question is: Which of the EVM chains is the most important one? This is easily answered with “the Ethereum blockchain.” For this reason we are currently designing the Phase 2 for a native performant Ethereum integration, not requiring API keys to be managed securely. Our current design with having a subnet with collocated Ethereum full nodes that are accessed using HTTP outcalls to localhost. This is a trustless integration and does not need any additional trust assumptions besides the ones of the IC and Ethereum.

Ethereum, as a base chain to all the Layer-2 chains, secures all assets used in Ethereum and its Layer 2s. This means that an Ethereum integration allows for tapping into the liquidity of all those chains either directly or indirectly. Thus, my argument is that a native Ethreum integration is the best step we can do as a next step and provides tremendous value to the IC.

Coming back to an integration with the other EVM chains (and possibly other chains in the future): I’d love to have such integrations available, but this will take time. There are different ways on how such integrations can be done, some examples of approaches being presented next:

  • Native as for Bitcoin
    • Running a full node of the target chain on chain
    • Strongest trust model
    • Hard for chains that have more state than the Bitcoin network
  • Native as for Ethereum
    • Runs collocated full nodes of the target chain in one subnet
    • Strongest trust model (essentially like running a full node on chain)
    • Requires lots of resources
  • Using on-chain or collocated light clients (particularly the collocated option is interesting as it could reuse a lot of what is done now for Ethereum)
    • Light-client trust model, weaker than a native integration, but still good
    • Requires few resources to run
    • Implementation effort differs a lot based on concrete architecture
  • Using HTTPS outcalls to cloud nodes
    • Weaker trust model: Trusting a quorum of cloud nodes
    • Easiest to realize,; would benefit from additional protection of the API keys
    • Good approach for larger number of integrations with different, but not for the key integrations with the major chains

The Phase 2 of the Ethereum integration does not need API keys, but other integrations built on HTTPS outcalls do. And for those we should definitely think about an approach to allow different API keys to be used for different nodes as you suggest. This would also immediately allow for observing if a node leaks and misuses an API key. That’s a discussion that should be have when considering integrations with further blockchains using the HTTPS outcalls feature following the Ethereum integration. We should, however, not think about this for Phase 1 of the Ethereum integration as this is a stop-gap solution towards Phase 2 that is obsoleted once Phase 2 launches.

Opinions?

5 Likes

Seems very reasonable to me.

2 Likes

Trying to integrate every blockchain (with a native integration at least) could get out of hand very quickly and become a large time-drainer, I think this would be a big mistake.

But perhaps tEDdsa combined with a phase 1 template would allow many integrations to flourish, while only the most important chains get native integrations (like Bitcoin and Ethereum). Objective criteria could be applied to determine which native integrations deserve to be created.

I fear this could easily run away and become something that wastes a lot of time while other important protocol tasks remain undone. Demergent Labs has a list of tasks we need from the protocol just as a few examples.

3 Likes

@lastmjs, I fully agree with you that only the most important integrations should be done natively, i.e., Bitcoin and Ethereum for now. The Phase-1 implementation can indeed help integrate any chain the API of which is available through one, ideally multiple, cloud API providers. Anyone can then use t-ECDSA and the Phase 1 integration to roll their own integration with any blockchain. I think this is a very realistic way forward for integrations with many more chains quickly.

One can think about few other important chains to receive thought about a native-style integration into the system, but it should be few of them, and only once Ethereum has bee integrated. Anyway, Ethereum captures probably most of the value by far already and is by a huge margin the most important chain to integrate with.

2 Likes

I think that it is important to integrate all EVM chains, including Ethereum. Ethereum may have the lion’s share of TVL, but for DeFI activity and trading this is not the case. Anyway, the API key attack needs to be addressed for both bridging and oracles, so it would be good if the Foundation could put resources on this, otherwise as soon as the Foundation gets any serious kind of TVL a bad actor will hack it.

Currently the only other chain I think is important enough for a native integration is Cosmos.

2 Likes

Here is a document summarizing the efforts planned for Phase 1: Ethereum Integration — Phase 1 - Google Docs

Have a nice weekend!

3 Likes