I would recommend if it’s too much extra work (and potentially a security/scalability issue to add the testnet into the IC) then just skip it and get the spec for the main Bitcoin adapter out, sure it’s a bit expensive to test but there’s workarounds, and a well funded grants program .
The IDL is really the main thing devs need to start developing apps with the functionality mocked out prior to threshold ECDSA signatures. So rather than the testnet adapter, I think integration with the IC protocol stack should be Dfinity’s top priority right now.
The full feature requires threshold ECDSA to be finalized. January will be tight for this, so we will likely see a developer preview in late December or January so people can already start building smart contracts using the Bitcoin functionality. The full release is expected later in Q1/2022.
You are definitely thinking into the right direction! We are currently evaluating what it takes to integrate with the testnet and do it if it’s a reasonable effort. Efforts spent on testing support for customers pay off in the long run in my experience. If it’s too much, we’ll need to see whether we launch a developer preview without it and let people play on Bitcoin mainnet.
The IDL is really the main thing devs need to start developing apps with the functionality mocked out prior to threshold ECDSA signatures. So rather than the testnet adapter, I think integration with the IC protocol stack should be Dfinity’s top priority right now.
The goal is to release the developer preview with the final or very-close-to-final interface so that people can build on it. The integration with the protocol stack to reach this goal has very high priority. I guess that’s what you are mainly interested here, right?
We have thought again what the quickest path is to bring the Bitcoin feature to our community so that smart contract canisters can be implemented against its API. The outcome was that our immediate focus should be on releasing the Bitcoin feature as part of the local dfx development environment in January. The local replica will only connect to a locally-running bitcoind instead of the Bitcoin mainnet or testnet. According to current thinking we release it as a beta version of dfx that contains everything the latest dfx version contains as well as the Bitcoin feature.
This setup will allow community members to start implementing their smart contracts using Bitcoin against the IC Bitcoin API. We think this is the fastest path forward to bring something to our valued community so you can start working with it. Clearly, threshold ECDSA will not be available and be replaced by either our mock API or just an ECDSA library running in the canister. This can easily be swapped out with the real threshold ECDSA API once it is available.
The biggest remaining chunk of work to obtain this MVP is the IC stack integration on which we are focussing right now. This is cutting across the whole IC stack and has a couple of remaining issues open. There’s also further small work items still to be done, which are not interesting enough to talk about them here. We think it is realistic to get all of this done in Januray and ship this MVP in late January, unless we hit some unexpected roadblocks.
This approach spares us lots of effort we would need to do for an IC mainnet launch of the feature, thus we can launch it earlier than we would be able to with the feature running on mainnet. This will be the next step to be worked on, of course.
Besides the above, we have made quite good progress in terms of the testing plan and figuring our how to do test the feature itself reasonable and provide something canisters authors can use for their own tests. More about this in another post.
We are interested in what you think about this modified fast-path approach to a first MVP release of the feature.
How much Bitcoin logic (e.g. parsing utxo and transaction data, creating transactions) is expected to be handled by the canisters? I assume (given that we rightfully keep the system APIs primitive if possible) a significant bit…
So are there concrete plans (and resources) to provide Motoko libraries? Or should developers expect to use Rust if they want to use it as soon as the feature goes live?
These are excellent questions, let me try to answer them.
The Bitcoin API will provide means for querying the UTXO set for specific addresses and for submitting transactions. Then there will be the threshold ECDSA API with methods for obtaining the canister’s public key and signing.
That is, there is indeed some processing to be performed by the canister (e.g., through a library).
We want to offer a library for both Rust and Motoko. Rust is easier as we can build upon a large body of open source libraries. Motoko is more challenging as we need to implement quite some huge chunks of logic ourselves. For this reason we have been thinking of asking the community for help with the Motoko library. That has been on my plate for some time now and your question is an excellent trigger for this.
Some of the larger items we need for the library are the following:
SHA-512: There is a SHA256 implementation that can be used as a basis, this should be quick for someone who knows this well
RIPEMD-160 (for address computation)
Elliptic curve scalar multiplication on the secp256k1 curve (for BIP-32 key derivation)
Bitcoin address derivation
BIP-32 unhardened derivation function
Offering high-level API based on the system APIs
Some of those items are required for computing Bitcoin addresses from public keys and for BIP-32 key derivation. The main problem is that Motoko currently has no FFI to reuse those things from implementations in different languages.
The question now is whether anybody here on this subforum would be interested in helping with those items towards obtaining a Motoko library. If you are interested, please just post here. We would be very happy to work with you on this and provide all the information required to get this done.
Naive question: would implementing a FFI for Motoko be more or less work than individually implementing all of these libraries? I would assume that Motoko compiling down to wasm makes implementing FFI a little bit easier.
I ask because while some of these libraries are more generally useful (e.g. SHA-512), I don’t know if most of them will ever be needed by most application developers.
It is relatively straight forward to embed extra libraries (C or Rust) in the RTS for all canisters. We do that already for bignum arithmetic, for example. This is a shortcut, but it doesn’t scale really, because the set of libraries would still be fixed by the compiler.
Allowing library author to include such code is a hard problem that we shied away from so far: what’s the syntax? How do the type systems interact? How unsafe do we want this to be? How to solve the engineering issues of multi-language builds?
Also it seems that to achieve some of the goals we’d have to wait for newer Wasm features (multi-memory) to land. Given that we still don’t even have simple extensions like multi-value on the IC, this puts this likely into the far future.
Joachim, thanks for the detailed response on this subject. Indeed, we will not see FFI for Motoko any time soon for those reasons. This is somewhat a challenge when it comes to features such as Bitcoin which require a breadth of functionality available for Motoko – we will need to write those functions from scratch. On the positive side, this will help provide a richer library ecosystem for Motoko which we would want anyway in the future.
Please, if you like the Bitcoin Integration idea, what do you think about Bitcoin fork integration to ICP? I mean, DOGE, BCH, DASH… More useful and quick work, if you are interested by this idea, please we can talk about it here : Integrate all Bitcoin FORKs on ICP [BCH, DOGE, DASH, etc...]
Interesting thoughts! We have thought about further integrations as well as part of the motion proposal on “General Integration” (Long Term R&D: General Integration (Proposal) - #9 by dieter.sommer) as this is important to bring other cryptocurrencies into the IC for use cases such as decentralized exchanges. The question is, how exactly to bring those cryptocurrencies over. There are multiple ways of doing so, and we should discuss what the community feels should be the way to go:
Bridging ERC-20 coins that already exist in Ethereum: weak trust model as we rely on bridges, but readily available soon (e.g., through the Terabethia bridge)
Bridging individual coins: trust assumptions of a bridge; can be done by the community
Relay-based integration: Integrating a cryptocurrency via a relay-based integration, where reliance on the externally-operated relay is mainly for availability. Using threshold ECDSA ensures a still rather strong trust model as no party holds the signing key for transferring cryptocurrency coins; this would not require extensions to the IC protocol stack and could thus be done by the community, based on a blueprint and maybe reference implementation by the Foundation
Direct integration: integration a la Bitcoin; strongest possible trust model; can only be done by the Foundation; this would only be feasible in case we can reuse the Bitcoin implementation more or less completely and parameterize it for each currency and enable it on given subnets (does anyone here have experience how similar / different the currencies interesting to us are to Bitcoin)
The above is early thinking, but can be a trigger for some broader discussions about what people expect to have.
We are progressing well towards our goal of a Developer Preview at the end of January. The plan is to release the Bitcoin functionality first as part of the Canister SDK (dfx environment). We are currently working on the integration of the code with the IC stack, where much of the code is available already. The Canister SDK will likely be shipped in a beta version with Bitcoin support enabled.
We want to provide, as part of the release, smart contract developers a reasonable means of testing their smart contracts. We settled on providing only support for the locally-running bitcoind initially, which gives our users a flexible means of testing their code during initial development. The Bitcoin IC mainnet release later in Q1 / 2022 will allow for integration with the Bitcoin testnet and mainnet to allow for further testing of smart contracts before a production release.
Good question!
We have a direct integration of the IC with Ethereum on our roadmap. This would allow for bringing the ERC-20 tokens of Ethereum into the Internet Computer. However, the ERC-20 tokens on Ethereum which are wrapped tokens of Bitcoin-like cryptocurrencies are all bridged to Ethereum, i.e., require the additional trust assumptions of the bridge. A native integration would be able to go without those additional trust assumptions, i.e., be much more secure. Also, transferring ERC-20 tokens from Ethereum to the IC using the direct integration would incur Ethereum gas costs, which would be avoided with a direct integration of those other cryptocurrencies.
Hey Timo, I’m hoping the foundation could use lightning to make it cheap and fast to move BTC to the IC. Some may not want to wait a long time for BTC confirmations, especially if lightning grows in popularity; indeed waiting an hour may not be ideal for certain DeFi applications. Say you want to move BTC into the IC to partake in a long or short position, and need to make the trade near real-time.
I see. So you want a lightning channel to be established between a user and a canister on the IC. Or, alternatively, between one of the big lightning nodes that offer routing and a canister and then the user can indirectly route Btc to the canister.
First of all I would love to see this implemented just because it is cool. But we also have to think if it really gives you the benefits that you think it does. The user already has to have the Btc locked in the lightning channel in advance before the need for a real-time trade occurs. Then the user might just as well lock the Btc in the canister-DEX directly. With centralized exchanges you want to take funds off the exchange as quick as possible to reduce risk but with a DEX in a canister the situation is different. So the need for a lightning channel may be reduced.
With lightning in general the problem is always the other direction, taking funds out, because the exchange cannot keep channels with all its users that are all funded on the exchange side.
Technically I think there are still some open question if this can really be done in both direction. There is some problem with the way that HTLCs work. The IC can do threshold signing but cannot create hashed secrets in a distributed way that are later to be revealed. This has to be investigated in detail. Maybe there are future lightning version that use a different hash function who’s computation can be distributed. Or there might be some hybrid solution that runs almost but not entirely on the IC. That’s TBD.
.
If it’s too much, we’ll need to see whether we launch a developer preview without it and let people play on Bitcoin mainnet.