Direct Integration with Bitcoin

Roadmap
314

Very interesting ! Thank you for having let me know !

315

Itā€™s an honor to let you parti cipate in the reply. Thank you. As you know, the software here is not allowed. :sweat_smile:

316

In fact, he has told you the correct answer. The rocket is ready to take off.

3 Likes
317

If a subnet hosts BTC mainnet, does it become a BTC node?

318

Is an IC subnet more like a Bitcoin full node or a lightweight node? Storing the 350+ GB of blockchain data as a full node seems quite costly.

319

I think itā€™s dine compared to the data that the IC already storing, each canister has a 300gb limit and ic nodes are data centres.

320

Itā€™s somewhere in between. The IC, at least for now, will not be storing all 350+ GB of Bitcoin state. Instead, it uses these blocks to compute the UTXO set, and then discards them to save space. The UTXO set takes a relatively small amount of space (~15GiB).

We do store some of the recent blocks though to allow us to:

  1. Detect forks.
  2. Compute the UTXO set at different confirmations.
7 Likes
321

Short answer: No.

Long answer: It acts like a BTC node in many ways (and looks like one to the other BTC nodes) in that it connects to other BTC nodes, synchronizes with their view of the Bitcoin blockchain, sends transactions out to Bitcoin nodes, and validates blocks and transactions. However, it does not serve blocks to other nodes because we donā€™t keep the blocks, but only the extracted UTXOs for reasons of not storing 100s of GBs. Acting completely like a full node would not be the purpose it needs to serve.

4 Likes
323

Thank you for clarifying this.

324

@dieter.sommer I tried bitcoin-developer-preview but itā€™s Regtest network.
Could i get example to sync Bitcoin testnet from my local ? I donā€™t know how to use latest updates from github.com/dfinity/ic/tree/master/rs/bitcoin

Thank you so much !

325

First of all, @ktoanlba, welcome to the Developer Forum!

I donā€™t recommend you to try to sync Bitcoin testnet locally because it might take days but if you are really interested, here is how:

If you use the Manual Setup:

  1. use testnet=1 instead of regtest=1 in bitcoin.conf as in Setting up a local Bitcoin network
  2. use bitcoin/adapter/tests/sample/testnet.config.json instead of bitcoin/adapter/tests/sample/regtest.config.json as in Running the IC-Bitcoin Adapter

If you use the Docker Setup:

  1. use testnet=1 instead of regtest=1 in docker/bitcoind/conf/bitcoin.conf
  2. use testnet instead of regtest in docker/adapter/regtest.json

If you use testnet instead of regtest, make sure to change Regtest to Testnet in:

We are working on making the bitcoin-developer-preview working with the latest updates from github.com/dfinity/ic/tree/master/rs/bitcoin.

5 Likes
326

Update:
One subnet on IC mainnet started syncing Bitcoin mainnet on Friday!

25 Likes
328

All the team deserve a drink or two! Dfinity Prof of Work :wink:. I will drink a beer for this! Eva and the team must start working on promoting now, i hope will do a great job.

1 Like
329

Is there a link to see the syncing progress? :sweat_smile::sweat_smile::sweat_smile:

330

First the work, then the party! :wink:

What we have seen so far from our experiments with Bitcoin testnet on IC mainnet is that we still need to improve performance, both for syncing blocks as well as for querying UTXOs. Some of those potential improvements have been clear upfront already, e.g., an efficient processing of the UTXOs in the not-yet-finalized blocks. It takes quite some time to traverse those blocks (currently 6) for every UTXO query.
Once we have a better understanding of the performance of query calls, we can decide whether to open up the API for Bitcoin testnet soon or whether we want to deploy some performance fixes before opening it up.

The link to the graphs showing the synchronized Bitcoin block height does not work publicly AFAIK. You can have a look at the subnet that syncs Bitcoin mainnet in the Dashboard here: Subnet: fuqsr-in2lc-zbcjj-ydmcw-pzq7h-4xm2z-pto4i-dcyee-5z4rz-x63ji-nae - ICP Dashboard
It shows that finalization rate is going down due to ingesting blocks, via this you can see that syncing has started on Friday afternoon CEST.

As you can see, there has been lots of progress with this feature recently, but also there are still some things to address, e.g., performance.

5 Likes
331

Typically i am a big fan of opening up APIs for testing; so that we can also learn from real life use of these APIs. This is especially true when the APIā€™s emphasized concern is performance (& not correctness or coverage). This is, of course, assuming that ā€œwith all hands on deckā€ you have had a few weeks of internal testing and fixing.

The main reason to NOT open up API for testing assuming the overall correctness and coverage would be that the performance is so attrocious that it is unusable.

Curious to know if this is the overall thinking or anything else that comes to bear on this.

3 Likes
332

We also want to make the API available to the public as soon as reasonably possible. This may happen shortly depending on our assessment.

13 Likes
334

any update? syncing finished?

3 Likes
335

Progress update

We are nearing completion of the feature for a Beta launch. I would assess that we are currently between 90% and 95% complete for the Chromium / Beta launch.

The plan for the release of the Bitcoin integration feature (incl. threshold ECDSA) is as follows:

  • Chromium (Beta) release in some weeks, target is late June: functionally complete implementation
    • API for Bitcoin testnet Integration available on IC mainnet for public consumption
    • Threshold ECDSA deployment with a test key that will be deleted after a Beta phase available on IC mainnet for public consumption
    • Documentation, videos, and a sample dApp on how to use the feature
  • GA (general availability) release targeted at ~1-2+ months after Chromium: production-ready implementation
    • API for Bitcoin mainnet Integration available on IC mainnet for public consumption
    • Threshold ECDSA deployment with the production key that will be hosted on two large subnets on IC mainnet for public consumption (NNS as key backup subnet, another large subnet as signing subnet)
    • ckBTC canister

The Chromium (Beta) release will allow the community to build canisters based on the Bitcoin and threshold ECDSA APIs, i.e., create fully-functional integrations with Bitcoin. The GA release provides fixes towards production readiness, e.g., in the areas of performance, and enables Bitcoin mainnet and the final production threshold ECDSA key.

We are now very close to the Chromium / Beta release. Next, you find the items that are still missing for the Bitcoin part (threshold ECDSA not considered here). Many of those are small items and/or already progressed quite a bit in their implementation.

  • Finalizing the API for sending transactions
  • Implementing the API for transaction fees (get_current_fee_percentiles)
  • Finalizing pagination support in get_utxos
  • Parameterizing the cycles accounting for the feature
  • Improving checkpointing performance (optional for Beta)
  • Hard cap Bitcoin payloads to 4MB
  • Improving finalization rate behaviour under heavy load of Bitcoin requests
  • Make DoSing the Bitcoin subnet more difficult
  • Updating the replica version to be shipped for the SDK release
  • Writing a documentation Web page
  • Recording tutorial videos

Towards the GA release, these are the major items to be fixed:

  • Benchmarking
  • Further performance improvements
  • Security enhancement: Allowing the detection of very long forks
  • Security enhancement: Move to larger subnet (potential performance implications to be assessed)
  • All required integration tests completed
  • Code cleanup

Hope this gives you an impression of where we stand and what we still need to work on.

25 Likes
336

Will the stable data structures that were developed for the BTC integration also be released as part of Beta or GA? I suspect they may come in very handy for canister developers.

3 Likes