Direct Integration with Bitcoin

Severin · June 24, 2022, 12:02pm

Good point, I’m not actually sure about that. I’ll let @dieter.sommer answer that as I’m now too far in uncertain territory

dieter.sommer · June 27, 2022, 11:55am

This also reflects my knowledge about this. There have been discussions on node rotation, but no concrete design for it yet.

In every system you have a residual risk of a secret key getting lost. One can only reduce it to an as-low level as reasonably possible. With our approach of having the key secret shared on two large subnets, I think we have pretty good resilience against disasters.

dieter.sommer · June 30, 2022, 9:19am

Update

The Bitcoin feature is code complete from a replica perspective for the upcoming Beta / Chromium release. We are currently working on finishing an example dApp to showcase the feature and the feature documentation.

Threshold ECSDA, a crucial dependency for the Bitcoin integration, is about to be completed for the Beta / Chromium release as well.

Things are nearing completion, we will keep you posted on progress!

Stay tuned!

Edit: Just to clarify things, with “Bitcoin feature” above I meant the technical integration of the IC stack with the Bitcoin network, i.e., ingesting Bitcoin blocks, validating them, processing the UTXOs contained therein, and serving UTXOs to canisters as well as submitting Bitcoin transactions to the Bitcoin network. Threshold ECDSAhas been considered as a separate feature in that post that is also very close to being completed.
Sometimes, when we speak of “Bitcoin integration” or the Bitcoin feature" in public, we refer to the union of this technical integration of the IC stack with Bitcoin and threshold ECDSA. Just to clarify so that everyone understands the same thing here.

ielashi · June 30, 2022, 12:11pm

And to add to @dieter.sommer’s announcement, here’s an updated demo that showcases the various Bitcoin APIs.

The upgraded demo supports the following endpoints:

get_balance to get the balance of a Bitcoin address.
get_utxos to get the UTXOs of a Bitcoin address, including pagination support for addresses that have a very large number of UTXOs.
get_current_fee_percentiles to get the 100 fee percentiles over the last 10k transactions for fee estimates.
send_transaction for sending signed transactions to the bitcoin network.

This demo is a very bare-bones demo to demonstrate the raw API. A more fledged demo that showcases how canisters can generate bitcoin addresses and sign transactions will, as @dieter.sommer already mentioned, be released imminently,

Fatalerror · July 4, 2022, 8:37am

Any news about the date ?

spnrapp · July 12, 2022, 6:58pm

Is it possible to get the status of transactions given their txid? We find it quite limiting that only “unspent” transaction outputs can be queried. It is equally important to know if some previous outgoing transactions from a canister are indeed included in the blockchain (whether they are spent or unspent is irrelevant, but if they are later spent by the receiver, they can no longer be queried through get_utxos).

Benjamin_Loison · July 12, 2022, 8:16pm

The current management canister endpoints related to Bitcoin don’t make it possible to get for sure the status of any arbitrary transaction given their transaction ids (more precisely those that your canister haven’t sent). However when using the bitcoin_send_transaction endpoint, it means that you know the transaction (UTXOs used etc… and thus the transaction id). So you are able

to know [for sure] if some previous outgoing transactions from [your] canister are indeed included in the blockchain

Indeed by using bitcoin_get_utxos you can retrieve the UTXOs that you still own according to the provided number of confirmations, after having sent your transaction that will consume some of them once it got included in the blockchain.
As UTXOs are uniquely identified by their transaction id and index of the transaction output, because your canister is the only entity being able to sign transactions using the UTXOs held by your canister and because Bitcoin transactions are atomic, if none of the inputs were used in another broadcasted transaction by using bitcoin_send_transaction and if you haven’t updated your transaction by using BIP-125 Replace-by-fee, then if you still hold any UTXOs spent, according to a given number of confirmations, by the just sent transaction, it means that the transaction isn’t yet included in the blockchain, according to this number of confirmations.

So there isn’t any need to retrieve the UTXOs of any recipient address.

Example:

I. Your canister holds two UTXOs on the Bitcoin address maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (you can call bitcoin_get_utxos to retrieve this information):

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the to-be-used transaction output: 0
Value: 1 BTC
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the to-be-used transaction output: 0
Value: 2 BTC

II. Your canister builds the following transaction (of id 3333333333333333333333333333333333333333333333333333333333333333), which sends 2.5 BTC to mbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb and sends this transaction to the Bitcoin network by using the bitcoin_send_transaction endpoint:

Inputs:

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the transaction output: 0
Value: 1 BTC
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the transaction output: 0
Value: 2 BTC

Outputs:

Address: mbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
Value: 2.5 BTC
Address: maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Value: 0.49 BTC

III. If afterwards you call bitcoin_get_utxos, the result can either be:

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the transaction output: 0
Value: 1 BTC
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the transaction output: 0
Value: 2 BTC

or:

Transaction id: 3333333333333333333333333333333333333333333333333333333333333333
Index of the to-be-used transaction output: 1
Value: 0.49 BTC

According to the passed number of confirmations to bitcoin_get_utxos:

In the first case, it means that your transaction isn’t confirmed yet.
In the second case, it means that your transaction got confirmed.

spnrapp · July 12, 2022, 9:44pm

Great explanation! However, when the sender itself isn’t one of the receivers, it is out of luck. It could mean that we just always have to keep some dust in an address even if it is supposed to be of one time use. Not the end of world of course.

Benjamin_Loison · July 12, 2022, 11:05pm

However, when the sender itself isn’t one of the receivers, it is out of luck.

As far as I know, the case when the sender itself isn’t one of the receivers isn’t a problem.
In the previous example we haven’t used the last bitcoin_get_utxos call to look for the generated UTXO our canister received. We were looking for any input of transaction spending:

Just checking whether or not any of these two UTXOs:

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the transaction output: 0
Value: 1 BTC
or
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the transaction output: 0
Value: 2 BTC

were spent and thus not part of your canister’s UTXO set.

This sample pseudo-Motoko code may make it clearer:

type TransactionId = Text;

/// An unspent transaction output representation.
type Utxo = {
    transaction_id : TransactionId;
    transaction_output_number : Nat32;
};

/// A Bitcoin transaction representation.
class Transaction(...) {
    // These UTXOs will be used as the inputs.
    public let utxos : [Utxo] = // ...

    /// Returns the transaction as a byte array.
    public func toBytes() : [Nat8] {
        // ...
    };

    /// Returns the transaction id.
    public func id() : TransactionId {
        // ...
    };
};

/// Cache of pending transactions that get cleaned automatically by using `is_transaction_pending`.
let pending_transactions : HashMap.HashMap<TransactionId, Utxo> = HashMap.HashMap(0, Text.equal, Text.hash);

/// Sends a Bitcoin transaction to the Bitcoin network and adds it to the pending transactions cache.
func send_transaction() : async TransactionId {
    // ...
    // Assumes having built a new `transaction`.
    // "new" means that none of the inputs were used in another broadcasted transaction.
    let transaction_bytes : [Nat8] = transaction.toBytes();
    // Sends the Bitcoin transaction to the Bitcoin network.
    await management_canister_actor.bitcoin_send_transaction(transaction_bytes);
    // ...
    // Gets the id of the transaction.
    let transaction_id : TransactionId = transaction.id();
    // We only need to check later on if the first input got spent in order to know whether or not the whole transaction was confirmed.
    let first_input = transaction.utxos[0];
    // Adds "the sent Bitcoin transaction" to the pending transactions cache.
    pending_transactions.put(transaction_id, first_input);
    transaction_id
};

/// Returns true if the sent transaction is already included in the blockchain according to the provided `minimum_of_confirmations`, otherwise false.
/// This function assumes you passed a `transaction_id` that got obtained by using `send_transaction`.
func is_transaction_confirmed(transaction_id : TransactionId, minimum_of_confirmations : Nat32) : async Bool {
    let utxo_option : Option<Utxo> = pending_transactions.get(transaction_id);
    switch (utxo_option) {
        case (?utxo) {
            // We don't use the possibly generated UTXO back to the Bitcoin canister address.
            // We are only looking for spending of pending spending UTXOs.
            let is_utxo_spent = await is_still_owning_utxo(utxo, minimum_of_confirmations);
            if(is_utxo_spent) {
                pending_transactions.delete(transaction_id);
            };
            is_utxo_spent
        };
        case null {
            true
        };
    }
};

/// Returns true if the canister is still holding the provided `utxo` at a depth of `minimum_of_confirmations`.
func is_still_owning_utxo(utxo : Utxo, minimum_of_confirmations : Nat32) : async Bool {
    let utxos = await management_canister_actor.bitcoin_get_utxos(bitcoin_canister_address, minimum_of_confirmations);
    Option.isSome(Array.find(utxos, func(haystack_utxo) = { haystack_utxo == utxo })))
};

Otherwise here is another example.

Example:

I. Your canister holds two UTXOs on the Bitcoin address maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (you can call bitcoin_get_utxos to retrieve this information):

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the to-be-used transaction output: 0
Value: 1 BTC
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the to-be-used transaction output: 0
Value: 2 BTC

II. Your canister builds the following transaction (of id 3333333333333333333333333333333333333333333333333333333333333333), which sends 2.9 BTC to mbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb and sends this transaction to the Bitcoin network by using the bitcoin_send_transaction endpoint:

Inputs:

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the transaction output: 0
Value: 1 BTC
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the transaction output: 0
Value: 2 BTC

Outputs:

Address: mbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
Value: 2.9 BTC

0.1 BTC goes to the miner.

III. If afterwards you call bitcoin_get_utxos, the result can either be:

Transaction id: 0000000000000000000000000000000000000000000000000000000000000000
Index of the transaction output: 0
Value: 1 BTC
Transaction id: 1111111111111111111111111111111111111111111111111111111111111111
Index of the transaction output: 0
Value: 2 BTC

or:

(nothing)

According to the passed number of confirmations to bitcoin_get_utxos:

In the first case, it means that your transaction isn’t confirmed yet.
In the second case, it means that your transaction got confirmed.

spnrapp · July 12, 2022, 11:48pm

That is not always the case. They may disappear from your canister’s UTXO set because the blockchain has re-organized.

ielashi · July 13, 2022, 8:46am

By the blockchain re-organizing, you mean a fork overtaking the current tip? As long as you’re spending UTXOs that have a sufficiently high number of confirmations, the probability of them being removed from the chain due to a fork falls to practically zero, no?

spnrapp · July 13, 2022, 3:37pm

That of course works, but is perhaps not the most efficient way of building on bitcoin chain. You should be able to continue building transactions/blocks on top of the “current longest”, and when that changes, you build on the “new longest”, without having to wait for for high number of confirmations. It could mean that a number of previous transactions you sent were not included in the “new longest”, but you should always deal with that anyway, they do not pose security risk in general (since double spending is not possible by construction).

spnrapp · July 13, 2022, 3:45pm

In any case, we are happy to accept that “make yourself one of the receivers if you want to know whether a transfer-out is included in the chain” as an answer for now.

But it would be good if a more general API that can query status of arbitrary txid can be provided in the future. Thanks!

ielashi · July 14, 2022, 12:59pm

Noted, thanks for your feedback.

Having an API to query any bitcoin transaction (spent and unspent) was ruled out initially because of the storage requirements. We’d need to store 100s of GiBs of additional state.

One idea would be to support this API for only “recent” transactions (e.g. transactions from the 24 hours). Another idea would be for the canister to use the HTTP API to fetch this information from some external API.

We can iterate on this and figure out the best step forward as we collect more feedback.

Tony89 · July 14, 2022, 6:55pm

hi, when will we have BTC integration beta? Where can I track it?

gatsby_esp · July 14, 2022, 7:04pm

Hi can you give us any update?

dieter.sommer · July 16, 2022, 4:40pm

Update

Here’s an update on where we stand with the Bitcoin feature (apologies for the few responses recently, it’s been too busy ):

The Bitcoin feature is code complete and we are currently finishing the documentation. On the threshold ECDSA front, we are very close to get something out to mainnet (actually, we have been out already briefly, but needed to address two issues and turn the feature off again, one in the implementation of message routing that routes the calls to the ECDSA functionality to the ECDSA-enabled signing subnet, and another one that is related to a technical issue that lead to a problem when upgrading the threshold ECDSA subnet). Hotfixes for both are out or on their way, so t-ECDSA should be on IC mainnet soon.

The release of Bitcoin integration and threshold ECDSA beta / Chromium with Bitcoin testnet integration and a test threshold ECDSA key is imminent. A release in the coming week or the one thereafter is a probable scenario if we don’t hit any further unexpected issues.

Tony89 · July 16, 2022, 4:57pm

Thanks a lot for the update, have a nice weekend @dieter.sommer

Ideaseed · July 17, 2022, 4:51am

Dieter,
My gut is telling me these snags are much larger. If you pull this off the company will get its wings.

Sal_Paradise · July 18, 2022, 3:04pm

Sorry, im not sure what you mean by this?