I was able to reproduce this. The problem is that the current version of the ic-starter doesn’t overwrite the subnet configuration if it already exists. My understanding is that this is in the process of being fixed.
What this means is if you run dfx start or dfx replica, without enabling bitcoin integration, and then try to run dfx start --enable-bitcoin or dfx replica --enable-bitcoin, the subnet won’t have the bitcoin feature enabled.
The workaround is either dfx start --enable-bitcoin --clean, or to remove the .dfx/state and .dfx/local directories first.
I may have lost 0.01277662 BTC, worth about 300 u.
This is entirely my own fault and has nothing to do with the security of the ECDSA and Bitcoin interface.
Here is the details:
A few days ago, I deployed canister to the mainnet and tested the BTC on the testnet, everything works fine!
So why not test the mainnet BTC? Although I saw the announcement saying not to use valuable BTC, I thought this is mainly used to explain that the key of ecdsa may be changed when it is officially released in the future, but it does not matter, because I will transfer the BTC out immediately.
So I ran:
dfx canister --network ic install basic_bitcoin --argument='(variant {Mainnet})' -m=reinstall
dfx canister --network ic call basic_bitcoin get_p2pkh_address
("1T6xcKM23GjYNNeZXrQrctSEFAKcfmGq2")
Everything looked good, and I could get the address of the BTC mainnet correctly. It seemed everything is normal, so I withdrew a small amount of BTC from the exchange to this address. I knew the risks, so I limited the possible losses to what I can afford. In the boring waiting, (compared to IC, BTC is really slow), I tried to randomly obtain an address with a balance from the BTC explorer, to test whether the balance can be obtained correctly:
dfx canister --network ic call basic_bitcoin get_balance '("<btc address>")'
Error: Failed update call.
Caused by: Failed update call.
The Replica returned an error: code 5, message: "Canister zso2k-nqaaa-aaaak-aao3q-cai trapped explicitly: Panicked at 'called `Result::unwrap()` on an `Err` value: (CanisterReject, "Received request for mainnet but the subnet supports testnet")', src/bitcoin_api.rs:31:17"
I realized that something was wrong, so I quickly went to check the withdrawal interface, the 2min cancellation period has passed, the transaction has been sent to the node’s memory pool, and I want to consume the utxo by sending another transaction with a higher fee price again to cancel the previous transaction, but the withdrawal is through the exchange, and it may time out waiting for customer service.
Since the interface of ecdsa is not banned (getting address requires getting public key from ecdsa), then theoretically I should be able to withdraw from this address.
The interface get_current_fee_percentiles to get fee_price (fee per byte) is not available, I deleted it, and then checked some recent transactions from the btc explorer, most of them are below 100, so setting it to 110.
Then, deleting the get_utxos, input the UTXOs as a parameter. The height and value in Utxo are easy to find from the BTC explorer, and the txid in OutPoint generated from tx_hash:
// the hash is from: https://www.blockchain.com/btc/tx/e1d3f2afe6ef975a1c2675cac813d4fe1c981e6beeb3edec1ad0d872996edb21
let hash = bitcoin_hashes::sha256d::Hash::from_str(
"e1d3f2afe6ef975a1c2675cac813d4fe1c981e6beeb3edec1ad0d872996edb21",
)
.unwrap();
let txid = bitcoin::hash_types::Txid::from_hash(hash);
let txid_vec = txid.to_vec();
vout may be the index of utxo under this address, because there is only one transaction, so I set it to 0;
send_transaction doesn’t work either, so I return the signed transaction bytes.
Finally achieved this effect:
dfx canister --network ic call basic_bitcoin tx '(record {destination_address= "3QnYGgtC1hNrSRBHdUnDn7EBZ3R8fwezVh"; amount_in_satoshi=1247662; }, record {height=747887;value=1277662; outpoint=record {txid=vec{33;219;110;153;114;216;208;26;236;237;179;238;107;30;152;28;254;212;19;200;202;117;38;28;90;151;239;230;175;242;211;225;};vout=0;}})'
(
"020000000121db6e9972d8d01aecedb3ee6b1e981cfed413c8ca75261c5a97efe6aff2d3e1000000006b483045022100e2644f76115c826a73488967e11390a444f0d673686b1f4e00b1b16a9ebb1e670220131e74ffb8cbf116446c84c99c805b0c1b3824d453a6a85d3ffcd60a78161d6c012103dcdb13eda6c212e807626e6e72ef8aca901a4c0f31f1bf7036803e3756240b96ffffffff02ae0913000000000017a914fd564e358f1f3f7f88990244ed1b5293946964558718750000000000001976a91404efca051ff7d7d8ba81f7dcc5c783c7381566b188ac00000000",
blob "\02\00\00\00\01!\dbn\99r\d8\d0\1a\ec\ed\b3\eek\1e\98\1c\fe\d4\13\c8\cau&\1cZ\97\ef\e6\af\f2\d3\e1\00\00\00\00kH0E\02!\00\e2dOv\11\5c\82jsH\89g\e1\13\90\a4D\f0\d6shk\1fN\00\b1\b1j\9e\bb\1eg\02 \13\1et\ff\b8\cb\f1\16Dl\84\c9\9c\80[\0c\1b8$\d4S\a6\a8]?\fc\d6\0ax\16\1dl\01!\03\dc\db\13\ed\a6\c2\12\e8\07bnnr\ef\8a\ca\90\1aL\0f1\f1\bfp6\80>7V$\0b\96\ff\ff\ff\ff\02\ae\09\13\00\00\00\00\00\17\a9\14\fdVN5\8f\1f?\7f\88\99\02D\ed\1bR\93\94idU\87\18u\00\00\00\00\00\00\19v\a9\14\04\ef\ca\05\1f\f7\d7\d8\ba\81\f7\dc\c5\c7\83\c78\15f\b1\88\ac\00\00\00\00",
)
curl -H "Authorization: Bearer <API TOKEN>" -d '{"tx": "020000000121db6e9972d8d01aecedb3ee6b1e981cfed413c8ca75261c5a97efe6aff2d3e1000000006b483045022100f79de358f564aed17cb8fd7cf95d79aee65218698bcd1cf037e136605029f93702203dba9fed10b25da52d6db6bd643631af832442e87051eea8f19a0f6e8d72b2f9012103dcdb13eda6c212e807626e6e72ef8aca901a4c0f31f1bf7036803e3756240b96ffffffff02ae0913000000000017a914fd564e358f1f3f7f88990244ed1b5293946964558718750000000000001976a91404efca051ff7d7d8ba81f7dcc5c783c7381566b188ac00000000"}' -X POST https://ubiquity.api.blockdaemon.com/v2/bitcoin/mainnet/tx/send
{"type":"bad-request","code":16397,"title":"Bad Request","status":400,"detail":"failed to send transaction. Error: bad-txns-inputs-missingorspent. Code: -25"}%
# If I changed to vout to 1, and got another error, it seems vout = 0 is correct:
{"type":"bad-request","code":16397,"title":"Bad Request","status":400,"detail":"failed to send transaction. Error: mandatory-script-verify-flag-failed (Script evaluated without error but finished with a false/empty top stack element). Code: -26"}%
I have added permission control to interfaces that needs to be signed with an ECDSA key.
the code of tx interface
pub async fn tx(
network: Network,
derivation_path: Vec<Vec<u8>>,
key_name: String,
dst_address: String,
amount: Satoshi,
utxos: Vec<Utxo>,
) -> Vec<u8> {
// Get fee percentiles from previous transactions to estimate our own fee.
let fee_per_byte = 110;
// Fetch our public key, P2PKH address, and UTXOs.
let own_public_key =
ecdsa_api::ecdsa_public_key(key_name.clone(), derivation_path.clone()).await;
let own_address = public_key_to_p2pkh_address(network, &own_public_key);
print("Fetching UTXOs...");
let own_utxos = utxos;
let own_address = Address::from_str(&own_address).unwrap();
let dst_address = Address::from_str(&dst_address).unwrap();
// Build the transaction that sends `amount` to the destination address.
let transaction = build_transaction(
&own_public_key,
&own_address,
&own_utxos,
&dst_address,
amount,
fee_per_byte,
)
.await;
// Sign the transaction.
let signed_transaction = sign_transaction(
&own_public_key,
&own_address,
transaction,
key_name,
derivation_path,
ecdsa_api::sign_with_ecdsa,
)
.await;
let signed_transaction_bytes = signed_transaction.serialize();
signed_transaction_bytes
}
Finally, still failed.
Is there any problem with the above steps?
It has been determined that the ECDSA key will be modified before the application subnet supports the BTC mainnet interface, so waiting for the development of the BTC interface will not solve the problem, is there any other solution?
The ICP BTC integration only supports BTC testnet at the moment. So you cannot use the bitcoin API to send transaction or receive utxos or check balances of anything on the BTC mainnet.
Since you have already sent some bitcoin to the address, you will have to manually craft a raw transaction, and send it via other means (e.g. bitcoin-cli command line) to the BTC mainnet to get the fund out. You will have to use your canister to sign on the raw transaction, which uses threshold ECDSA API, which is not related to and thus not affected by the bitcoin API.
So your fund is not lost, and retrieve it definitely can be done, just that you can’t use existing bitcoin API to send the transaction. Hope it helps!
This doesn’t seem to be the reason why your transaction got rejected but please note that as far as I know your fee_per_byte in your tx interface is below the relay fee as fee_per_byte is in millisatoshis per byte. Maybe you wanted to type 1100 instead of 110 ?
Thanks for the reminder, I ended up setting this value to 3000.
More importantly, you guys gave me confidence, and let me know it’s going to work right away. So I searched for error: bad-txns-inputs-missingorspent and found the specific meaning of vout from here: In a certain tx, there may be multiple utxos output, and these utxos have an index, which is vout. The vout of the utxo used above should be 2, after changing that, The transaction is sucessfully!
Like @spnrapp, are you looking for transaction information only about tx_hash that you created and broadcasted using your canister or you are looking for any arbitrary transaction information for a given tx_hash not necessarily created and broadcasted by your canister ?
If you are looking for transaction information for an arbitrary tx_hash, a DFINITY staff may give current DFINITY point of view. Anyway using the HTTP request feature you can use a website like blockchain.com to retrieve the transaction information associated to a given tx_hash.
I am not very keen on the HTTP request feature but using the documentation and some details I shared on StackOverflow, you should be able to code something doing the trick.
Not sure you saw but last week we released the testnet, sample code and documentation so devs are building are now building smart contracts with BTC on ICP.
I think you don’t understand me. The btc integration release date was in June (you did not mean the test-net release at this date)
It is been about 3 months then you released the btc integration test-net.
The manipulation is, the mainnet release date was at June. Now in August you did not give any (detailed) information about the process including mainnet release date.
When we ask the update, you were delayed the release date.
If something went wrong at the backside for processing the btc-integration we should know.
On the other hand dfinity team has given a public information the btc and ethereum integrations should have processed at the end of 2022.
With this information how is it possible to release the integrations at the end of this year?
The btc integration still in process since approximately 1 year.
I am not arguing with you but you hidden my previous post. It was a rude behaviour.
Kind regards
I don’t mean to be defensive, especially on their behalf, but why are you being so impatient with such an incredibly complex and significant feature? They’re working incredibly hard to make sure this feature is built the right way and secure. The dates they have given in the past were never concrete.
It’s one thing to ask for progress updates every once in a while, but many people who frequently ask (seeming impatient) the same question over and over again don’t seem to have the best intentions for the network as a whole. I’d assume you’re just speculating that the release will boost the price of ICP.
Let the team do their work without constantly feeling as though they need to feed updates to those who are impatient and unappreciative of their amazing contribution to the network.
To be honest I am not impatient person. I believe the team are working too much to develop this feature. I am supporting the project since the genesis.
I did not mention anything about the price but I bought ICP when it was about 200-300 Euros at coinbase.
Then I still believe the price will be restored next days but my faith is dwindling day by day because of the community.
However, I am following other projects as well. They are doing community calls they giving an informations about the project and updates.
For instance ethereum community are giving a lot of informations about the process of merge. They sharing what are they doing at the background.
I mean I just wanted to ask why it is delayed too much? Which kind of thinks went wrong? Am I wrong?
Also this is the forum page where can people discuss each other. I am here to discuss not arguing but my previous comment was hidden without any comment.
I didn’t expect to write these things but that was a rude behaviour.
Kind regards
I see that your post was innocuous and should not have been flagged. Your question is legitimate.
However for all of us who are following the btc integration, updates are practically everywhere. It is true that in this post the last update was 27d ago.
On the “complete” ETH integration, we are not going to make it this year, i believe. That said , you might want to look up @lastmjs 's tweet (or a post here, i cant remember) on what can be done with ETH after a couple of features being released on mainnet.
Tbh, I think there is a misunderstanding, but maybe not the original one I thought.
The BTC integration team gives regular updates on the progress. The testnet going live is such an update. So when you ask, “any updates in the progress?” its reasonable to say “lots of progress and lots of updates.” If BTC project were a train from Paris to Madrid, there are constant updates like “reached pamplona station”, “reached Zaragoza station,” etc… updates like this imply the listener knows how close Zaragoza is to Madrid (which may not be the case).
I suspect (based on your follow up post) your intent is not “progress on the project”, but rather: “what is the latest projected mainnet release? How close are we to mainnet release?” that’s a reasonable question!
I also suspect that you are asking on the dev forum because although updates are common on dev forum and Reddit (I have an update few days ago for example), you are referring to more vocal updates (from Dfinity Twitter for example).
The best answer for this is divided into two components.
On the Threshold ECDSA side (necessary feature for BTC integration): time + tests. Just want to make sure we test and test to make sure there are no issues or bugs (if there are, we will of course fix them). But no major functional pieces left.
On the BTC side: in addition to testing, there are a number of improvements that needed security-wise and performance-wise for to be ready for GA. This also highly depends on what bugs or security issues people find in testing (this needs to be bulletproof). There may be bugs we are unaware of. “Weeks” would be optimistic (given the pace the last few months), so somewhere between weeks and months for GA.
Does that help?
PS not sure why your original comments was hidden (it was certainly not me).
There are many updates on the projects in this thread (possible they are not helpful), but I think the helpful answer is that things took longer to build and test right than expected since no one has ever built such an integration. As they designed a whole new crypto protocol (tecdsa), built and tested they found things along the way which they did not expect when they started.
it may be helpful for future folks to have a list showing all the surprises or unexpected things between planning and implementation, but i don’t have one available.
More importantly… my current intent is to satisfy YOU (and folks like you). What kind of update or level of information would be helpful? I am aware in some areas my answers are too high level and too low level in others, so not the right level to be helpful.
thank you for quick response.
I prefer to use developer forum to follow updates on the IC platform.I do not use reddit usually.
Your response is very helpful. Thank you so much.
However I wonder something. What did you mean exactly about the bugs and security issues on the btc integration? I mean the bridge platforms are hacked to steal funds from the platforms. That’s why the btc integration is important.
What if I find the exploit or bug in this feature what can I do with that? What are the possibilities?
