ckBTC minter Types and Motoko Language Server

I’m trying to resolve an issue with my dev environment in VS Code.

In dfx.json I have the following:

"ckbtc_minter": {
          "type": "custom",
          "candid": "target/ckbtc_minter.did",
          "wasm": "target/ckbtc_minter.wasm",
          "remote": {
              "id": {
                  "ic": "mqygn-kiaaa-aaaar-qaadq-cai"
              }
          }
      }

The target/ckbtc_minter.did has the following type:

type Utxo = record {
    outpoint : record { txid : vec nat8; vout : nat32 };
    value : nat64;
    height : nat32;
};

When I run dfx deploy ckbtc_minter both the constructor.did and the service.did that are produced and put in .dfx/local/ckbtc_minter have the following:

type Utxo = record {
  height : nat32;
  value : nat64;
  outpoint : record { txid : blob; vout : nat32 };
};

…and sure enough, through out my code I get the following errors:

expression of type
  Blob
cannot produce expected type
  [Nat8]

for

//details derive its type from CkBtcMinter.UtxoStatus (#Minted)
let _tnx_id: [Nat8] = details.utxo.outpoint.txid;

The dashboard has vec nat8 for this utxo type: https://dashboard.internetcomputer.org/canister/ml52i-qqaaa-aaaar-qaaba-cai (search for type utxo

And it is vec nat 8 here:

The reference in my file is: import CkBtcMinter "canister:ckbtc_minter";

I’m trying to figure out why dfx is converting the type from vec nat8 to blob. I think these are actually handled the same by the replica/moc, but the compiler doesn’t like it. It doesn’t make much sense that the language server and dfx build myCanister --check would be converting the vec nat8 in the did to blob.

This is dfx 0.24.0 but I get the same error with 0.27.0.

Where in the pipeline would it possibly covert this from vec nat8 to blob? Is it maybe pulling from somewhere on the IC? Even then, if the dashboard has it as a vec nat8, what would make it write blob into the .did file in .dfx/local/canisters? Is .dfx/local/canisters where it pulls things when you use the canister: syntax?

1 Like

That’s weird. In the past, I’ve seen dfx convert blob to vec nat8 when it decides to process a candid file that contains a service argument to extract the argument did and service did, where it normalizes blob to vec nat8.

But I haven’t seen it do the opposite: take [vec nat8] and emit that as blob.

Has there been a recent change to the candid tooling that could provoke this?

(related PR experiment: modify candid tooling to preserve source occurrences of blob types when parsing and pretty-printing candid sources by crusso · Pull Request #537 · dfinity/candid · GitHub (never merged))

I suspect the culprit is this line in the candid pretty printer.

    Vec(ref t) if matches!(t.as_ref(), Nat8) => str("blob"),

dfx is parsing the candid and pretty printing part of it to extract the service type and in doing so replacing vec nat8 by blob, which is causing moc to import the type as Blob.

This is the commit that introduced the line:

I’ve converted the code to use blob. Since we are reading from an external service here I want to make sure that I’m making a poor assumption here. It will come across the wire the same and when motoko gets it, it shouldn’t matter if I declare it as a blob or [Nat8], correct?

I have things compiling now this way and hoping it won’t barf once it comes across the wire.

That’s ok, but if we fix dfx to not replace vec nat8 by blob (since I don’t think dfx should be messing with our your did syntax that much) then your Motoko code might break next time you compile.

As an experiment could you (in a branch), replace the did by
this one below (that removes the service argument) and see if dfx still produces blob where you expect to see vec nat8? I think it just use the source did if it doesn’t specify a service argument, avoiding the rewrite.

// Represents an account on the ckBTC ledger.
type Account = record { owner : principal; subaccount : opt blob };

type CanisterStatusResponse = record {
  status : CanisterStatusType;
  memory_size : nat;
  cycles : nat;
  settings : DefiniteCanisterSettings;
  idle_cycles_burned_per_day : nat;
  module_hash : opt vec nat8;
  query_stats : QueryStats;
  reserved_cycles : nat;
};

type QueryStats = record {
  response_payload_bytes_total : nat;
  num_instructions_total : nat;
  num_calls_total : nat;
  request_payload_bytes_total : nat;
};

type CanisterStatusType = variant { stopped; stopping; running };

type DefiniteCanisterSettings = record {
  freezing_threshold : nat;
  controllers : vec principal;
  memory_allocation : nat;
  compute_allocation : nat;
  reserved_cycles_limit : nat;
  log_visibility: LogVisibility;
  wasm_memory_limit : nat;
};

type LogVisibility = variant {
    controllers;
    public;
    allowed_viewers : vec principal;
};

type RetrieveBtcArgs = record {
    // The address to which the ckBTC minter should deposit BTC.
    address : text;
    // The amount of ckBTC in Satoshis that the client wants to withdraw.
    amount : nat64;
};

type RetrieveBtcWithApprovalArgs = record {
    // The address to which the ckBTC minter should deposit BTC.
    address : text;
    // The amount of ckBTC in Satoshis that the client wants to withdraw.
    amount : nat64;
    // The subaccount to burn ckBTC from.
    from_subaccount : opt blob;
};

type RetrieveBtcError = variant {
    // The minter failed to parse the destination address.
    MalformedAddress : text;
    // The minter is already processing another retrieval request for the same
    // principal.
    AlreadyProcessing;
    // The withdrawal amount is too low.
    // The payload contains the minimal withdrawal amount.
    AmountTooLow : nat64;
    // The ckBTC balance of the withdrawal account is too low.
    InsufficientFunds : record { balance : nat64 };
    // The minter is overloaded, retry the request.
    // The payload contains a human-readable message explaining what caused the unavailability.
    TemporarilyUnavailable : text;
    // A generic error reserved for future extensions.
    GenericError : record { error_message : text; error_code : nat64 };
};

type RetrieveBtcWithApprovalError = variant {
    // The minter failed to parse the destination address.
    MalformedAddress : text;
    // The minter is already processing another retrieval request for the same
    // principal.
    AlreadyProcessing;
    // The withdrawal amount is too low.
    // The payload contains the minimal withdrawal amount.
    AmountTooLow : nat64;
    // The ckBTC balance of the withdrawal account is too low.
    InsufficientFunds : record { balance : nat64 };
    // The allowance given to the minter is too low.
    InsufficientAllowance : record { allowance : nat64 };
    // The minter is overloaded, retry the request.
    // The payload contains a human-readable message explaining what caused the unavailability.
    TemporarilyUnavailable : text;
    // A generic error reserved for future extensions.
    GenericError : record { error_message : text; error_code : nat64 };
};

type RetrieveBtcOk = record {
    // Returns the burn transaction index corresponding to the withdrawal.
    // You can use this index to query the withdrawal status.
    block_index : nat64
};

// The result of an [update_balance] call.
type UtxoStatus = variant {
    // The minter ignored this UTXO because UTXO's value is too small to pay
    // the check fees.
    ValueTooSmall : Utxo;
    // The Bitcoin checker considered this UTXO to be tainted.
    Tainted : Utxo;
    // The UTXO passed the Bitcoin check, but the minter failed to mint ckBTC
    // because the Ledger was unavailable. Retrying the [update_balance] call
    // should eventually advance the UTXO to the [Minted] state.
    Checked : Utxo;
    // The UTXO passed the Bitcoin check, and ckBTC has been minted.
    Minted : record {
        block_index : nat64;
        minted_amount : nat64;
        utxo : Utxo;
    };
};

// Utxos that don't have enough confirmations to be processed.
type PendingUtxo = record {
    outpoint : record { txid : vec nat8; vout : nat32 };
    value : nat64;
    confirmations: nat32;
};

// Number of nanoseconds since the Unix Epoch
type Timestamp = nat64;

type SuspendedUtxo = record {
    utxo : Utxo;
    reason : SuspendedReason;
    earliest_retry: Timestamp;
};

type UpdateBalanceError = variant {
    // There are no new UTXOs to process.
    NoNewUtxos : record {
        current_confirmations: opt nat32;
        required_confirmations: nat32;
        pending_utxos: opt vec PendingUtxo;
        suspended_utxos: opt vec SuspendedUtxo;
    };
    // The minter is already processing another update balance request for the caller.
    AlreadyProcessing;
    // The minter is overloaded, retry the request.
    // The payload contains a human-readable message explaining what caused the unavailability.
    TemporarilyUnavailable : text;
    // A generic error reserved for future extensions.
    GenericError : record { error_message : text; error_code : nat64 };
};

type BtcNetwork = variant {
    // The public Bitcoin mainnet.
    Mainnet;
    // The public Bitcoin testnet.
    Testnet;
    // A local Bitcoin regtest installation.
    Regtest;
};

type Mode = variant {
    // The minter does not allow any state modifications.
    ReadOnly;
    // Only specified principals can modify minter's state.
    RestrictedTo : vec principal;
    // Only specified principals can convert BTC to ckBTC.
    DepositsRestrictedTo : vec principal;
    // Anyone can interact with the minter.
    GeneralAvailability;
};

// The initialization parameters of the minter canister.
type InitArgs = record {
    // The minter will interact with this Bitcoin network.
    btc_network : BtcNetwork;

    // The principal of the ledger that handles ckBTC transfers.
    // The default account of the ckBTC minter must be configured as
    // the minting account of the ledger.
    ledger_id : principal;

    // The name of the ECDSA key to use.
    // E.g., "dfx_test_key" on the local replica.
    ecdsa_key_name : text;

    // The minimal amount of ckBTC that can be converted to BTC.
    retrieve_btc_min_amount : nat64;

    /// Maximum time in nanoseconds that a transaction should spend in the queue
    /// before being sent.
    max_time_in_queue_nanos : nat64;

    /// The minimum number of confirmations required for the minter to
    /// accept a Bitcoin transaction.
    min_confirmations : opt nat32;

    /// The minter's operation mode.
    mode : Mode;

    /// The fee paid per Bitcoin check.
    check_fee : opt nat64;

    /// The fee paid per check by the KYT canister (deprecated, use check_fee instead).
    kyt_fee : opt nat64;

    /// The canister id of the Bitcoin checker canister.
    btc_checker_principal: opt principal;

    /// The canister id of the KYT canister (deprecated, use btc_checker_principal instead).
    kyt_principal: opt principal;

    /// The expiration duration (in seconds) for cached entries in the get_utxos cache.
    get_utxos_cache_expiration_seconds: opt nat64;
};

// The upgrade parameters of the minter canister.
type UpgradeArgs = record {
    // The minimal amount of ckBTC that the minter converts to BTC.
    retrieve_btc_min_amount : opt nat64;

    /// Maximum time in nanoseconds that a transaction should spend in the queue
    /// before being sent.
    max_time_in_queue_nanos : opt nat64;

    /// The minimum number of confirmations required for the minter to
    /// accept a Bitcoin transaction.
    min_confirmations : opt nat32;

    /// If set, overrides the current minter's operation mode.
    mode : opt Mode;

    /// The fee per Bitcoin check.
    check_fee : opt nat64;

    /// The fee paid per check by the KYT canister (deprecated, use check_fee instead).
    kyt_fee : opt nat64;

    /// The principal of the Bitcoin checker canister.
    btc_checker_principal : opt principal;

    /// The canister id of the KYT canister (deprecated, use btc_checker_principal instead).
    kyt_principal: opt principal;

    /// The expiration duration (in seconds) for cached entries in the get_utxos cache.
    get_utxos_cache_expiration_seconds: opt nat64;
};

type RetrieveBtcStatus = variant {
    // The minter does not have any information on the specified
    // retrieval request.  It can be that nobody submitted the
    // request or the minter pruned the relevant information from the
    // history to save space.
    Unknown;

    // The minter did not send a Bitcoin transaction for this request yet.
    Pending;

    // The minter is obtaining all required ECDSA signatures on the
    // Bitcoin transaction for this request.
    Signing;

    // The minter signed the transaction and is waiting for a reply
    // from the Bitcoin canister.
    Sending : record { txid : blob };

    // The minter sent a transaction for the retrieve request.
    // The payload contains the identifier of the transaction on the Bitcoin network.
    Submitted : record { txid : blob };

    // The amount was too low to cover the transaction fees.
    AmountTooLow;

    // The minter received enough confirmations for the Bitcoin
    // transaction for this request.  The payload contains the
    // identifier of the transaction on the Bitcoin network.
    Confirmed : record { txid : blob };
};

type ReimbursementRequest = record {
  account : Account;
  amount : nat64;
  reason : ReimbursementReason;
};

type ReimbursedDeposit = record {
  account : Account;
  mint_block_index : nat64;
  amount : nat64;
  reason : ReimbursementReason;
};

type RetrieveBtcStatusV2 = variant {
    // The minter does not have any information on the specified
    // retrieval request.  It can be that nobody submitted the
    // request or the minter pruned the relevant information from the
    // history to save space.
    Unknown;
    // The minter did not send a Bitcoin transaction for this request yet.
    Pending;
    // The minter is obtaining all required ECDSA signatures on the
    // Bitcoin transaction for this request.
    Signing;
    // The minter signed the transaction and is waiting for a reply
    // from the Bitcoin canister.
    Sending : record { txid : blob };
    // The minter sent a transaction for the retrieve request.
    // The payload contains the identifier of the transaction on the Bitcoin network.
    Submitted : record { txid : blob };
    // The amount was too low to cover the transaction fees.
    AmountTooLow;
    // The minter received enough confirmations for the Bitcoin
    // transaction for this request.  The payload contains the
    // identifier of the transaction on the Bitcoin network.
    Confirmed : record { txid : blob };
    /// The retrieve Bitcoin request has been reimbursed.
    Reimbursed : ReimbursedDeposit;
    /// The minter will try to reimburse this transaction.
    WillReimburse : ReimbursementRequest;
};

type Utxo = record {
    outpoint : record { txid : vec nat8; vout : nat32 };
    value : nat64;
    height : nat32;
};

type BitcoinAddress = variant {
    p2wpkh_v0 : blob;
    p2wsh_v0 : blob;
    p2tr_v1 : blob;
    p2pkh : blob;
    p2sh : blob;
};

type MinterInfo = record {
    min_confirmations : nat32;
    // This amount is based on the `retrieve_btc_min_amount` setting during canister
    // initialization or upgrades, but may vary according to current network fees.
    retrieve_btc_min_amount : nat64;
    // The same as `check_fee`, but the old name is kept here to be backward compatible.
    kyt_fee : nat64;
};

type ReimbursementReason = variant {
    CallFailed;
    TaintedDestination : record {
        kyt_fee : nat64;
        kyt_provider: principal;
    };
};

type SuspendedReason = variant {
    // The minter ignored this UTXO because UTXO's value is too small to pay
    // the check fees.
    ValueTooSmall;
    // The Bitcoin checker considered this UTXO to be tainted.
    Quarantined;
};

type Event = record {
    timestamp : opt nat64;
    payload : EventType;
};

type EventType = variant {
    init : InitArgs;
    upgrade : UpgradeArgs;
    received_utxos : record { to_account : Account; mint_txid : opt nat64; utxos : vec Utxo };
    accepted_retrieve_btc_request : record {
        amount : nat64;
        address : BitcoinAddress;
        block_index : nat64;
        received_at : nat64;
        kyt_provider : opt principal;
        reimbursement_account : opt Account;
    };
    distributed_kyt_fee : record {
        kyt_provider : principal;
        amount : nat64;
        block_index: nat64;
    };
    removed_retrieve_btc_request : record { block_index : nat64 };
    sent_transaction : record {
        requests : vec nat64;
        txid : blob;
        utxos : vec Utxo;
        change_output : opt record { vout : nat32; value : nat64 };
        submitted_at : nat64;
        fee: opt nat64;
    };
    replaced_transaction : record {
        new_txid : blob;
        old_txid : blob;
        change_output : record { vout : nat32; value : nat64 };
        submitted_at : nat64;
        fee: nat64;
    };
    confirmed_transaction : record { txid : blob };
    checked_utxo : record {
        utxo : Utxo;
        uuid : text;
        clean : bool;
        kyt_provider : opt principal;
    };
    checked_utxo_v2 : record {
        utxo : Utxo;
        account : Account;
    };
    checked_utxo_mint_unknown : record {
        utxo : Utxo;
        account : Account;
    };
    ignored_utxo : record { utxo: Utxo; };
    suspended_utxo : record { utxo: Utxo; account: Account; reason: SuspendedReason };
    retrieve_btc_kyt_failed : record {
        address : text;
        amount : nat64;
        owner : principal;
        kyt_provider : principal;
        uuid : text;
        block_index : nat64;
    };
    schedule_deposit_reimbursement : record {
        account : Account;
        burn_block_index : nat64;
        amount : nat64;
        reason : ReimbursementReason;
    };
    reimbursed_failed_deposit : record { burn_block_index : nat64; mint_block_index : nat64 };
};

type MinterArg = variant {
    Init : InitArgs;
    Upgrade : opt UpgradeArgs;
};

service : /* (minter_arg : MinterArg) -> */ {
    // Section "Convert BTC to ckBTC" {{{

    // Returns the Bitcoin address to which the owner should send BTC
    // before converting the amount to ckBTC using the [update_balance]
    // endpoint.
    //
    // If the owner is not set, it defaults to the caller's principal.
    // The resolved owner must be a non-anonymous principal.
    get_btc_address : (record { owner: opt principal; subaccount : opt blob }) -> (text);

    // Returns UTXOs of the given account known by the minter (with no
    // guarantee in the ordering of the returned values).
    //
    // If the owner is not set, it defaults to the caller's principal.
    get_known_utxos: (record { owner: opt principal; subaccount : opt blob }) -> (vec Utxo) query;

    // Mints ckBTC for newly deposited UTXOs.
    //
    // If the owner is not set, it defaults to the caller's principal.
    //
    // # Preconditions
    //
    // * The owner deposited some BTC to the address that the
    //   [get_btc_address] endpoint returns.
    update_balance : (record { owner: opt principal; subaccount : opt blob }) -> (variant { Ok : vec UtxoStatus; Err : UpdateBalanceError });

    // }}} Section "Convert BTC to ckBTC"

    // Section "Convert ckBTC to BTC" {{{

    /// Returns an estimate of the user's fee (in Satoshi) for a
    /// retrieve_btc request based on the current status of the Bitcoin network.
    estimate_withdrawal_fee : (record { amount : opt nat64 }) -> (record { bitcoin_fee : nat64; minter_fee : nat64 }) query;

    /// Returns the fee that the minter will charge for a bitcoin deposit.
    get_deposit_fee: () -> (nat64) query;

    // Returns the account to which the caller should deposit ckBTC
    // before withdrawing BTC using the [retrieve_btc] endpoint.
    get_withdrawal_account : () -> (Account);


    // Submits a request to convert ckBTC to BTC.
    //
    // # Note
    //
    // The BTC retrieval process is slow.  Instead of
    // synchronously waiting for a BTC transaction to settle, this
    // method returns a request ([block_index]) that the caller can use
    // to query the request status.
    //
    // # Preconditions
    //
    // * The caller deposited the requested amount in ckBTC to the account
    //   that the [get_withdrawal_account] endpoint returns.
    retrieve_btc : (RetrieveBtcArgs) -> (variant { Ok : RetrieveBtcOk; Err : RetrieveBtcError });

    // Submits a request to convert ckBTC to BTC.
    //
    // # Note
    //
    // The BTC retrieval process is slow.  Instead of
    // synchronously waiting for a BTC transaction to settle, this
    // method returns a request ([block_index]) that the caller can use
    // to query the request status.
    //
    // # Preconditions
    //
    // * The caller allowed the minter's principal to spend its funds
    //   using [icrc2_approve] on the ckBTC ledger.
    retrieve_btc_with_approval : (RetrieveBtcWithApprovalArgs) -> (variant { Ok : RetrieveBtcOk; Err : RetrieveBtcWithApprovalError });

    /// [deprecated] Returns the status of a withdrawal request.
    /// You should use retrieve_btc_status_v2 to retrieve the status of your withdrawal request.
    retrieve_btc_status : (record { block_index : nat64 }) -> (RetrieveBtcStatus) query;

    /// Returns the status of a withdrawal request request using the RetrieveBtcStatusV2 type.
    retrieve_btc_status_v2 : (record { block_index : nat64 }) -> (RetrieveBtcStatusV2) query;

    // Returns the withdrawal statues by account.
    //
    // # Note
    // The _v2_ part indicates that you get a response in line with the retrieve_btc_status_v2 endpoint,
    // i.e., you get a vector of RetrieveBtcStatusV2 and not RetrieveBtcStatus.
    //
    retrieve_btc_status_v2_by_account : (opt Account) -> (vec record { block_index: nat64; status_v2: opt RetrieveBtcStatusV2; }) query;

    // }}} Section "Convert ckBTC to BTC"

    // Section "Minter Information" {{{
    // Returns internal minter parameters.
    get_minter_info : () -> (MinterInfo) query;

    get_canister_status : () -> (CanisterStatusResponse);
    // }}}

    // Section "Event log" {{{

    // The minter keeps track of all state modifications in an internal event log.
    //
    // This method returns a list of events in the specified range.
    // The minter can return fewer events than requested. The result is
    // an empty vector if the start position is greater than the total
    // number of events.
    //
    // NOTE: this method exists for debugging purposes.
    // The ckBTC minter authors do not guarantee backward compatibility for this method.
    get_events : (record { start: nat64; length : nat64 }) -> (vec Event) query;
    // }}} Section "Event log"
}