Thoughts on the token standard

We have added DSCVR, OPENCHAT, DISTRIKT, WEACT to the list of keys supported by extend info.

Improve: add IC native social media(DSCVR,OPENCHAT,DISTRIKT,WEACT)

But I didn’t find any information about NUANCE, so I haven’t added it yet. If I find information about NUANCE, I would like to add NUANCE to the list, and any PRs are welcome.

Thanks @ililic for the suggestion again.

The token standard is an important basic standard in the dfinity ecosystem. The establishment of a complete standard requires everyone to participate. Anyone is welcome to comment and submit a PR

2 Likes

The motoko implementation has been completed, but the call of approve and transfer has not been implemented for this reason.

how to test Motoko implementation?

git clone from here , then run

make test_motoko
3 Likes

Hey everyone! Consider checking out Non Fungible Token (NFT) Standard [Community Consideration] this thread if you’re interested in some new ideas around token standards.

3 Likes

DeLand Labs DFT latest update:

  1. Dropped Dfinity Self Describing Standard

Motoko canister support default method __get_candid_interface_tmp_hack to get did, if rust canister implement this method ,Developer can check canister’s interface through __get_candid_interface_tmp_hack .

candid::export_service!();

#[query(name = "__get_candid_interface_tmp_hack")]
#[candid_method(query, rename = "__get_candid_interface_tmp_hack")]
fn __export_did_tmp_() -> String {
    __export_service()
} 
  1. Rust implementation has removed supportedInterface(text) -> (bool)

Deland Labs will continue to improve DFT.Come on with us to improve it,any comments, PR are welcome.

3 Likes

Updates summary:


  owner : () -> (principal);
  setOwner : (principal) -> (bool);
  setExtend : (vec KeyValuePair) -> (bool);
  setFee : (Fee) -> (bool);
  setFeeTo : (text) -> (bool);
  setLogo : (vec nat8) -> (bool);

  allowancesByHolder : (text) -> (vec record { TokenHolder; nat }) query;
  tokenInfo : () -> (TokenInfo) query;

  lastTransactions : (nat64) -> (TxRecordsResult) query;
  transactionById : (text) -> (TxRecordResult) query;
  transactionByIndex : (nat) -> (TxRecordResult) query;

The repository is here , Leave your comments & advices to make it better , and any PR are welcome..

Updates summary:

  • add burnable & mint extension , and example code
  • refactor code

How to use rust to create a fungible token with 1 line of code?

check it out at here

1 Like

Oh and if you’re planning to deploy a token right now, use Rust. Currently the motoko GC has real problems running lots of small update calls on big heaps which is exactly the kind of workloads that tokens have.

Is this still a problem with Motoko? I wonder if any of the recent updates have fixed this.

2 Likes

I just went through the entire thread. Here’s a (somewhat) condensed summary…


Ethereum and IC are different. Platform differences may require different token standards.

  1. Ethereum cross-contract transactions are atomic. IC cross-canister updates are not.

This matters if you want the token canister to be able to “notify” other canisters, which is pretty important. Notifications can be implemented with pre-defined callbacks or hooks (see ERC-677 and ERC-777), or with a pub/sub pattern where interested canisters can subscribe to the token canister. Either way, one canister is calling another, which is not atomic in IC. Developers will need to implement two-phase commit or sagas if they want cross-canister atomicity.

  1. Ethereum transaction history is stored and publicly available, both in the event log and in the blocks themselves. Neither is stored in IC.

IC token canisters will need to keep track of transaction history themselves. This can be in the token canister itself or in separate canisters. But at some point the 8 GB canister storage limit will be reached, meaning a multi-canister solution is probably necessary. For example, the ICP ledger canister maintains a bunch of dynamically created archive canisters.

  1. Ethereum smart contracts are immutable. IC canisters are upgradable by default (but can be made immutable).

Upgradability is both a blessing and a curse. It’s good because a token canister could be upgraded to support future extensions, e.g. minting, burning, batching, etc. It’s because because malicious controllers can “rug pull”. Either way, source code verifiability is needed. Etherscan provides this in Ethereum, but no service like this currently exists in IC.

  1. Ethereum transactions are “pay-as-you-go”. IC updates are paid for with pre-funded cycles.

How will token canisters be funded? The reverse gas model gives us flexibility here. We can require fees on every update, like in Ethereum, or we can requires fees on only some. We can charge the sender, or we can charge the receiver. There are plenty of possibilities, and they each have different implications on how various payment flows could work with a IC token canister.

  1. Ethereum transactions are run serially (i.e the EVM is “single-threaded”). IC updates can be run in parallel.

Canisters running in different subnets are effectively running on different blockchains. I’m not too clear on what this means for a token standard. @senior.joinu describes an interesting scenario where this may cause problems when multiple assets are managed by different canisters, each of which makes progress independently.


I’m still trying to better understand the tradeoffs between the different ways for a user to authorize a canister to make transactions on their behalf, e.g. using ERC-20 style approve and transferFrom or pub/sub. I’ll summarize my findings later this week…

8 Likes

Personally, I think having a great token standard is super important and becoming a more urgent need, especially as the SNS design becomes finalized and as an increasing number of dapps (including ours) look to add token functionality in the near future.

6 Likes

Agree.
More meaningful discussions will help improve the token standard

Really fascinating read, thanks for writing this.

In other words, you could simply fool such a system, approving money, sending an application and then quickly disapproving them back - there is a chance for you to spot the exact moment when the middle-man canister did already checked their allowance of your payment, but did not yet transferedFrom it.

I don’t think this is correct. Standard transferFrom implementations check that the allowance is sufficient before modifying any balances. If the check fails, the transferFrom call will trap, and the middle-man canister will throw before actually transferring the train and concert tickets to the user.

The more I think about this, the more difficult I see the problem of atomic cross-canister transactions. (Sorry, this will be a long post.)


Some background…

A common use case of tokens is for a user to spend some tokens to perform some action on the blockchain. For example, a user wants to spend 20 LINK to convert it to BAT on some DEX. This entire flow should be treated as a transaction.

There are 3 parties involved:

  • Spender (either user or canister)
  • Token canister
  • Receiver canister (e.g. the DEX canister in the above example)

Let’s abbreviate these parties as S, T, and R, respectively.

S always initiates the transaction, since they own the tokens. So S → T is the first step. The question is what happens next?

In ERC-20, the flow is S → T (approve), S → R (startAction, e.g.), and R → T (transferFrom).
In ERC-667, the flow is S → T (transferAndCall) and T → R (onTokenTransfer).

Notice that the caller-callee relationship for T and R is inverted in ERC-667 versus ERC-20. The benefit of ERC-667 over ERC-20 is one transaction for the spender instead of two, which means lower gas fees (on Ethereum) and a simpler UX. The drawback is that reentrancy may be an issue if implemented incorrectly.

@senior.joinu’s proposal to use pub/sub is an extension of the ERC-667 flow, where T calls multiple Rs and not just the receiver of the spent tokens. Generally speaking, it inherits the same properties as the simpler ERC-667 flow, so I don’t talk about it here.


So I tried porting over the transferAndCall function from ERC-667 onto IC in Motoko. Since updates that call other canisters are not atomic, I added a rollback in the event of a callback failure.

This is what it looks like:

actor Token {

    private stable var balances: Trie.Trie<Principal, Nat> = Trie.empty();

    private func _key(p: Principal) : Trie.Key<Principal> { { key = p; hash = Principal.hash(p) } };
    
    private func _getBalance(p: Principal): Nat {
        let balance = Trie.find(balances, _key(p), Principal.equal);
        switch (balance) {
            case (null) { 0 };
            case (?balance) { balance };
        };
    };

    private func _setBalance(p: Principal, value: Nat): () {
        balances := Trie.put(balances, _key(p), Principal.equal, value).0;
    };

    public shared(msg) func transferAndCall(to: Principal, value: Nat): async () {
        let fromBalance = _getBalance(msg.caller);
        let toBalance = _getBalance(to);

        assert fromBalance >= value;

        _setBalance(msg.caller, fromBalance - value);
        _setBalance(to, toBalance + value);

        // Skip this entire try block if `to` is a user principal and not a canister principal.
        // There may be no good way to determine that...
        try {
             let toActor = actor Principal.toText(to): actor {
                 onTokenTransfer: (from: Principal, value: Nat) -> async ()
             };
             await toActor.onTokenTransfer(msg.caller, value);
        } catch (err) {
            // Rollback in the event of callback error
            _setBalance(msg.caller, fromBalance);
            _setBalance(to, toBalance);

            // Can't assert (i.e. trap) instead of throw, otherwise rollback would be reverted
            throw Error.reject("transferAndCall failed");
        };
    };
};

In the distributed transactions world, I think this would be an example of the saga pattern, with the rollback called a “compensating transaction”.

Here’s the problem…

Isolation is poor.

For example, let’s say transferAndCall has updated the balances and is currently await-ing on the onTokenTransfer callback to complete. In the meantime, either the spender or the receiver queries their balance. Then, onTokenTransfer failed and the balances are reverted. The data queried is no longer accurate. This is a dirty read.

Even worse, someone could have tried making illegal transfers during that await-ing time, e.g. spending tokens they have at the time but won’t after onTokenTransfer eventually fails. But if they would’ve had enough tokens either way (had onTokenTransfer failed or succeeded), then their transfer would be reverted by the rollback, leading to a lost update.

How can we solve this?

  1. We could move the onTokenTransfer call to before we update the balances but after we assert. That way, we don’t need to roll anything back, because the balance updating code can’t fail and it’s also the last thing that happens. The problem is: what if onTokenTransfer actually needs to do something with those tokens, e.g. a DEX canister wants to swap those tokens for another type of token? Those tokens wouldn’t exist if we made the call before updating the balances.
  2. Alternatively, we could keep the current order, but keep tracking of “pending balances” in addition to the actual balances. Initially, we update the pending balances, and then we call onTokenTransfer. If that call succeeds, we “finalize” the pending balances by updating the actual balances. If that call fails, we revert the pending balances. This way, dirty reads don’t happen because they never see the pending balances, only the actual balances. Lost updates can be prevented by not proceeding if a pending transfer is in progress (not clear on the details here). This general approach is called semantic locking. The issue is that a) locking isn’t great, and b) the same issue as above, where the onTokenTransfer function wants to do something with its newly received tokens but can’t because they are still “pending” and not “finalized” here.

Not really sure what to conclude from all of this…

Am I overcomplicating things? Does an ERC-20 flow with approve and transferFrom avoid some of these problems with an ERC-667 flow? I’d be really interested to hear what others think.

5 Likes

Hi @jzxchiang,Let me try to understand your concerns:

transferAndCall contains two non-atomic operations, you give a way like saga, great, this is the solution I recommend.

But I think that transfer and call do not need to pursue strong consistency. Transfer is verified by the logic inside the token. Before the call occurs, it will be determined whether the transfer can be successful, but the call should be allowed to fail. Use the call as an alternative pub/sub trigger mechanism or notification mechanism will be great.

Let us think about the application scenario of call in transferAndCall:
If the call fails due to a bug, it will be fixed, so ignore this scenario
If the call fails after a successful transfer, the receiver canister (such as the DEX in your example), how to deal with the amount of the transfer should be a matter for the DEX to consider, such as transferring it back to the user or recording a user balance.
Saga should not have a rollback in the strict sense, but different processing branches (such as the above step: successful transaction or transfer back to the user), and the subsequent processing steps belong to the canister of DEX. Token should not consider how to intervene in the subsequent processing steps, you know , the caniter of the subsequent processing can be DEX or other, and their processing steps will be different.

You mentioned the problem of reentrancy. Another option is to use approveAndCall, which is similar to the usage of ERC20’s approve/transferFrom. In the DEX usage scenario you gave, after approve, the DEX can call transferFrom to get the token when processing the transaction, which can avoid the reentrance problem.

What I mentioned above has been implemented in Deland Labs’ fungible token standard, just have a look at here

And the document will be released next week, I will send you a link to the document in a few days.

1 Like

I believe, this is not a problem at all.

The reason for that is “the island mindset” - once you have that, everything seems reasonable.

If you want to make your action atomic, you should perform it by a single canister. In a DEX workflow this would mean to use DEX canister as a trusted intermediary. Just like Binance asks you to deposit your tokens before you start trading, this DEX canister would do the same maintaining internal balance list.
Once your tokens are there, you can safely exchange them. Once you’re done - you withdraw them, transforming them back to a real tokens on a separate canister.

4 Likes

If the call fails after a successful transfer, the receiver canister (such as the DEX in your example), how to deal with the amount of the transfer should be a matter for the DEX to consider, such as transferring it back to the user or recording a user balance.

Saga should not have a rollback in the strict sense, but different processing branches (such as the above step: successful transaction or transfer back to the user), and the subsequent processing steps belong to the canister of DEX.

I’m not sure this is true.

If a user wants to pay for a service, they want to make sure that the service is successfully executed if their tokens are transferred. In other words, the token transfer and the service execution should happen in an atomic transaction.

Trusting the canister executing the service (e.g. DEX canister) to refund the tokens in the event of failure doesn’t seem safe, especially in IC where canisters aren’t necessarily autonomous.

If you want to make your action atomic, you should perform it by a single canister. In a DEX workflow this would mean to use DEX canister as a trusted intermediary.

I’m not sure I follow. In my mind, there’s a single “canonical” token canister that keeps track of everyone’s balances. How would you deposit tokens to a DEX canister without interacting with the token canister? The internal balances kept by the DEX canister would need to be kept in sync with the canonical token canister anyways. Would this syncing be done in some cronjob? If so, can’t it go out of sync, leading to bad situations?

In the future, there will be a wide variety of canisters that depend on the token canister. Some will be autonomous (i.e. a smart contract), others will not. The “island mindset” you describe may be feasible if we assume all depending canisters are autonomous, but the “trusted intermediary” assumption breaks down if that’s not true.

Hmm…
Sorry if I didn’t make clear what I mean.

The DEX canister has a principal. One could transfer their tokens to that principal (same way they do for any other person). The DEX could listen for the event emitted by this transfer and account these tokens as a tokens deposited to the DEX by the sender.

Withdraw is the same but backwards - DEX canister transfers tokens to the reciever.

Yes, but that publish call from token canister to DEX canister should be atomic with the token canister balance updates, which is the same problem I described earlier.

The document: https://dft.delandlabs.com/

Let me try to reproduce this scenario once again.
Let’s imagine, we have:

  • Users A and B
  • Token canisters T1 and T2;
    • both tokens have the exact same transferAndNotify() mechanics - transfer part is performed inside the token canister itself (balances modification) and notify part is just a call to another canister;
    • user A has some amount of T1 tokens, user B - some amount of T2;
    • let’s imagine that the exchange rate T1:T2 = 1:1;
  • The DEX canister;
    • this canister holds an internal list of balances for each token canister T1, T2;
    • these internal balances are made only of tokens transferAndNotify-ed to DEX canister’s principal.

Let’s imagine the flow now:

  1. Both users want to swap their tokens (A wants to swap their T1 for T2 and vice versa - B wants T2T1).
  2. They both go to the DEX's frontend and see the Deposit tokens button.
  3. They select the tokens they want to deposit (A goes with T1, B - with T2).
  4. They both click on that button and enter the amount they wan’t to deposit - 100 tokens each.
  5. Once user A clicks on that button, the frontend sends a request to canister T1 - “please transferAndNotify() the DEX canister 100 tokens”.
  6. Token canister T1 subs 100 tokens from the balance of user A and adds them to the balance of the DEX canister. (atomic)
  7. Token canister then notifies the DEX canister, sending it an inter-canister call: “hey, this principal just transferred some tokens of this token canister (T1) to your principal”. (not atomic)
  8. DEX canister receives the notification and sets its internal balance of token T1 of user A to 100 tokens. (atomic)
  9. Steps 4-7 are the same for user B and token T2.
  10. Now both users have “deposited” their tokens to the DEX.
  11. We don’t care, how the actual swap is performed, but let’s imagine it’s an order book model.
  12. User A clicks the button Create a sell order for token T1 (qty = 100).
  13. DEX canister subs 100 T1 tokens from the internal balance of user A and stores the order in its memory. (atomic)
  14. User B clicks the button Create a buy order for token T1 (qty = 100).
  15. DEX canister calculates the exchange rate (1:1) and subs 100 T2 tokens for the internal balance of user B; then it matches this buy order with the sell order of user A and destroys these orders, adding 100 T2 tokens to user A and 100 T1 tokens to user B (to their internal balances); important. (atomic)
  16. Both users click the “Withdraw” button (user A selects T2 token, user B - T1).
  17. DEX canister clears their internal balances and sends an inter-canister call to token canisters - for user A in sends a call to the token T2, for user B - to the token T1.
  18. This is the same transferAndNotify() call as it was at the beginning, but in reverse - DEX canister sends its real tokens back. (not atomic)
  19. Token T1 subs 100 tokens from DEX balance and adds them to user B's balance. (atomic)
  20. Token T2 subs 100 tokens from DEX balance and adds them to user A's balance. (atomic)
  21. The swap is performed, we’re arrived at the desired state.

Yes, there are non-atomic actions. But you can imagine a process when they do no affect the security at all. The only thing left is to make sure, that if any inter-canister call fails (due to an error, or a network issue) - state changes are reverted.

5 Likes