Thoughts on the token standard

Developers
122

Thanks for sharing this. I just took a look, and there’s a lot of stuff in there.

I’m gonna focus on transaction atomicity and try to figure out whether this proposed interface can solve the standard DEX user journey.

In this user journey, we have the following entities:

  • Alice, who wants to swap Token A for Token B
  • Token A canister
  • Token B canister
  • DEX canister (assume there’s only one canister for simplicity)

Now, this is the typical ERC-20 flow that would work on Ethereum:

  1. Alice calls approve on Token A canister to let DEX canister transfer some number of Token As on her behalf
  2. Alice calls some custom function (e.g. swap) on DEX canister to initiate the swap, which does the following:
    a. DEX canister calls transferFrom on Token A canister to transfer Token As from Alice’s address to the DEX canister’s address (to add to its Token A reserves)
    b. DEX canister internally calculates how many Token Bs Alice should get based on some formula like x * y = k
    c. DEX canister calls transfer on Token B canister to transfer Token Bs from DEX canister’s address to Alice’s address (which subtracts from its Token B reserves)
    d. DEX canister returns true on Alice’s swap call if steps 2a, 2b, and 2c were successful; otherwise, it returns false and the whole transaction is atomically rolled back

Note that:

  • The DEX canister makes 2 inter-canister calls: transferFrom and transfer. This example assumes that inter-canister calls are atomic because they are in Ethereum. On IC, they are not.
  • transferFrom is used for Token A but transfer is used for Token B. This is intentional because the only 3rd party transfer (i.e. Alice authorizing DEX canister to transfer on her behalf) happens for Token A. For Token B, the entity that owns the tokens and who initiates the transaction is one and the same, i.e. DEX canister.
1 Like
123

OK, so how would we use this new DRC20 interface to implement this same flow?

Based off what I read, here is my guess (please correct me if I’m wrong):

  1. Alice calls approve on Token A canister, same as before
  2. Alice calls some custom function (e.g. swap) on DEX canister to initiate the swap, which does the following:
    a. DEX canister calls lockTransferFrom on Token A canister to prepare to transfer Token As from Alice to DEX canister, with itself as the “decider”
    b. DEX canister internally calculates how many Token Bs Alice should get, same as before
    c. DEX canister calls lockTransfer on Token B canister to prepare to transfer Token Bs from DEX canister to Alice, with itself as the “decider”
    d. Now, depending on whether if steps 2a, 2b, and 2c were successful, DEX canister does different things…
    • If all were successful, DEX canister calls executeTransfer on both Token A canister and Token B canister to complete the transfers (using sendAll as the execute type). Once both executeTransfer calls complete successfully, DEX canister returns true to Alice
    • If at least one step failed, DEX canister calls executeTransfer on both Token A canister and Token B canister to rollback the transfers (using fallback as the execute type). Once both executeTransfer calls complete successfully, DEX canister returns false to Alice

This assumes that executeTransfer must succeed if lockTransfer (or lockTransferFrom) succeeds.

^ Does this flow sound right to you? @bitbruce

124

You are right in your understanding.

lockTransfer/executeTransfer is an abstract base function for improving the atomicity of Tokens on the IC. It can be applied to many scenarios. SWAP is one of them.

executeTransfer may fail. Dex canister can remedy this. It can execute executeTransfer again. To be safe, Dex canister should execute Token A canister’s executeTransfer first.

2 Likes
125

I thought the whole point of lockTransfer returning true is so that executeTransfer doesn’t fail. In a two-phase commit protocol, isn’t the point of phase 1 so that phase 2 won’t fail?

Also, I’m curious why DEX canister should execute Token A canister’s executeTransfer first.

If you could provide a bit more documentation on how you expect clients to use this standard, I think it would be very helpful to the community.

General observations about the standard:

  • The standard is a little bloated. Is it necessary to continue to support the approve / transferFrom flow? ERC-20 has it for historical reasons, but since we are developing a new standard in IC, we don’t have to follow in their footsteps. And my understanding is that callback-based approaches like transferAndCall (or in this case, pub/sub) is the next-gen replacement for approve / transferFrom.
  • I’m not sure I understand how gas works. Why do you let users pick between paying gas in tokens or cycles? If they choose to pay in tokens, how will the token canister convert those tokens to cycles, since it can only burn cycles to continue running? Why is there a distinction between gas and setGas to begin with?
  • We might need certified variables to ensure that query methods (e.g. balanceOf) return the correct content, without resorting to slow updates. This is what the ICP ledger canister does, and it’s not a trivial implementation. For example, they store ICP transactions in some sort of Merkle tree as an internal data structure in the canister.

Also, I’m curious what you plan on doing with this standard. Are you also implementing a wallet or DEX? Thanks for the great work.

2 Likes
126

You are right, it was my misunderstanding of your example.

I talked about another scenario: if Dex has token pool (with Token A and Token B), and if Aice wants to exchange Token A for Token B (transfer Token A to Dex pool and get Token B from Dex pool), then Dex should first execute Token A canister’s executeTransfer before executing Token B canister’s transfer.

approve / transferFrom, transferAndCall, pub/sub are not interchangeable and can be satisfied with different scenarios, which can expose different problems in atomicity. As transferAndCall has difficulty handling Callback failures, which is bad for IC‘s without atomicity support, we drop transferAndCall in favor of approve / transferFrom, pub/sub.

Let sender pay for gas, the purpose is to prevent ddos attack. The charging method is cycles, token, if token is charged as gas, it will be destroyed (default) or sent to FEE_TO, canister’s cycles need to be provided by someone else; if cycles are charged as gas, it will be used as canister’s cycles (default) or sent to FEE_TO.

We will provide some use cases at a subsequent time.
You guessed right! ICLighthouse is working on a wallet and will provide Defi Dapps later.

2 Likes
127

approve / transferFrom, transferAndCall, pub/sub are not interchangeable and can be satisfied with different scenarios, which can expose different problems in atomicity. As transferAndCall has difficulty handling Callback failures, which is bad for IC‘s without atomicity support, we drop transferAndCall in favor of approve / transferFrom, pub/sub.

Hm… but pub/sub also involves a token canister calling external canisters (i.e. subscribers). How does pub/sub avoid the atomicity problems that transferAndCall suffers from? From my perspective, it seems like pub/sub would be even harder to make atomic, since there can be an arbitrary number of subscribers who need to be notified.

We will provide some use cases at a subsequent time.
You guessed right! ICLighthouse is working on a wallet and will provide Defi Dapps later.

Awesome—looking forward to it!

1 Like
128

transferAndCall is often used in synchronous calls.
pub/sub is often used for asynchronous message consumption.
In the IC development environment, pub/sub is more friendly to programmers.
In terms of functionality pub/sub can override transferAndCall.

129

Why is pub/sub more friendly to programmers if it’s not done atomically?

What type of use case were you envisioning developers using pub/sub for instead of approve / transferFrom?

130

It supports both pub/sub and approve/transferFrom

131

Dfinity Fungible Token Standard v0.1.3 Released

132

Dfinity Fungible Token Standard Change Logs

v0.1.4 Changes

v0.2.0

v0.2.1

Let me introduce the major change in v0.2

Nonce: approve/transfer/transferFrom/burn/mint support optional nonce to avoid submitting the same transaction repeatedly
Remove notify & call : why remove?

Talking to untrustworthy canisters can be risky, for the following (likely incomplete) reasons:

The other canister can withhold a response. Although the bidirectional messaging paradigm of the Internet Computer was designed to guarantee a response eventually, the other party can busy-loop for as long as they are willing to pay for before responding. Worse, there are ways to deadlock a canister.
The other canister can respond with invalidly encoded Candid. This will cause a Motoko-implemented canister to trap in the reply handler, with no easy way to recover. Other CDKs may give you better ways to handle invalid Candid, but even then you will have to worry about Candid cycle bombs that will cause your reply handler to trap.

Fee struct : the old logic

The fee rate has a default decimals 8, this one is undocumented, just a default setting that confuses developers. Now rate_decimals is added to Fee Struct to clearly identify the decimal places of rate, which is more clear

type Fee = record { rate : nat; rate_decimals : nat8; minimum : nat };

Tx Record: DFT Standard compatible with AccountId and Principal, if the caller in the TxRecord use principal, the original Principal of the accountid will be exposed. For privacy reasons, the caller of the tx record is changed from Principal to TokenHolder to avoid privacy leakage.

4 Likes
133

v0.4.0 is released, this release contains a large number of automated integration test cases to verify the reliability of the code. As the infrastructure of DEFI, reliability and security are the most important things

2 Likes
134

v0.4.0 is released

Is this standard supported / maintained by Dfinity the org?

135

No, the Dfinity Foundation is not involved with Deland-Labs’s project.

136

Hmmmm. Perhaps there should be at least some talk about how to guide teams in using the dfinity brand name in their projects. I understand that a lot of projects would benefit from the exposure, and we’re still early in ecosystem adoption, but at some point having dfinity in their name will start back-firing. At least ask the teams to clearly differentiate between IC / dfinity and state their affiliation or lack thereof …

137

The name has been changed to fungible-token-standard :slight_smile:
v0.5.0 is released
Major Updates.

  1. Blockchain support for Token, integration of the ledger design, interface optimization design
  2. TokenHolder type changed to AccountIdentifier to ensure that Principal receipts and default AccountIdentifier are consistent
  3. Use bincode serialization to optimize the size of stored data
  4. Auto-scaling failure and archiving failure of the fallback policy optimization
  5. Remove nonce, use transaction hash to do better anti-duplication
  6. Auto-scaling storage optimization
1 Like
138

I had some issues with the ICP ledger interface and with your design, too, which I outlined in a reply to this thread:

Maybe you could incorporate something like this into your design?

139

Please read Question 8 and Question 9 that were considered when designing https://dft.delandlabs.com/

8. TransferAndCall vs Receiver Notify

* Question: which option is more suitable
* Consideration:
 * Notify can meet the basic notification needs. Although it cannot support better flexibility, it is sufficient to meet the transfer scenario
 * TransferAndCall provides better flexibility, but it depends on the transfer caller to fully understand the method and parameters corresponding to the call, which is not needed for most transfer scenarios
 * For security reasons, For security reasons, do not call the canister of the location inside the canister,[why?Inter-canister calls](https://www.joachim-breitner.de/blog/788-How_to_audit_an_Internet_Computer_canister)
* Solution:
 * Neither is supported

9. ApproveAndCall VS TransferAndCall

* Question: We compare ApproveAndCall and TransferAndCall. ApproveAndCall and TransferAndCall are two sets of non-atomic operations, there is no difference essentially. Which one should be retained?
* Consideration: In some scenarios, when multiple Tokens need to be transferred at the same time, TransferAndCall can not meet such needs. After approval, execute transferFrom in the final call to pay multiple tokens at once
* For security reasons like Q8
* Solution: call is not supported

We are very familiar with ERC20 and its optimized version. In the initial DFT Standard design, we also referred to these (in the v0.1.0 version, there was a design of notify), and for security reasons, all these designs were finally deleted.

Why we remove it? please check it here Inter-canister calls

140

I get that it’s hard to correctly handle inter-canister calls, but in some instances, they just have to be made, if you want to have any functionality that doesn’t rely on users issuing multiple transactions to achieve a single task. Especially if you want a service-side guarantee that a multi-step process will finish once it started, you need inter-canister calls. But if you really don’t want to support those, then your token standard is not appropriate for our purposes.

141

Hi RmbRT,

I’m leading the team responsible for the ICP Ledger and I can explain to you why there is a notification mechanism. Before I start though, I want to emphasise that most standards on the IC don’t have a notification mechanism. That is for a very good reason.

Let me explain why this is the case. There are several ways to notify a Principal about a transaction on the IC. The approach you propose is the more automatic one, which requires the Ledger to notify when a transaction has been successful. With this approach, the Ledger must be sure that the notification arrives at the receiver canister. On the IC, this requires the receiver to answer to the on_ft_received call with either a simple ack in case the handler is implemented on the receiver side, or with an error in case the handler is not implemented. What is the problem then? Well, there are two problems with this: 1) the Ledger could not get any answer from the receiver canister when it calls on_ft_received and 2) the Ledger receives a different error than the one saying the receiver canister misses the on_ft_received endpoint. Let’s talk about each one of these.

If on_ft_received doesn’t return then the Ledger canister is effectively stuck. Remember that the workflow relies on the fact that the message is received. The Ledger theoretically could not even stop the call to the receiver canister because then the payment would be lost. Note that it’s very problematic to upgrade a canister when there is an in-flight message and there is no way for the Ledger to do anything about it.

The situation is not better if the ledger receives a different error from the call, e.g. if the canister doesn’t exist or the canister exists but its queue is full. The Ledger doesn’t know and cannot know whether the canister will exist in future and even if it did, there would be no way to tell when the canister will be available and when it will be able to receive messages. The Ledger could retry indefinitely but then we have another issue: the more notification pending, the more space and time the Ledger would need.

Consequently, I believe that a notification mechanism that involves the ledger (directly), inevitably leads to issues at some point.

Needless to say, no safe standard should support notify without a well defined and tested approach.

The Ledger notify method is another approach. It is more secure than the one you proposed because it gives the responsibility to deliver the notification to the sender. Despite this, the notify method is still problematic and less convenient and we are deprecating it.

A final approach to the issues pertaining to notify is the so called fire and forget mechanism where the ledger sends the event to the canister and doesn’t wait for a response. This is possible and theoretically safer but then you end up with the same issue you have without notify. The caller must be able to call this mechanism multiple times in case the first notify doesn’t reach the receiver canister and that means that the client will need to track double-spending related information.

Note that most of the canisters on the IC work with the ICP Ledger. For instance, the CMC canister and the governance canister work just fine.

2 Likes