We need a DeFi subnet

think solutions to these problems exist within the IC without a need for a subnet that could fragment the attention and resources. Shouldn’t the focus be on improving the IC now rather than making it into another Ethereum?

lack of atomicity: Most real world financial transations are non-atomic. In fact Ethereum transactions are one of the few things that are synchronous. Synchronous blockchain are a limit to scalability. Yet the world is a distributed super-computer not just one thread of execution.

No history transaction records: Some form of scalable storage on the IC would solve this issue, why not implement this for the IC and improve the whole IC experience, rather than just a subnet. Lots of apps need scalable storage.

Agree on the async issues, we really need safe async calls, and why doesn’t this have a timeout for safety?

I think the other issues are challenging but they can be worked out, and will strengthen the IC. Build one IC, stronger together and less so with fragmentation. DeFi on the real IC and DeFi on the subnet will surely fragment the ecosystem.

11 Likes

In a later release (Carbon), as also mentioned in a post above, we plan to enable atomic swaps to better enable DeFi.

Could you explain how you plan on enabling atomic swaps? My understanding is that the asynchronous programming model inherent to canisters (i.e. cross-canister calls need to be manually rolled back) prevents purely atomic swaps.

1 Like

This one is a head scratcher to me. From what I’ve seen the only thing standing in the way of de-fi is proper architectural imagination and implementation. We have a piece of general purpose computing that can be programmed to do pretty much anything. There are a few things we still need exposed in motoko, but all the pieces are there. We need good tooling, standards, and patterns…but none of those require a new subnet at this time. Just hard work and imagination.

If you want transaction history, build it into your canister, merklize it, and publish an endpoint. If you want atomicity then create a canister where that can happen and interact with it.

If you want an EVM, build it. There are 0 impediments from building an EVM in a canister today. You just have to write keckkak in motoko, a Patricia tree in motoko, RLP encoding in motoko, write the op codes in motoko, and the precompiles. The list is long, but you can get paid to get started: ICDevs.org - Bounty #16 - EVM Witness Library - $8,000+

De-fi is coming. But what comes after will make you forget about it.

15 Likes

One possible mechanism to implement this would be 2-phase commit. This can be implemented as part of the execution layer of the IC. In an example scenario, we would make 2 calls to other canisters and
only if both return with success, we send a commit, otherwise we abort.

There is another idea around that would use orthogonal persistence to address the issue: Only once both transactions are committed, the state changes are persisted. However, this has the huge drawback that an actor cannot process other messages until the state change gets persisted, so this does not really scale in my opinion.

2 Likes

Agreed, you can do most part of an EVM as an IC smart contract. However, it becomes tricky when we want to expose an Ethereum JSON RPC API to the outside world. This requires extensions to parts of the IC in order to accept ordinary HTTP(S) calls as update calls. Currently this only works for queries. However, an extension of the icx-proxy is currently being built that will allow a canister to “upgrade” a HTTP request from a query to an update call. Once this is in place, a fully EVM-compatible EVM can be built as a canister or set of canisters.

One thing we clearly need to put more attention on is developer experience, guidelines, and best practices. This would likely solve many of the problems addressed in this thread. We recognize this deficiency and are currently putting some efforts into this. Specifically for DeFi, concrete examples, guidelines, and best practices will help the community a lot to write DeFi dApps on the native programming model of the IC.

3 Likes

This got merged a couple of months back:

There is some uncertainty around whether is functionality is live yet. I’ve been focused on other things and haven’t gotten around to trying it on the IC proper.

2 Likes

I’m not a great fan of Cardano but I wonder if it might make sense to borrow their idea of having an distinct asset settlement layer: Such a layer would

  • Hold ledger information about ownership of assets.
  • Hold Tx History
  • Execute transfers atomically e.g. by:
    • Checking pre-conditions. (existence, ownership, signature, agreement on pre-and post state)
    • Package all changes as Atomic commit. (exact mechanism ideally lock free TBD)
    • Checking post-conditions. (Actual transfer resulted in expected balance changes)
    • Success or rollback notification.

Rationale:

  1. Makes it easier to reason about ownership of assets.
  2. Atomic swap logic (pre and post condition checks) could be cleanly enforced at the interface.
  3. The settlement layer / computational layer distinction is likely to happen anyway due to BTC and Ethereum integration, so pattern is likely to evolve anyway and it is nice to have common way of doing all transfers regardless of whether they are native or non-native.
  4. Settlement layer can be optimised as pure settlement layer. In terms of performance, security, decentralisation, (optional) privacy and (optional) traceability and so on.
  5. Enforces standard and can be integrated at lower level.
  6. Governance is not necessary for a pure asset ledger. We can thus make it easier to reason about security risks of governance, as asset ownership would be cleanly separated from the ability of owners/SNS to upgrade canisters. The NNS itself could even be excluded from arbitrarily altering asset distribution.
  7. Storage costs of ledger entries could be free (with fee levied per transaction) ensuring that asset balances are not at risk of being erased if canister cycle balance falls to zero.

I imagine that the asset settlement layer would thus operate a bit more like a traditional blockchain, and could be designed to give people confidence that it is a rock solid extremely decentralised, immutable and censorship resistant settlement layer, while more advanced capabilities would exist in other layers.

2 Likes

Thank you for your thoughts on the settlement layer. Apologies for not coming back to this earlier, but I was too diverted with other work. When now trying to grasp the full understanding of what you had in mind, I do need to ask some clarifying questions back to you, hope you have some time to clarify where needed. The topic you are talking about may be core to DeFi on the IC.

Questions:

  • This settlement layer would host all assets on the IC, i.e., all fungible and non-fungible tokens?
  • When you talk about atomic transactions and atomic swaps, this particularly includes transactions between different tokens (ledgers) and tokens from different chains? If you could elaborate a bit more on what your scope here is, this would be extremely helpful.
  • In (7) you talk about canisters running out of cycles. Is your intention that the settlement layer be implemented in user space as canisters? Or do you have in mind to implement such settlement layer as an extension / modification of the IC stack? Some more insights into your thought process may be very helpful for getting a better view.
  • Assume we implemented a protocol that allows us to realize atomic transactions over multiple ledgers, would this be something that would realize (part of) the functionality of this settlement layer? I have been talking to some people about 2-phase commit and how we could realize this in the IC.
2 Likes

I was mainly thinking about fungible tokens. Simple Ethereum ERC-721 (single) and ERC-1155 (collections) like NFTs could be handled as well. But their metadata would be restricted to being a simple pointer to another canister so that all complex logic for content and so on would be hosted on another subnet.

I was mainly thinking about swaps within the settlement layer. But some kind of abstraction layer which allows a similar interface to be called atomic swaps between IC tokens and tokens on other chains would be very useful, though the logic there is likely to be different due to different latency and security assumptions.

Yes so the IC has a reverse gas model whereas most token or DeFi orientated blockchains have a fee / gas model. I’m advocating for the settlement subnet to have a fee model because:

  1. It is unacceptable for ownership records to be wiped because someone failed to pay the gas fee.
  2. Canisters running on normal subnets can pay fees for settlement layer transactions so users don’t actually have to think about gas.
  3. Without any form of fees we open the door to spam.

In terms of how it could be implemented. The easiest way short term is just to copy the ICP ledger subnet which AFAK operates in a zero cycle cost manner. Then add fees to message it. Long term I would see the settlement subnet having a radically different design for example a zero knowledge token ledger similar to Aztec

Atomic swaps within the settlement layer would be by a 2 phase commit mechanism built into the settlement subnet. For swaps with other chains I’m thinking of this as a separate feature of a convenience wrapper library built over the settlement layer.

1 Like

I still don’t understand why all of this can’t be done with the current canister configuration.

Deposit an NFT or tokens to a canister, do atomic swaps, Withdraw tokens or NFT.

Once these can be wrapped on the IC with ECDSA you can do the same as above.

Just add fee code to your swap canister. If the ICP is deposited on your server you control it and the contract can take fees.

Again…you can do a two phase commit with a canister today. You can even add heartbeat and have withdrawals sent automatically back to the source chains.

I’d be happy to do an architecture session where we hash this out so I can better understand what the perceived issues are.

7 Likes

Dear forum members!

Triggered by, among other things, the discussions in this forum topic, we decided to start a DeFi initiative for the IC. The higher intent of this initiative is to help the Internet Computer and its community grow and be successful. The concrete goal of the initiative is to fully enable DeFi on the IC. We think a good approach to tackling this is as follows:

  • Investigating what is missing for fully enabling DeFi on the Internet Computer, particularly also by considering community use cases and their requirements.
  • Proposing different options on how to proceed w.r.t. DeFi on the IC.
  • Agreeing on a roadmap for fully enabling DeFi on the IC.
    Once we have an agreed-upon roadmap of prioritized features, we can execute it and implement its features, thereby removing existing barriers to DeFi on the IC.

The forum (you!) will be a major source of input for this initiative – in the end we need to enable you to build the dApps you have in mind. We intend to intensify the discussions here so that everyone who has an opinion can speak up and voice their opinion. We are also planning to run a workshop with the community to go into further details regarding the major issues to address as part of this initiative and get an aligned view on it.

As the IC is an asynchronous system, the architecture of DeFi dApps on the IC will have some inherent differences to that on a synchronous system (unless the community builds an EVM for the IC :wink: ), please also keep this in mind. Knowing about those differences will also be instrumental for a deeper understanding of how DeFi on the IC is best done.

Here is a current list of issues we have seen mentioned so far:

  • Token standard (WG launched)
  • DeFi on asynchronous architecture
  • Inherent limitations of asynchronicity
  • Atomic multi-canister transactions, 2PC
  • Token standard
  • Transaction logging facility
  • Best practices, HOWTOs, examples
  • DEX patterns
  • Limitations of asynchronous architectures for DEXs
  • Dedicated DeFi subnet
  • Stablecoin(s)
  • Asset settlement layer
  • Integration with Web 2.0: direct HTTP calls from canisters and oracles
  • Integrations with other blockchain networks
  • X-Chain DEXs
  • Making the IC the global DeFi hub
  • EVM on the IC
  • Ledger DoS protection
  • Ledger scalability

Thank you already now for further participating in the discussions!

19 Likes

Whoa, this is a lot of stuff.

I think it’s a great idea to set up this initiative. It will be difficult to move forward on some of these issues without leadership from DFINITY, in my opinion.

5 Likes

Indeed, this is a long list of potentially relevant / interesting things. The idea is to use this as a starting point for discussions with the goal of figuring out what we need to focus on – we have limited resources that we need to spend in a best-possible manner. The DeFi initiative is something that will have a defined end, so we need to distill a list of features we want to build as part of this initiative in order to improve DeFi on the IC. Input from the community will be crucial to get there.

Maybe just asking to existing teams developing DEX Dapps like Sonic and others would be more to the point than opening a discussion in the forum

2 Likes

Good point! We plan to have discussions with those stakeholders. But it’s important that everyone can voice their opinion in order to get a comprehensive picture and not limit to people already having a large DeFi project going.

2 Likes

This sounds great! I am planning to invite for a DeFi workshop to discuss various aspects of DeFi. Maybe this “architecture session” could be part of this workshop.

1 Like

World Computers and Real-World Computers for DeFi

Let us discuss the challenges related to atomic transactions over multiple canisters on the Internet Computer. There seem to be two camps of people here: Those who think that atomic transactions offered as a system service are required for realizing certain use cases, and those who think they are not. My personal opinion is that the more you think about it the more you converge towards the second line of thinking. Let me elaborate on this.

When coming from an Ethereum-biased world, one clearly will want transactions also on the IC. That’s just how the world must work. It is true that the Ethereum world, or the one of any synchronous blockchain, works like this: You have essentially one thread of execution and this makes it easy to devise a system with global world state and transactions over a sequence of actions.

The synchronous world (EVM)

In this synchronous world it is trivial to implement things like swapping tokens within a single transaction without ever needing to take custody beyond the duration of a single transaction of one side of the tokens of the swap. Also it is easy to implement a DEX where one can swap a token A to a token D, even if there exists no trading pair (A, D), by “routing” via intermediate tokens B and C when trading pairs (A, B) (B, C), and (C, D) are listed. And by packing all the trades required for swapping A to D into a single transaction, the use gets the guarantee that either the full chain of swaps is executed according to the user’s constraints, or it is not executed at all. In the case it is not, the user only incurs gas fees, but has no other risk, e.g., ending up with tokens B or C.

Also, flash loans can be implemented in such a world with ease if one understands the execution environment like the EVM sufficiently well: A user (or their smart contract acting on their behalf) can borrow funds from a liquidity pool, earn money by arbitrage trading with those funds, pay back the loan with interest, and still earn some profit. All of that is happening within a single transaction, i.e., if anything goes wrong, e.g., money is lost during the arbitrage attempt, the transaction can be aborted, the loan has never been issued, the arbitrage loss never incurred, and thus there is no risk for the user. It’s a little time machine. That is, flash loans allow for doing financial transactions in a riskless fashion, which would not be possible in the real world. Only the gas fees for the transaction would need to be paid by the user.

Use cases like multi-hop swaps in a single transaction or flash loans are not possible in the real world. They are only possible in an artificial environment like an EVM-based blockchain because such environment comprises essentially the equivalent of a single computer realized as a replicated state machine, with a single execution thread and a global state space. The protagonists of this model (implicitly) postulate that a single computer is enough for the whole world. I.e., in terms of scaling you can only scale up, not out. If you would ever need to scale beyond the boundaries of what can be done within a single single-threaded machine, you would have to leave the model of synchronous execution, because your world state and execution would be split across more than one machine. This would mean to embrace asynchronous communication between those shards and thereby moving towards an asynchronous architecture and leaving the realm where multi-hop transactions and flash loans can be implemented as outlined above.

The asynchronous world (IC)

An asynchronous architecture like the IC comprises many independently-operating subnets where each subnet is its own blockchain. All state-changing invocations between smart contracts are asynchronous, incurring short latencies for intra-subnet calls, and longer latencies for inter-subnet calls. Welcome to the asynchronous model of execution for blockchains!

In the asynchronous model of the IC you currently do not have inter-canister transactions that span canisters of the same or different subnets. You neither have those in the real world. Thus, the IC still closely models what you can do in the real world and the real-world financial system. So almost every financial service of the real world will be implementable on the Internet Computer. Exceptions will be highly-specialized things like high-frequency trading which will not be implementable in the same way as in the real world just because of a blockchain having a block time of one or more seconds and HFT operating on a very different timescales on real-world centralized exchanges. And even less so it works in the same way as in the real world on EVM-based blockchains with their higher block latencies.

Let’s now look at a common example of a real-world service being implemented on the IC: A decentralized exchange as the blockchain equivalent of a centralized exchange. In an EVM-based blockchain an exchange is mostly implemented through the mechanism of an AMM for the reasons of compute and memory efficiency, as both compute cycles and memory is scarce in the EVM world (Remember, the world must share one computer for all their computational needs!). On the IC, one can easily implement a full-blown order book exchange that much more closely resembles the real world of financial services than an AMM. Notable advantages are that it resembles the concept of two parties exchanging assets with each other, which is the core concept behind an exchange, as well as slippage losses not going to arbitrage traders, i.e., this premium of the AMM world not having to be paid by the users.

Let’s next have a look at the more concrete implementation of the actual asset exchange: Different users (or market makers in the traditional sense) have placed orders and thereby created liquidity in the order book that others users can now trade against. This liquidity needs to be under custody by the exchange, that is, the corresponding assets (tokens) have been transferred to the exchange either using a direct ledger transfer or by the user allowing the exchange to transfer the tokens itself. Either way works, one constraint is that the order can be listed in the order book only once the tokens are under custody of the DEX. The approve-way is the one known from ERC-20 and the EVM world and is maybe the more convenient one.

Now let’s assume a user wants to place a spot order, i.e., buy tokens at market price. Again, the user can send their tokens via a transfer call to the corresponding ledger or approve the DEX to make the transfer on the user’s behalf. Once the tokens are on the DEX’s account, the actual exchange can take place: The DEX code matches the user’s order with the order book and finds one or more matching orders. It then computes how much of the target asset the user receives and updates internal accounts to update the state w.r.t. the swap. The important thing is that the actual exchange can happen atomically (in the scope where it matters, the rest of the world still moves on asynchronously at the same time, but this is not a problem) because the DEX canister can execute transactions atomically. The user can request the payout of the target tokens at any point or place further trades with them. This is just how an exchange in the real world would do it. An implementation of an order book exchange has been presented in yesterday’s DeFi workshop by DFINITY’s Tim and Eero.

This example shows that there is no need for system support for atomic transactions over multiple canisters to realize an exchange that closely resembles a real-world exchange as this can be done by regular canisters without any issue. It’s important to note that there is a difference to how things are done in a typical EVM-based DEX implementation, but we can implement a much more real-world exchange than on most other chains. Importantly, there is no weakening of the trust model by the DEX having custody of the funds for some time as it is a smart contract that people can publicly audit and that they need to trust anyway when using the DEX.

If one wants, an AMM-based DEX can be implemented easily on the IC as well that is much closer to what you know from Ethereum than an order book DEX. Let’s have a look at how a swap can be implemented here. Let’s assume the liquidity pool has been populated with tokens by liquidity providers, those tokens are under custody of the AMM canister much like they would be under custody of the AMM smart contract on an EVM chain. When a user now wants to make a trade on the AMM, there are two options how this can be implemented:

(1) The user makes a payment to the AMM first which brings the funds under custody of the AMM for a short duration and then places the trade. Once the trade is executed against the liquidity pool, the AMM computes the amount of the target asset the user receives and initiates a payout of the target tokens to the corresponding token ledger. The actual trade is computed in an atomically-executing thread of execution of the canister up to and including the initiation of the payout of the target tokens.

(2) The user approves the AMM to make transfers on their token account on the ledger of the source token of the trade. Then the user places the trade, the AMM initiates the transfer, and once the tokens have been credited to the AMM, it executes the trade as above atomically and initiates a transfer of the target tokens back to the user.

This model is very similar to how it is done on EVM chains, main differences are the following:

  • Token transfers are asynchronous and incur a latency, multiple seconds for Xnet transfers.

  • The AMM has custody over the source token for a short period until it initiates the transfer back out. We do not use the concept of a system-level transaction. We achieve the same outcome with the AMM on the IC as with one on an EVM chain, but with a slightly different implementation. I would like to claim that having the tokens under custody for a short time by the smart contract is not a practical issue that would make the model less powerful or less DeFi. It does not make any difference to the user.

Looking at DEXs where you can swap a token A to a token D via tokens B and C as further above on the Internet Computer: This cannot be directly implemented as in the synchronous world as we do not have those global transactions that could be reverted. That again reflects the real world. The implementation on the EVM as an atomic transaction is only possible because of the unrealistic assumption that the world’s DeFi can be handled by a single computer with a global state space and single-threaded execution, which, of course, does not scale at all.

But we can also implement this on the IC, just in a way that more closely resembles how this would be done in the real world. Options are:

(1) Implement the currency pairs all within a single canister and do it atomically therein. This does not work for Inter-DEX-multi-hop swaps of course and constrains scaling out.

(2) Just execute the swap optimistically with the risk of it getting stuck in an intermediate token the user does not want. That’s how the real world works for this scenario.

(3) Layer additional financial instruments on top as suggested in private communication by @timo which allows the user to insure themselves against volatility for a fee. This models also how the real-world financial system could address this.

We can see that also this use can be handled on the IC, even in a way much closer to how the real world works.

Inter-canister transactions

I do not want to claim that inter-canister atomic transactions across subnets would not be helpful in certain circumstances. E.g., they might help simplify the implementation for canister authors, but they come with a major complication: An atomic transaction that spans multiple canisters on different subnets needs to “lock” the assets concerned in the transaction, thereby incurring an opportunity cost for the canister holding the asset. This opportunity cost would need to be factored into the transaction model as a fee that the initiator of the transaction would need to pay. This fee would need to cover the opportunity cost in order for a transaction model to make sense from an economic perspective. This will make transactions (much) more complex than on EVM chains.

An inter-canister transaction could be implemented by a canister that offers it as a service. The canister could be run by a DAO. The security properties of this would be essentially the same as for a system-level transaction offered directly by the IC. But a canister might be the better place to implement the economic semantics of the transaction outlined above.

Conclusions

From the above elaboration, my hypothesis is that most interesting scenarios can be implemented on the IC in a way much closer to how the real world works. Some things, like flash loans are inherently not implementable the same way they work on EVM chains, just because this contradicts inherently the properties of the asynchronous architecture that the IC is. The question of whether we need transactions as a system service is a secondary one: They can always be implemented by a canister and offered as a service to users. I am looking forward to a discussion with you on this!

26 Likes

All your posts are jewels. Thanks a lot @dieter.sommer :pray:

1 Like

Hey community, its been some time since this post. Do you have enough resources now to create DEFI dapps on ICP or do you feel you need more.

1 Like