ICRC-45 Live DEX Data

Greetings,
starting ICRC-45, everyone is welcome to join our DeFi Working Group.

The ICRC-45 standard is conceived to standardize interfaces for decentralized finance (DeFi) applications operating on the Internet Computer. This standard aims to provide a unified framework for the exchange and representation of live market data, such as current token exchange rates, trading volumes, and market depth.

Requirements

  • work for all existing DeFi DEXes
  • support different systems - Order book, AMM, AMM with ranges
  • support different architectures - single canister and multi canister, EVM
  • allow the fastest possible refresh rate
  • ledger standard agnostic
  • compact at the cost of precision
  • canisters serve it fast without too much computation
  • practical to use

ICRC 45 quick draft:
https://github.com/Neutrinomic/wg_defi

7 Likes

Great initiative!

The repo seems to be still private though.

Let me know if you want to make the Working Group a bit more formal like the DeAI Working Group:

1 Like

Fixed now, thanks!
That would be great, hopefully other DeFi projects will join and we can do that.

3 Likes

Just a suggestion, but so that we can provide a breadcrumb as icrc activity increases:

  1. Create a new issue at GitHub - dfinity/ICRC: Repository to ICRC proposals to get your number
  2. Fork GitHub - dfinity/ICRC: Repository to ICRC proposals.
  3. Add your icrc details under ICRCs/ICRCX/
  4. Ask DFINITY to create a branch icrcX so you can file pull requests too it.

This way we can see all the forks directly from the ICRC repo.

3 Likes

Thanks for the feedback.
Yes it has a number https://github.com/dfinity/ICRC/issues/45
We are following the what the ID WG does https://github.com/dfinity/wg-identity-authentication
So we made this: https://github.com/neutrinomic/wg_defi
Sure if Dfinity wants them inside one repo as directories or subrepos, that can be done.

2 Likes

Returning Volume24 and Depth is going to be the hardest part when implementing.
The rolling 24-hour volume window is going to be similar to this https://github.com/infu/sonic_contrib_volume/tree/78cb6034896b5f1cc9c8905075cad5d0fcf721a1
The system keeping track of the Depth should be something similar and also fast.

Another way of getting this data would be to follow transaction logs and reduce them to the desired live state. The problem with this in my opinion - most DeFi canisters can be upgraded and algorithms may change, so to rebuild the live state one may need to have the exact algorithms and parameters used for every section in the tx log. Example: if fees change or a curve changes somehow at different points in the tx log it will result in wrong live state. So that probably won’t be very practical. I don’t think anyone has tried reducing their state from the tx log so far, and the probability of errors and missing information is high.

1 Like

Interesting and well organized…but it would be nice if it all rolled up to a central repo once all the WG work is done.

The CYCLES-TRANSFER-STATION team will contribute and implement this standard if possible at the final version.

Couple of first thoughts:

  • For the goal of supporting multi-canister architectures, an optional pair_data_canister field on the PairId record would let the caller know where to call the icrc_38_pair_data method for a given pair.
  • For the Depth type, bids and asks are each shown as a vec Amount, what is the vec for? if it is for different depth at different rates maybe it should be vec record { rate: Rate; depth: Amount; }?
  • What is the rate field in the Depth type, is it the latest trade rate? or current best rate? or a kind of weighted average?
1 Like

Great to hear!

The way it supports multi-canister and single-canister architectures is each canister can host 1+ pairs. The ‘list_pairs’ function just returns what pairs the target canister has.
I see what you mean. We can have another function - like index that has all pairs and provides the canister ids or callbacks to the canisters that contain pair data. Maybe it should be in another icrc standard and also contain ledgers ids and other useful metadata.

The order books these DEXes have can be quite large and not fit one call. We wanted to have something compact one can call every few seconds and not waste a lot of IC resources. That’s why the Depth is like a summary of all orders inside the canister.

bids: [Amount]; // 0.1%, 0.5%, 1%, 2%, 5%, 10%, 15%, 20%, 25%, 30%, 50%, 75%, 100%

example:
pair: ICP/ckBTC
output [2.3, 5.1, 30.2, …, 1232.2]

This would mean that there are bids to buy the token for 2.3 ICP at rates greater than -0.1% of the current price; 5.1 ICP for prices greater than -0.5% of the current price; 1232.2 ICP total bids

For basic AMM without ranges, this will be easy to do. To do it fast in a crowded order book or AMM with ranges it will require a special algo that caches amounts in buckets and doesn’t recalculate everything every request, because the order book can contain hundreds of thousands of orders.

For platforms that use a different canister for each trading pair, when a new trading pair is added, the consumers of this standard will have to somehow find the specific canister of the trading pair. If not included in the standard, the only way would be to contact/message each platform and ask. My thought is that a platform would have 1 canister that implements the list_pairs method and that would point to the rest of the canisters if it is multi-canister. Then a consumer of this standard would only have to have one conversation with each platform one time and then even if a new token comes out, be able to get the trading pair data without any contact. Is there a better way to do it? If I have 10 platforms contacting me every time there is a new token launch I’ll look for ways to automate it.

If current price here means the rate of the latest trade then if there is a bid with a rate higher(more expensive) than the latest trade, is it included in the amounts? Current price can also be defined as the halfway point between the highest bid and the lowest ask but not sure if that’s what people want.

How does it work if there is an ask (or many asks) for greater than 100% of the current price?

With this setup, as the price of a token grows, the data returned by this setup will contain less information because the granularity of the depth in this case is based on a percentage of the current price. As the price goes up, 0.1% of the price will represent a wider range of the spread/depth. So as the price goes up the granularity of the data will become less and less. If the price keeps going up, the depth will converge on the first 0.1% marker since 0.1% will represent a wider range of the depth and since the width of the range of the spread/depth is not directly correlated to the price. Do you think this is a concern?

1 Like

I agree. It’s a bit awkward for the standard to have one canister implement one of its methods and others another method. But if nobody minds, what you are suggesting is better in practice.

Right. Using the current price was wrong. It should be using the highest bid and the lowest ask. These have to be also included in the output. Thanks for pointing out!
We can also add last_trade_price and then the rate could be what you’ve pointed out = (lowest_ask - highest_bid)/2. Also called mid price. Works well when there aren’t a lot of trades but orders change.

Once the price goes up and someone calls get_pairs they will get updated info and new depth should have the same granularity. For a few moments, before they get the update, they will have to work with less granularity. I imagine clients of this function will be updating their depth every few seconds.

The exact percentages are up for debate.

1 Like

The alternative for depth:

Here is how ccxt library displays depth and most CEX APIs provide it this way.
https://github.com/ccxt/ccxt/wiki/Manual#market-depth

const limit = 30000
const response = await exchange.fetchOrderBook ('ICP/USDT', limit)


Looks like around 5000 orders is the limit. The result range is from 0% to 200%.
If we use this, perhaps its better to split into two functions. ccxt names: fetchMarkets, fetchOrderBook

Sounds good. ICRC-3 does something similar btw.

Cool sounds good, maybe we can name it mid_price to make it more clear.

I’ll try to say clearer what I mean. Lets say TokenA trades against USD, and most of the time no matter what the price is, there are 1$ bids for 1,2,3,4, and 5 dollars below the current price. If the current price is at 100$, the Depth type (as it is in the first version of this standard) would return bids: [0,0,1,2,5,5,5,…]. If the price moves up to 500$, it would return bids: [0,2,5,5,5,5,…]. If the price moves up to 1000$, it would return bids: [1,5,5,5,5,…]. If the price moves up to 5000$, it would return bids: [5,5,5,5,5,…].

1 Like

Yea this way works. We can set in the standard that order amounts must be grouped/summed by price.

Still there might be many different prices of orders so its possible that it might not fit in one call, but we can chunk it if needed opt start_after_price: Rate;.

Putting the depth (fetchOrderBook) into its own method sounds good to me.

1 Like