New Exchange Rate Mechanism

Proposal is live: https://dashboard.internetcomputer.org/proposal/76767

1 Like

How do you mange the risk of your data sources going down or being taken down? Are we dependent on an emergency proposal being adopted to get us out of a hole? Could an attacker target this mechanism?

I generally wary of remote data sources becoming dependencies.

Can we keep the existing system in place in case we need it?

2 Likes

@THLO It would be helpful to first receive an complete overview of what the current exchange mechanism looks like, something that would show all the dependencies, and maybe a flow chart/diagram to show how the exchange/price data coming from source X ends up affecting the cycles exchange rate.

From the proposal, it mentions an exchange rate canister:

“The exchange rate canister, running in a system subnet, will enable any canister to request the exchange rate between any pair of fiat currency and cryptocurrency - including pairs of fiat currencies and pairs of cryptocurrencies. It will further be possible to request rates in the recent past.”

@diegop A few follow-up questions.

  1. The proposal mentions that it will be possible to request rates in the “recent past” - how far back is the recent past? How far back, as well as the update interval has significant implications with respect to how much data might be stored in the canister.

  2. I understand the importance of this feature in terms of ICP/USD, but why is DFINITY developing and owning an exchange rate canister that will keep track of all other currencies? That puts a lot of trust and eyes on this specific exchange rate canister.

  3. Why does it make sense for DFINITY to go beyond ICP/USD and to build the tooling for what could become a cryptocurrency exchange on the IC? Does DFINITY have plans to build their own DeFi ecosystem?

2 Likes

to be perfectly honest, I submitted it on behalf of R&D, but I am not the right person to ask. @THLO is better suited.

1 Like

Thanks a lot for your questions! Let me try to answer all of them.

How do you manage the risk of your data sources going down or being taken down? Are we dependent on an emergency proposal being adopted to get us out of a hole? Could an attacker target this mechanism?

We already have this risk in the current mechanism: Our conversion rate providers periodically query data from exchanges and market data providers. Since there is a lot of redundancy (every source is queried every time), an attacker would have to take down or corrupt many data sources. More precisely, the conversion rate providers currently get the ICP/USD rate from 7 different exchanges and select the median rate. They further query 3 market data providers to get the current USD/XDR rate and again select the median rate. We currently have paid subscriptions for the USD/XDR sources, so a lower number of sources is deemed acceptable.
Since we will only make use of (free) public sources in the new mechanism, we plan to query Forex data from numerous (more than 3!) sources.

In short, targeting the new mechanism won’t be easier than targeting the current mechanism.
It is correct that we have to monitor the process and create a proposal to upgrade the canister if we want to add or remove data sources.

It would be helpful to first receive an complete overview of what the current exchange mechanism looks like

Once the median rates are selected as described above, the two rates are multiplied to obtain the current ICP/XDR rate. Using a round-robin mechanism, one of the conversion rate providers creates a proposal and the others simply vote, accepting the proposal if the proposed rate does not deviate much (at most 1%) from the locally computed rate.
That’s the mechanism in a nutshell. If you think it would help, I can post a diagram and add more details.

The proposal mentions that it will be possible to request rates in the “recent past” - how far back is the recent past?

The idea is to store the daily Forex rates (one data point for each supported fiat currency) for some time, possibly one year. In this case, the amount of data that needs to be stored is 365 times the number of fiat currencies times the size of the rate data structure.

Note that cryptocurrency rates are queried on demand, so the canister only stores very little data.

why is DFINITY developing and owning an exchange rate canister that will keep track of all other currencies?

As mentioned above, the exchange rate canister will not “keep track” of other currencies, it will merely provide an interface to query exchange rates. Developers are free to use the HTTP call feature directly.

Why does it make sense for DFINITY to go beyond ICP/USD and to build the tooling for what could become a cryptocurrency exchange on the IC? Does DFINITY have plans to build their own DeFi ecosystem?

It is not meant to become a cryptocurrency exchange. It simply offers a convenient way to access exchange rates. We could restrict it to just provide the ICP/XDR rate but we can get the functionality to query the rate for any currency pair basically for free.

It may appear that the exchange rate canister offers a lot but, in my view, it is just a generalization of the required system functionality to maintain the ICP/XDR rate (which will still be stored in the cycle minting canister).
DFINITY does not plan to build any applications on top.

6 Likes

Got it, thanks for the explanation.

If the exchange rate data is not kept and “cached” on the canister, could this open up the exchange rate canister to a cycle drain attack?

According to some initial feedback from DFINITY it looks like HTTP calls are expensive (400M cycles per call). Since the exchange rate canister would be the one paying for the HTTP calls, how do you plan on keeping it topped up and preventing cycle drain overuse?

3 Likes

First of all, the plan is to cache requested rates but only for a relatively short time (around one minute). So, when I said that “the canister only stores very little data”, I made the assumption that there won’t be too many requests for different rates simultaneously. In any case, there will be a maximum number of cached rates to bound the maximum canister state size.

The caching mechanism mainly serves to provide quick answers if the same rate was requested recently and to conserve bandwidth. A cycle drain attack won’t be possible because the exchange rate canister will run on a system subnet, i.e., it will be exempt from paying cycles.

4 Likes

Note that the pricing referred to here is provisional and pretty conservative (= expensive). We are currently working on an updated pricing model which will also determine the price for the HTTP outcalls feature and it is likely that it will become cheaper in this new model.

5 Likes

Will this be true for any canister that is deployed to a system subnet? Is it possible to have a canister on a system subnet that does pay cycles?

How this lazy loading can work in general? If I want to use the “free” exchange canister from an app subnet:

  1. Canister A requests an exchange rate in a query call which will fail first. Who should initiate the update call to load the data? And will the request fail again, after the cache expires?
  2. If the request should be an update call in the first place then cross subnet update call will be slower than a direct http call (data is heavily cached on exchange, takes just a few milliseconds) because 2 subnets need to finalize instead of one.
1 Like

Enabling/Disabling cycles accounting is a subnet-type wide setting, so (at least at the moment) it is not possible to have a canister that consumes cycles on a (currently only one, but we could have more) system subnet.

To see how it works in practice, you can run your local replica in system mode as well. Here are the lines to configure it that way.

5 Likes

The exchange rate canister will charge cycles for its service.
While it just makes sense to pay the canister for its service, it is further an effective mechanism to prevent spam.

Since we currently don’t support cross-subnet query calls, the plan is to use update calls for the time being.

You are right, these calls may be quite a bit slower than using the HTTP call API directly. There is a trade-off: If you use the exchange rate canister, you get a rate derived from inputs from many sources, so the effort to get accurate rates is low. However, you have to accept a higher latency. On the other hand, if you use the HTTP call functionality, the latency will be lower but you need to implement (and maintain) the code to compute accurate rates yourself.

4 Likes

Thanks, makes sense, I didn’t consider the multi exchange scenario (however such aggregation services probably already exists).

IF cross-subnet query calls would be possible

  • what would be the flow?
  • the canister would be designed to use that?

With the http request feature in theory it is possible to make a query call (via http) to another subnet, right? Of course strictly until the official way became supported :slight_smile:

1 Like

There is no design for that at this stage, so these are just my current thoughts.
The flow could work as follows: The exchange rate canister certifies the rates that are currently cached. A query call would return the cached and certified rate as long as the rate is valid. Once a rate expires, a query call would return an error message indicating that there is no valid rate in its cache.

We might also consider fetching the rate if it is not cached as part of a query call instead of returning an error. However, there are two problems that need to be considered:

  1. This option only becomes feasible once we are able to charge for query calls.
  2. There is no way to certify the rate, so there is a risk that a malicious replica returns a wrong rate.

With the http request feature in theory it is possible to make a query call (via http) to another subnet, right?

This is not possible in practice because the firewall rules disallow it. It is probably safer to wait until there is a proper solution for cross-subnet query calls anyway.

3 Likes

Thank you Severin. I feel like I learn something new about the IC every day :slight_smile:

1 Like

Once heartbeat becomes cheaper, I think it may be worth exploring caching query results again.

That could actually save developers cycles in the long run, as most clients are probably interested in querying the same rate anyways. The cost of prefetching via heartbeat is therefore amortized across the clients who need that prefetched data.

Since the NNS proposal got accepted, it is time to go to work! :slight_smile:

We could actually use your help: We need reliable, trustworthy Forex data sources that are accessible via IPv6 and that provide end-of-day rates for the most commonly used fiat currencies, ideally providing the rates in a JSON structure.

As it turns out, it’s not so easy to find such data sources. After searching the web for a while, we found the following free, IPv6-enabled Forex data sources:

If anybody is aware of (or manages to find) additional/better Forex data sources, it would be great if you could post links in this thread!

3 Likes

Far from JSON but Hungarian National Bank (central bank of Hungary) has a SOAP service and IPv6 address (agree, surprisingly rare).

2 Likes

Thanks a lot for taking the time to look around! We will check if we can use it.

Luckily, we did manage to find some additional sources in the meantime but we’re still looking for more.

2 Likes

One more question: I’m confused why are you looking for IPv6 services if the http call uses IPv4 boundary nodes?