There is no design for that at this stage, so these are just my current thoughts.
The flow could work as follows: The exchange rate canister certifies the rates that are currently cached. A query call would return the cached and certified rate as long as the rate is valid. Once a rate expires, a query call would return an error message indicating that there is no valid rate in its cache.
We might also consider fetching the rate if it is not cached as part of a query call instead of returning an error. However, there are two problems that need to be considered:
This option only becomes feasible once we are able to charge for query calls.
There is no way to certify the rate, so there is a risk that a malicious replica returns a wrong rate.
With the http request feature in theory it is possible to make a query call (via http) to another subnet, right?
This is not possible in practice because the firewall rules disallow it. It is probably safer to wait until there is a proper solution for cross-subnet query calls anyway.
Once heartbeat becomes cheaper, I think it may be worth exploring caching query results again.
That could actually save developers cycles in the long run, as most clients are probably interested in querying the same rate anyways. The cost of prefetching via heartbeat is therefore amortized across the clients who need that prefetched data.
Since the NNS proposal got accepted, it is time to go to work!
We could actually use your help: We need reliable, trustworthy Forex data sources that are accessible via IPv6 and that provide end-of-day rates for the most commonly used fiat currencies, ideally providing the rates in a JSON structure.
As it turns out, itās not so easy to find such data sources. After searching the web for a while, we found the following free, IPv6-enabled Forex data sources:
As @dieter.sommer wrote, using boundary nodes for the HTTP call feature is just āone avenue towards enabling IPv4 connectivityā. By default, the replicas will attempt to reach the targeted service directly, that is, over IPv6.
Since we aim for maximum decentralization, the goal is to use as many IPv6-enabled data sources as possible.
Hi all,
FYI after a discussion with @THLO, I did a short analysis on the interplay of the new exchange rate mechanism and financial incentives for spam. In particular I quantified how much ICP an individual (or a group) needs to hold in order to be able to offset the rejection cost of 10 ICP by increased rewards due to submitting one spam governance proposal.
Used input:
Based on historical data we have on average approx 0.5 governance and 6 non-governance proposal per day (in addition to 144 exchange rate proposals).
For the staked ICP by the (group of) spamer(s) I assume a dissolve delay bonus of 2.
Outcome
Required ICP to be able to offset
pre new exchange rate
post new exchange rate
weekday
307,166
256,985
weekend
68,178
17,572
Conclusion
The new exchange rate mechanism increases considerable the financial incentives for spamming, in particular on week-ends.
It would desirable to implement a spam prevention mechanism in parallel with the new exchange rate mechanism.
Are you sure this is really going to be an incentive for spam? It still seems unlikely to me with the current Governance proposal weight of 1x. What was the equivalent analysis of financial incentive for spam when the Governance proposal weight was 20x? Would it be accurate to say 68,178/20 = 3409 was the ICP required to offset the proposal reject fee on the weekends pre new exchange rate? Thatās a lot lower than 17,572.
If we see spam, then a proposal reject fee of 38.8 ICP would be equivalent disincentive to spam compared to what exists pre new exchange rate. Instead of going straight to that proposal reject fee, it could be slowly increased until the spam stops. We could even advertise that spam will cause an increase in proposal reject fee so any donors to a spam funding campaign know up front that their financial analysis of the spam opportunity will change during the spam campaign. That seems like a deterrent as well. I would be willing to submit proposals to increase the proposal reject fee to avoid more spam.
People submitting register known neuron proposals is also a disincentive to spam because they are legit Governance proposals. There have been a lot of them lately and I see no reason why the community shouldnāt continue encouraging registration. In fact, Iād be willing to offer my services for submitting these proposals for anyone who is not comfortable with DFX command line yet still have an interest in participating in governance at a higher level.
My main concern is rushing into a spam prevention solution that goes beyond the tools that are already built into the NNS and what has already been deliberated and voted. I know DFINITY has been working on this exchange rate change, but Iām not aware that any activity has been occurring on spam prevention solutions. Hence, coordinating exchange rate change with spam prevention seems rushed on the spam prevention side. Perhaps activity has been in progress that hasnāt been released publicly yet. In that case, Iād love to hear more about the plans.
This plus the re-selection every six months gives us a broad-based solution, but my understanding is that the re-selection is going to take a bit longer than this would.
The according numbers for governance weight 20 are 92k ICP (weekdays) and 20k ICP (weekend). So in particular for weekends the numbers are quite similar post the implementation of the new exchange rate mechanism (17k vs 20k). For your reference, the underlying calculation is here.
I have done the analysis above a week ago (there was no further activity within DFINITY on this during the summer apart from the tactical fix for setting the reward weights back to 1). Given these results, we wanted to point this out and trigger the discussion on spam again.
Now that the HTTPS outcall feature is being rolled out, you may be wondering about the status of this feature.
In short, the implementation effort is progressing nicely. We should have a working canister and start testing it extensively in early October. If everything goes according to plan, the roll-out should follow soon thereafter.
At first, we will simply stop creating exchange rate proposals. Once weāre sure that everything works reliably, we can then proceed to remove support for exchange rate proposals from the NNS.
Any update on the feature? I am working on a protocol which uses the old XDR exchange rate to quantify risk on borrows, as such I would like to have an idea of an ETA so I can schedule an upgrade to the protocol canisters without interrupting users or corrupting the state.
Sure, I can provide an update. Itās been roughly two weeks, so thanks for the reminder!
There are a few minor things that need to be implemented but overall the implementation is nearly complete. My expectation is that we can focus on getting the canister ready for production at some point next week.
However, there is a risk that the roll-out will be delayed: Since weāve been struggling to find sufficiently many data sources (including cryptocurrency exchanges!) that are accessible over IPv6, we are exploring ways to interact with IPv4-only sources as well to make sure that the exchange rate canister offers at least the same level of security as the current exchange rate mechanism.
One option that is being discussed is the use of public NAT64 servers for IPv4-only sources via custom DNS resolvers in HTTPS outcalls. Since there is no way to set a custom DNS resolver at the moment, a change to the HTTPS outcall feature would be required.
The plan is to agree on a way forward early next week. Iāll provide more information about the expected timeline as soon as I can!
Just a quick follow-up to last weekās update: After some more discussion, using custom DNS resolvers to reach more data sources via NAT64 servers is still considered the best (short-term) option.
Work on this task will hopefully begin soon. In the meantime, we will continue to make the exchange rate canister production-ready.
Itās time for another update!
After more rounds of discussion, we decided to proceed with a conservative roll-out plan. Unfortunately, the plan somewhat contradicts what I posted earlier but we hope that everybody agrees that being cautious is the right approach for such a critical feature. The plan is the following:
A fully functional exchange rate canister will be released on the w4rem system subnet for testing purposes. We call this the beta release, which is planned for mid November. We have to wait with the beta release until the targeted subnet is upgraded to a replica version that incorporates the latest changes to the HTTPS outcalls feature. Once the exchange rate canister is released, everybody is free to use and test it! However, there is no guarantee that this beta version will be around forever.
After an initial round of battle-testing, the exchange rate canister will be released on a larger system subnet (the current plan is to use uzr34 but there is no final decision yet), which will be the main release. It is important to note that this canister will have a different canister ID. There is no fixed date for this release because it heavily depends on the experience with the beta release. The current expectation is that the main release will still happen this year.
In parallel to the activities above, a secure and reliable solution for IPv4 connectivity will be developed, which may not use NAT64 servers as proposed earlier. The exchange rate canister (beta and main release) will initially rely on IPv6 data sources only. Once IPv4 data sources are supported, the exchange rate canister will be upgraded to make use of additional data sources to improve decentralization.
After having battle-tested the main release (with IPv4 data sources) for some time without any major issues, the cycle minting canister will be upgraded to make use of the exchange rate canister to set the ICP/XDR rate. At this point, the current exchange rate proposal mechanism will be deactivated. However, there will be code in place to quickly fall back to the NNS proposal mechanism if any issues are observed. Ideally, there wonāt be any more conversion rate proposals at this stage. The timeline for this step depends on the experience with the main release.
After having monitored the new mechanism for some time without any issues, the cycle minting canister can be upgraded to only accept the rate from the exchange rate canister, making the conversion rate proposals obsolete. Since there is no rush to completely remove the current conversion rate functionality, this last step will not happen until later next year.
I hope that I managed to shed some light on the current plan. Please let me know if you have any questions or concerns!
Iām happy to report that the beta version of the exchange rate canister is now live!
The canister ID of the beta version is gvu7d-7aaaa-aaaan-aaaba-cai.
This is the content of the Candid file:
type AssetClass = variant { Cryptocurrency; FiatCurrency; };
type Asset = record {
symbol: text;
class: AssetClass;
};
type GetExchangeRateRequest = record {
base_asset: Asset;
quote_asset: Asset;
timestamp: opt nat64;
};
type ExchangeRateMetadata = record {
decimals: nat32;
base_asset_num_received_rates: nat64;
base_asset_num_queried_sources: nat64;
quote_asset_num_received_rates: nat64;
quote_asset_num_queried_sources: nat64;
standard_deviation: nat64;
};
type ExchangeRate = record {
base_asset: Asset;
quote_asset: Asset;
timestamp: nat64;
rate: nat64;
metadata: ExchangeRateMetadata;
};
type ExchangeRateError = variant {
CryptoBaseAssetNotFound: null;
CryptoQuoteAssetNotFound: null;
StablecoinRateNotFound: null;
StablecoinRateTooFewRates: null;
StablecoinRateZeroRate: null;
ForexInvalidTimestamp: null;
ForexBaseAssetNotFound: null;
ForexQuoteAssetNotFound: null;
ForexAssetsNotFound: null;
RateLimited: null;
NotEnoughCycles: null;
FailedToAcceptCycles: null;
InconsistentRatesReceived: null;
Other: record {
code: nat32;
description: text;
}
};
type GetExchangeRateResult = variant {
Ok: ExchangeRate;
Err: opt ExchangeRateError;
};
service : {
"get_exchange_rate": (GetExchangeRateRequest) -> (GetExchangeRateResult);
}
Here is a sample invocation using dfx:
dfx canister --network ic call ---candid xrc.did -with-cycles 5000000000 --wallet $WALLET gvu7d-7aaaa-aaaan-aaaba-cai get_exchange_rate '(record { base_asset = record { symbol = "ICP"; class = variant { Cryptocurrency } }; quote_asset = record { symbol = "USDT"; class = variant { Cryptocurrency } }; timestamp = opt 1668693600 })'
As you can see from the command, the service is not free: Every request must send 5B cycles, which may be partially refunded depending on the number of HTTPS outcalls that are required to serve the request. If the requested rate is found in the cache, the cost is only 200M cycles.
We are currently working on open-sourcing the repository. Iāll let you know when the code is publicly available.
