Oracles. I’m suspicious. - re: coming http requests

General DFINITY
1

Maybe I’m just a math nerd, but I like determinism. I like my contract inputs to be replayable and to end up with the same thing at the end of every run.

Everyone is looking forward to outgoing http requests and all I see are foot guns. I’ll try to explain my position and hopefully, by the end you at least have a better idea of the kind of issues you are going to have if you try to pull data from the internet into your IC canister and what price you will pay to do so.

TLDR: Oracles push data into a smart contract. The IC can ping internet services. Great! The IC can pull data from the internet. Whoa there! You just broke blockchains.

  1. Classic oracles have users push data into smart contracts with their keys and the contract applies security rules to determine who can put data and how many users have to agree to validate the piece of data. There are lots of security issues here if some of the signers become malicious. You have to have some really good crypto economics going on and the forethought may depend on how the data is used. If it is used in financial calls you better have some really good crypto. If it is pushing a tweet onto your contract, maybe less…unless someone has a betting contract based on how much you will tweet, and then that contract is only as good as your crypto.

  2. The IC runs on scalable computers with way more power than most blockchains. Making http requests to the real internet possible. This is great, because now when your smart contract needs fresh data it can send out a ping to an internet service saying it needs data. This external service can turn on the bat signal(or send some emails or api calls) telling our oracle signers that we need new data. They get the ping and push a new round of data agreements.

The important thing here is that the response from the internet for the initial ping is not important. If the request fails, your contract can create a nonstate commiting queue that keeps retrying, but your main logic doesn’t care what the service actually says back in reply.

  1. The wizards at dfinity have figured out how to have these http requests go out and then have responses come back in in a reliable way. This is magic and I don’t know if I’ll believe that they can do it until I see it. The reason this is hard is that blockchains run on multiple computers. If they all make the same request to the same service they could get different responses. If they get different responses they won’t be able to agree on the contents of the next block. If 13 replicas call add_friend on Facebook-api the first one will get “added(friend if)” and the other 12 will get “error friend already exists). The end service can implement a form of consistent responses based on a request Id, but that takes cooperation from your service provider.

Getting your service provider to agree to not return different responses for the same response is just the first hurdle. The next issue is one that breaks the fundamentals of blockchains further. You want to be able to replay transactions on your blockchain since genesis and end up with the same result at the end. That is how they work. If you call https://weather.com/gettodaysweather today, that service will not return the same result if you replay the chain tomorrow.

You don’t have this problem with classic oracle because pushing the response from outside parties is part of the transaction chain. When you replay they get approved in the same way and your data is the same.

Maybe you could get your service to agree to keep hit requests on file forever and ever and always replay to you, but you’ve created a massive “maybe”. Blockchains do not do well with maybe.

All of this to say(and to open myself up to learn where I’m wrong), but if you intend to call an external oracle from the IC and use that réponse as stateful, contract-affecting behavior, you may be getting yourself into a situation where you are forever stuck in web 2.5.

Maybe you don’t care about this limitation, but if you want to do blockchainy things with your app down the road, please consider thinking some more about it.

The ping and wait pathway of #2 above can result in the same application while preserving your replayability.

5 Likes
2

I always thought that the async world would present it’s own challenges.

In response to your point #1, I agree that it is highly unlikely that most end-service provider will simply provide a new idempotent end-service: addFriend (reqId, friend) as opposed to addFriend(friend) day one. A little help from another end-service from the same service-provider, isFriend(friend) would help.

The implementation could have two functions:
the first one addFriend(friend) ALWAYS assumes failure.
the second one isFriend(friend) is used to query whether addFriend actually succeeded.

In response to #2, we would need a in-direction mechanism. i.e. you would not use the gettempatureatlocation function directly. You would call this function to store the data immutablly in a cansiter somewhere with a timestamp. Then when you need the data (or it’s replay), you would have a stream processing engine where you push this data to get replayability (at least in theory).

Rejoinders?

2 Likes
3

In my understanding the http call feature is not meant as a silver bullet, it has its limitations (like your addFriend example). But the contract can define a callback which can transform the response on each node so it can remove/round timestamps and “not important” differences. In this way the nodes can agree on the response.
Regarding the “blockchain transactions should be replayable” I don’t think it’s true for IC as blocks are dropped after catch-up packages.

I think http calls are required if we want IC to be adopted, at least until grow beyond the critical mass.

6 Likes