HTTP outcalls to LLMs

[ghostwhoowalks]

I have not tried this yet but want to be certain if this even makes sense. If my canister makes LLM Http calls and assume this is from a mutate method. The response I get will be agreed upon my the consensus layer as a result of the HTTP call correct? If so, where can I read about how this consensus is achieved. Because potentially every response from the LLM for the request could be slightly different. So for any kind of consensus every node will have to broadcast its response to every other node so they can agree on what the result ought to be. Correct? Where can I read about this? Thanks

[Severin]

Answer from our docs chatbot:

@Chatbot

You’re correct that when a canister makes an HTTPS outcall (including to an LLM service) from a mutate method, the response goes through the consensus layer to ensure all replicas agree on the same result. This is indeed a complex process, and I’ll explain it based on the available information and provide sources for further reading.

The consensus process for HTTPS outcalls works as follows:

1. Each replica in the subnet makes the HTTP request independently.
2. The responses are optionally transformed to normalize any differences.
3. The consensus layer then works to agree on a single response.

You’re right that potentially every response from an LLM could be slightly different. This is why the Internet Computer protocol includes a transformation step to handle such differences.

According to the HTTPS outcalls documentation:

“ICP consensus agrees on a response if at least 2/3 (to be more precise, 2f+1, where f is the number of faulty replicas tolerated by the protocol) of the replicas have the same response for the request as input.”

This means that the replicas don’t necessarily broadcast their responses to every other node. Instead, the consensus mechanism works to find agreement among at least 2/3 of the replicas.

For more detailed information on how this consensus is achieved, you can refer to the following resources:

1. Achieving Consensus on the Internet Computer
2. Consensus White Paper
3. Extended Abstract published at PODC’22

These sources provide in-depth explanations of the consensus mechanism used by the Internet Computer.

For a more specific look at how HTTPS outcalls are processed, including the consensus step, you can review the HTTPS outcalls documentation, which provides a step-by-step breakdown of the process.

Most interesting to you is probably the outcalls documentation. And I suggest you read up on what an ‘idempotency key’ is and/or use something like this