Introducing the LLM Canister: Deploy AI agents with a few lines of code

Developers
,
62

So technically, I can deploy and self hosted off-chain worker, which run my LLM. Then I deploy 2 canister, 1 is the LLM on-chain canister, 1 is my chat bot canister. Then I can do the same flow like you:

  1. Chatbot canister call to LLM canister
  2. LLM canister ingress the message → queued it
  3. Off-chain worker poll the queued message from the LLM canister
  4. Process it with local LLM or whatever
  5. Off-chain worker call LLM canister to update the state of the responded message state to proceeded and insert newly responded message
  6. LLM canister call chatbot canister to return message.

Am I right? If I do so what should I concern because I saw that you have working on it to improve and support many things like extend the input/output limit. Can you share some of the difficulties and obstacles you have encountered?