Introducing the LLM Canister: Deploy AI agents with a few lines of code

Developers
,
54

LLMs on the Internet Computer - Major Update

Hey everyone, I’m Dave and I recently joined the AI team at DFINITY. I’ve been working alongside Islam on some updates on the LLM canister:

tl;dr
We’ve added tool calling support and two new models (Qwen3 32B and Llama4 Scout) to make building AI agents on the IC even more powerful and accessible.

What’s New

Tool Calling Support

LLMs can now execute functions and interact with external systems! This enables building sophisticated agents that can:

  • Process real-time data
  • Interact with databases
  • Call external APIs
  • And much more!

New Models

We’ve expanded our model selection, this is our currently supported Model list:

  • Qwen3 32B (qwen3:32b) - Powerful multilingual model with excellent reasoning capabilities
  • Llama4 Scout (llama4-scout) - Latest from Meta with improved performance
  • Llama 3.1 8B (llama3.1:8b) - Still available and reliable

Tool Usage Examples And More Details

How Tool Calling Works, an Overview

  1. Define your tools - Specify what functions the LLM can call and their parameters
  2. Send request - Include tools in your chat request using the v1_chat endpoint
  3. LLM decides - The model determines if/when to use tools based on the conversation
  4. Execute function - Your application executes the requested function
  5. Return result - Send the tool result back to continue the conversation

The LLM response includes tool_calls when it wants to execute a function:

{
  "message": {
    "content": null,
    "tool_calls": [{
      "id": "call_123", 
      "function": {
        "name": "get_current_weather",
        "arguments": [
          {"name": "location", "value": "Paris"},
          {"name": "format", "value": "celsius"}
        ]
      }
    }]
  }
}

We’d love to hear your feedback on this LLM integration approach so we can continue refining it. And we’re always curious to learn about the projects you’re building with LLMs, feel free to share what you’re working on!

9 Likes