I noticed that as well. It happens a non-negligible number of times. I think this is a problem with the implementation of the sampler and not with the model itself. Looking into it.
Apart from the times where it returns gibberish, how did you find the model itself?
I think it works quite well for generating tweets based on a persona. Soon, we will start using them to analyze prediction markets, which will be a bit more challenging since they will need to estimate probabilities based on available data and compete.
If a closed model is not possible, it would be nice to have DeepSeek working.
@Mar Done some changes, and I think now you shouldn’t be seeing gibberish anymore, or at least it should be way less probable now. We’ll have a more permanent solution to that problem very soon.
@marcio I’ve tried DeepSeek extensively, and it’s impressive. We’ll explore supporting one of their distilled versions.
Thanks for building this @ielashi!
We’ve just launched the “LlamaBot” within OpenChat which allows users to easily send prompts to the LLM canister!
How were you able to build a bot for openchat with it @ielashi
I built it, you can find the source code here - open-chat-bots/rs/offchain/examples/llama at main · open-chat-labs/open-chat-bots · GitHub
Quick update: we released a new example of an agent, just to showcase what it’s like to build an agent that specializes in a specific task. In this case, the task is to lookup ICP prices.
A Rust and a Motoko implementation are provided in the examples folder here.
Indeed, we need to use better models for this, as the difference in probability estimates is too large (compared to 4o, for instance).
Hey everyone,
I’d like to get some feedback here on what you think should be the highest priority for the LLM canister. There will be continuous work on all the topics below, but I’d like to gauge which one is the most important to you at this stage:
Would love to hear also any comments you have wrt your choice.
I think supporting more models is the best choice in the near term. Then, we can spend more time decentralizing, plus we gain more time until open-source models catch up.
Thanks everyone for voting, I appreciate your input. I closed the poll now.
As I mentioned, we’ll be making progress across all three fronts, but this poll will definitely help with the prioritization.
Regarding bigger models: can you share which models you’d like to have access to? To allow for decentralization later on, we can only consider open weight models.
Regarding decentralizing the AI workers: I’ll start a separate thread about that, as it’s a bigger topic.
Regarding API improvements: We just released a typescript library to seamlessly integrate with Azle, so developers don’t need to learn Motoko or Rust to build AI agents on the IC. Checkout the quickstart example that can be used as a template for building agents in Typescript (cc @tiago89).
Prompting (single message)
import { IDL, update } from "azle";
import * as llm from "@dfinity/llm";
export default class {
@update([IDL.Text], IDL.Text)
async prompt(prompt: string): Promise<string> {
return await llm.prompt(llm.Model.Llama3_1_8B, prompt);
}
}
Chatting (multiple messages)
import { IDL, update } from "azle";
import { chat_message as ChatMessageIDL } from "azle/canisters/llm/idl";
import * as llm from "@dfinity/llm";
export default class {
@update([IDL.Vec(ChatMessageIDL)], IDL.Text)
async chat(messages: llm.ChatMessage[]): Promise<string> {
return await llm.chat(llm.Model.Llama3_1_8B, messages);
}
}
Thanks for this. Here is what I am using it for.
Very nice, I like the UX 
I created an Anima and had a short conversation with it. Thanks for sharing. I recommend demoing this to the DeAI Technical Working Group so that more people from the community can also see it and provide feedback.
I second that! cc @patnorris
That’d be awesome, yes! Would you be open to sharing your work in one of the upcoming DeAI calls @swift ?
Sure 
. Just let me know where/when I can join.
Hi @swift , the DeAI calls are each Thursday at 6pm CET in the ICP Discord voice channel. This should take you to this week’s event on Discord (and also show the time in your timezone): ICP
We’ve usually got an agenda for the call (and also for the next few calls) but leave a few minutes for new members to introduce themselves and what they are working on. So you could give a quick intro on one of the next calls and if you’re open to it, it’d be great to also have a dedicated session to showcase ANIMA during one of the calls in a few weeks, so there’s enough time for properly presenting it, discussion and questions (maybe one of the first Thursdays in May?). How does this sound to you?
@patnorris Sounds fun, I am interested
First, let me join everyone and congratulate you.
A useful LLM integration.
Three questions:
Thanks!
Joseph