Those of you who are into conversational AI are probably aware of this model that has taken the open source world by storm: karpathy/llama2.c Using icpp-pro , I am able run this LLM in a canister and do inference: dfx canister call llama2 inference '(record {"prompt" = "" : text; "steps" = 20 : nat…

Llama2.c LLM running in a canister!

ildefons August 7, 2023, 9:18pm 2

What is the average inference time? If it takes longer than the consensus period, what happens?

Topic		Replies	Views
Introducing the LLM Canister: Deploy AI agents with a few lines of code Developers rust , DeAI	76	4265	September 1, 2025
AI and machine learning on the IC? Developers	114	10146	June 20, 2024
How can I take an open source pretrained LLM model, deploy it to ICP and use as a private ChatGPT just fo me Developers	13	357	June 16, 2025
Llama.cpp on the Internet Computer Programs & Applications	11	517	February 2, 2025
Is llama-13B(or 7B) LLM possible to deploy on canister? Developers Discussing	4	500	June 6, 2024