Those of you who are into conversational AI are probably aware of this model that has taken the open source world by storm: karpathy/llama2.c Using icpp-pro , I am able run this LLM in a canister and do inference: dfx canister call llama2 inference '(record {"prompt" = "" : text; "steps" = 20 : nat…

Llama2.c LLM running in a canister!

ildefons August 7, 2023, 9:18pm 2

What is the average inference time? If it takes longer than the consensus period, what happens?

Topic		Replies	Views
Is llama-13B(or 7B) LLM possible to deploy on canister? Developers Discussing	4	467	June 6, 2024
Introducing the LLM Canister: Deploy AI agents with a few lines of code Developers rust , DeAI	10	626	February 21, 2025
Llama.cpp on the Internet Computer Programs & Applications	11	399	February 2, 2025
Chatbot Canister some advice Developers	1	18	February 15, 2025
Persistence for Llama2.c LLM weights in canister General	0	14	February 16, 2025