Those of you who are into conversational AI are probably aware of this model that has taken the open source world by storm: karpathy/llama2.c Using icpp-pro , I am able run this LLM in a canister and do inference: dfx canister call llama2 inference '(record {"prompt" = "" : text; "steps" = 20 : nat…

Llama2.c LLM running in a canister!

ildefons August 7, 2023, 9:18pm 2

What is the average inference time? If it takes longer than the consensus period, what happens?

Topic		Replies	Views
Now General LLM is running on a canister!🚀 Developers	2	628	October 20, 2023
Is llama-13B(or 7B) LLM possible to deploy on canister? Developers Discussing	2	159	April 19, 2024
Scalable Messaging Model Roadmap community-consideration	63	1534	February 13, 2024
EVM RPC Canister Developers	45	4096	April 3, 2024
Can some one do a YouTube video on optimizing cross canister calls? Developers	3	518	February 2, 2023