Those of you who are into conversational AI are probably aware of this model that has taken the open source world by storm: karpathy/llama2.c Using icpp-pro , I am able run this LLM in a canister and do inference: dfx canister call llama2 inference '(record {"prompt" = "" : text; "steps" = 20 : nat…

Llama2.c LLM running in a canister!

ildefons August 7, 2023, 9:18pm 2

What is the average inference time? If it takes longer than the consensus period, what happens?

Topic		Replies	Views
Llama.cpp on the Internet Computer Programs & Applications	6	205	November 8, 2024
Llama 3 8b is running on-chain! Programs & Applications	6	721	July 16, 2024
AI player on ICP - IChess Showcase Functional-Programming , Discussing , community-consideration	9	391	October 9, 2024
Now General LLM is running on a canister!🚀 Developers	2	815	October 20, 2023
Is llama-13B(or 7B) LLM possible to deploy on canister? Developers Discussing	4	422	June 6, 2024