Technical Working Group DeAI

[patnorris]

Hi everyone, thank you for today’s call (2025.03.20). Special thanks to @pu0238 for demoing, @jennifertran for organizing and @ielashi for leading the second part of the call! This is the generated summary (short version, please find the long version here ): Today’s DeAI Working Group call for the Internet Computer discussed recent advancements in efficiently running Large Language Models (LLMs) on-chain. The DeAI Chat team introduced a novel “chunking” strategy that sequentially processes small vector chunks, significantly reducing memory usage and enabling models with up to 1.5 billion parameters to run effectively. They demonstrated both an off-chain client method, optimizing speed through query calls, and an on-chain client method emphasizing decentralization via update calls, albeit with slower responses. Additionally, strategic paths for decentralizing AI worker nodes were explored: the “Classic” model, which aligns with current security but faces cost and scalability limitations, and the “Badlands” approach, which supports open participation and scalability but requires addressing security and decentralization concerns.

Links shared during the call:

• https://x.com/deai_chat?s=21
• deai - Google Drive
• https://vgjrt-uyaaa-aaaal-qsiaq-cai.icp0.io/
• Introducing the LLM Canister: Deploy AI agents with a few lines of code - #32 by ielashi