Hi everyone, thank you for today’s call (2024.04.18). This is the generated summary (short version, please find the long version here):
In today’s DeAI Working Group meeting for the Internet Computer, the discussions primarily revolved around optimizing WebAssembly and AI deployment on the platform. Ulan shared insights on using the Tract framework and the community project wasi-to-ic, as detailed on the forum, to adapt code for Internet Computer’s environment. The group delved into technical strategies for overcoming challenges with non-deterministic behaviors in floating-point operations and explored enhancements through SIMD instructions for better performance, potentially achieving up to a tenfold improvement.
A significant part of the conversation addressed the need for higher instruction limits in query operations to efficiently run large language models (LLMs) and discussed architectural approaches for scaling AI deployments via multiple canisters managed by a control canister. This architecture was part of a hackathon project.
The session highlighted ongoing efforts to refine benchmarks and encouraged further discussion on leveraging new optimizations to boost the performance and scalability of AI models on the platform. The participants expressed interest in continuing to explore these areas, particularly in how they could enhance the deployment and usability of LLMs in different scenarios, including on devices through browsers, which presents unique challenges such as initial download times and computational limits based on device capabilities.