# Technical Working Group DeAI **URL:** \ **Category:** Developers\ **Created:** [November 12, 2023, 4:34pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621 "2023-11-12T16:34:10Z")\ **Posts on this page:** 20\ **Page:** 8
**Author:** ![ktimam](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/ktimam/32/5964_2.png) [@ktimam](https://forum.dfinity.org/u/ktimam)\ **Post date:** [June 17, 2024, 10:36pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/143 "2024-06-17T22:36:57Z")
Thanks @icpp for your support and for all the work on icpp-pro. Thank you @patnorris, it would be my pleasure to share my progress with you. ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [June 19, 2024, 9:18pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/144 "2024-06-19T21:18:21Z")
awesome, looking forward to it! We have a call every Thursday at 6pm CET in the IC Dev Discord. This is our DeAI WG channel there: [ICP Developer Community](https://discord.gg/4UZ6XEpYuS) Would you be interested to share your work during one of our upcoming calls, e.g. next week Thursday or the Thursday after? ---
**Author:** ![ktimam](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/ktimam/32/5964_2.png) [@ktimam](https://forum.dfinity.org/u/ktimam)\ **Post date:** [June 19, 2024, 9:25pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/145 "2024-06-19T21:25:22Z")
Next week would be great. Sent you on discord. ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [June 20, 2024, 6:09pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/146 "2024-06-20T18:09:12Z")
Hi everyone, thank you for today’s discussion! This is the summary of the call (long summary here: [DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.06.20 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub](https://github.com/DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer/tree/main/WorkingGroupMeetings/2024.06.20)): In today’s call of the Decentralized AI Working Group for the Internet Computer (IC), discussions focused on enhancing developer documentation and planning future topics. The group emphasized creating structured AI documentation with sections on overview, workflows, tooling resources, and examples to meet developers’ specific goals. They discussed content suggestions, encouraging developer contributions, and aligning documentation with industry standards and ICP capabilities. Technical considerations included merging machine learning and inference workflows and detailing current and future capabilities to set expectations. Innovative AI concepts and iterative improvements based on feedback were highlighted. Future topics will cover current and future AI capabilities, speculative AI model re-architecting, regulatory impacts, and comparative analysis with similar projects. Action items include revising and posting the documentation outline for feedback, and planning discussions on GPU determinism, LLM sizes on ICP, training data pathways, and global AI governance regulations. ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [June 20, 2024, 6:10pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/147 "2024-06-20T18:10:53Z")
And what else do you all think should be included in the new DeAI Dev Docs? If you get the chance, please provide your feedback on the outline that @Jessie created: [AI Documentation - Google Docs](https://docs.google.com/document/d/1tf6kbzE2Vqxb5I6JUWZzJJpQ0qmGXkOfNqcU4yBWOSQ/edit) would be great to have everyone’s ideas included💪 ---
**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\ **Post date:** [June 21, 2024, 12:25am UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/148 "2024-06-21T00:25:40Z")
@Jessie , The outline of the new DeAI Dev Docs looks great! Excited to contribute. ---
**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\ **Post date:** [June 23, 2024, 12:00pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/149 "2024-06-23T12:00:19Z")
FYI, regarding the issues that @ktimam ran into, he was able to resolve them after Ulan explained that the wasm contained many exported globals that were unused. @ktimam was then able to correct the wasm, by hacking into it manually, and get it running on mainnet. 🎉 To fix the issue of those unused globals systematically, two things are planned: - I plan to update icpp-pro to run a post build wasm optimization tool. Ulan made a recommendation ([here](https://forum.dfinity.org/t/wasm-module-defined-globals-which-exceeds-the-maximum-number-allowed-200/10834/33)) what to run and I am investigating right now how to integrate it. - DFINITY team is planning to update the ic-wasm tool to always strip out unused globals ([here](https://forum.dfinity.org/t/wasm-module-defined-globals-which-exceeds-the-maximum-number-allowed-200/10834/34)) So, many thanks to @ktimam for pushing the boundaries, and helping improve the IC and tooling. ---
**Author:** ![pu0238](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/pu0238/32/14831_2.png) [@pu0238](https://forum.dfinity.org/u/pu0238)\ **Post date:** [June 24, 2024, 1:25am UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/150 "2024-06-24T01:25:52Z")
Sure, here is the corrected version: “Hi, this SIMD support doesn’t really work for me, or there are some unsupported features that Candle uses. :(” ![image](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/3X/4/1/415391b03fa687c42467440ba79ef61d728711f4.png) ---
**Author:** ![berestovskyy](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/berestovskyy/32/3989_2.png) [@berestovskyy](https://forum.dfinity.org/u/berestovskyy)\ **Post date:** [June 24, 2024, 6:58am UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/151 "2024-06-24T06:58:39Z")
Hey pu0238, I’m not sure where the screenshot comes from, but please make sure the `dfx` version is [at least 0.20.2-beta.0](https://forum.dfinity.org/t/technical-working-group-deai/24621/129) and the [SIMD instructions are enabled](https://internetcomputer.org/docs/current/developer-docs/smart-contracts/advanced-features/simd/#example) on the project/function level. ---
**Author:** ![therealbryanho](https://avatars.discourse-cdn.com/v4/letter/t/7bcc69/32.png) [@therealbryanho](https://forum.dfinity.org/u/therealbryanho)\ **Post date:** [June 25, 2024, 3:09pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/152 "2024-06-25T15:09:40Z")
Document outline looks promising! ---
**Author:** ![pu0238](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/pu0238/32/14831_2.png) [@pu0238](https://forum.dfinity.org/u/pu0238)\ **Post date:** [June 26, 2024, 6:56pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/153 "2024-06-26T18:56:38Z")
I am using 0.20.2-beta.0, and I added SIMD support to `rustflags` in Cargo as shown in the example below. However, libraries from CanDE still do not compile unfortunately. > **[WebAssembly SIMD | Internet Computer](https://internetcomputer.org/docs/current/developer-docs/smart-contracts/advanced-features/simd/#example)** > > Overview ---
**Author:** ![berestovskyy](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/berestovskyy/32/3989_2.png) [@berestovskyy](https://forum.dfinity.org/u/berestovskyy)\ **Post date:** [June 26, 2024, 7:00pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/154 "2024-06-26T19:00:42Z")
Push your changes somewhere, and I’ll have a look. ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [June 27, 2024, 2:46pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/155 "2024-06-27T14:46:51Z")
💪 we’ll get a demo by @ktimam today during our WG call. And @gip will share his progress on modular transformer models on the IC too 🙂 See you all then ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [June 28, 2024, 4:06am UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/156 "2024-06-28T04:06:19Z")
Hi everyone, thank you for today’s call (2024.06.27). This is the generated summary (short version, [please find the long version here](https://github.com/DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer/tree/main/WorkingGroupMeetings/2024.06.27): In the DeAI Working Group’s call for the Internet Computer, key points included highlighting AI product success stories via a shared survey to strategize marketing and measure metrics for partnerships, aiming to position ICP prominently in the AI space with data partners like CoinMarketCap. Kiko presented his integration of the Flashlight library into ICP, sharing his experiences and challenges in bringing C++ projects, such as a soccer simulator and a physics engine, to the platform, and demonstrated training a neural network on-chain. The discussion covered the use of SIMD instructions to enhance GPT-2 transformer performance, resulting in a 1,230% efficiency increase, and Jeshli’s plan to publish related code and tokenizers. Efforts to bring the LLaMA model to ICP were detailed by Giles, showcasing modular implementation in Rust and the feasibility of running computations on-chain despite latency issues. Community support, funding for computational tasks, and future directions focused on code optimization and deploying the first open-source LLaMA model on-chain were also discussed. Special thanks to @ktimam and @gip for sharing their exciting work! ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [July 1, 2024, 3:04pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/157 "2024-07-01T15:04:19Z")
As mentioned in the last summary, please participate in the [survey to highlight DeAI success stories here](https://docs.google.com/forms/d/e/1FAIpQLSeZU-6BkFXoNPB9i1P3g8Wl6hjIA_wrkFOaBpu4yzKgRjMumA/viewform) and please see its forum thread: > [@Featuring Success Stories of DeAI Products on ICP](https://forum.dfinity.org/t/featuring-success-stories-of-deai-products-on-icp/32523): > > Are you building a DeAI product on ICP or enhancing an AI product using ICP? As part of the recent Growth/Marketing/Adoption joint call with the [DeAI Technical Working Group](https://forum.dfinity.org/t/technical-working-group-deai/24621/135), we discussed highlighting success stories through DFINITY social media channels and amplifying key major metrics through any websites or publications that track web3 growth metrics by ecosystem. We invite you to fill out [this survey](https://forms.gle/fS58RzMwFZEC7gVL9) to help us promote any success stories of AI on ICP. If you have any questions or blocker… Looking forward to seeing all our community success stories being covered 🙂 Thank you @jennifertran for this opportunity! ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [July 4, 2024, 7:33pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/158 "2024-07-04T19:33:09Z")
Hi everyone, thank you for today’s call (2024.07.04). This is the generated summary (short version, [please find the long version here](https://github.com/DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer/tree/main/WorkingGroupMeetings/2024.07.04): In discussing the implementation of larger language models on the Internet Computer (IC), progress was highlighted in creating canisters for tensor computations despite memory constraints. Key discussions included the slow manual reimplementation of models for IC and the suggestion to use tools like PyTorch and TinyGrad for efficiency. Challenges with stable memory due to bugs and inefficiency were addressed, with proposed direct memory management improvements. The benefits of using frameworks like PyTorch and ONNX were emphasized, along with the potential of the Rust library Candle. Horizontal scaling, federated learning, and the IC’s decentralized data ownership were explored as future directions. Action items include experimenting with tools like TinyGrad and federated learning, improving stable memory handling, and fostering developer collaboration. links shared during the call: - [DeAI Success on ICP Survey](https://docs.google.com/forms/d/e/1FAIpQLSeZU-6BkFXoNPB9i1P3g8Wl6hjIA_wrkFOaBpu4yzKgRjMumA/viewform) - [GitHub - gip/yllama.rs: Inference with Transformers in Pure Rust](https://github.com/gip/yllama.rs) - [gip/yllama.oc · GitHub](https://github.com/gip/yllama.oc) - [https://www.youtube.com/watch?v=13pnH\_8cBUM&list=WL&index=102](https://www.youtube.com/watch?v=13pnH_8cBUM&list=WL&index=102) - [https://woglz-oqaaa-aaaal-act2q-cai.icp0.io/](https://woglz-oqaaa-aaaal-act2q-cai.icp0.io/) - [GitHub - jeshli/rust-connect-py-ai-to-ic: A streamlined open-source tool for deploying Python AI models on the Internet Computer (IC). Enables easy uploading and inference.](https://github.com/jeshli/rust-connect-py-ai-to-ic) - [GitHub - jeshli/tract-ic-ai: Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference](https://github.com/jeshli/tract-ic-ai) - [GitHub - tinygrad/tinygrad: You like pytorch? You like micrograd? You love tinygrad! ❤️](https://github.com/tinygrad/tinygrad) - [GitHub - huggingface/candle: Minimalist ML framework for Rust](https://github.com/huggingface/candle) - [https://0g.ai/](https://0g.ai/) ---
**Author:** ![SolarShogun](https://avatars.discourse-cdn.com/v4/letter/s/3bc359/32.png) [@SolarShogun](https://forum.dfinity.org/u/SolarShogun)\ **Post date:** [July 4, 2024, 11:15pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/159 "2024-07-04T23:15:31Z")
Would love to learn more about how ICP can support federated AI and crowdsource fine tuning models! Also how to set up ICP-network-owned parallel processing / supercomputing ---
**Author:** ![patnorris](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/patnorris/32/11764_2.png) [@patnorris](https://forum.dfinity.org/u/patnorris)\ **Post date:** [July 5, 2024, 1:43pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/160 "2024-07-05T13:43:14Z")
Hi there, yes, super fascinating topic. There is this thread as a starter: [Discord](https://discord.com/channels/748416164832608337/1184160869614108873/1249136286279794688) happy to also discuss more on this with you 👍 ---
**Author:** ![ulan](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/ulan/32/2705_2.png) [@ulan](https://forum.dfinity.org/u/ulan)\ **Post date:** [July 10, 2024, 3:00pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/161 "2024-07-10T15:00:30Z")
Hi folks! The recent `dfx 0.22.0-beta.0` comes with the floating point optimization and it also has Wasm SIMD (corresponds to the replica version [https://dashboard.internetcomputer.org/proposal/130984](https://dashboard.internetcomputer.org/proposal/130984)) If you see any performance improvement going from `dfx 0.20.1` to `dfx 0.22.0-beta.0` with Wasm SIMD and you project is open source, I’d be happy to mention your project and improvements in the upcoming blog post about these optimizations. ---
**Author:** ![icpp](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icpp/32/9310_2.png) [@icpp](https://forum.dfinity.org/u/icpp)\ **Post date:** [July 10, 2024, 6:03pm UTC](https://forum.dfinity.org/t/technical-working-group-deai/24621/162 "2024-07-10T18:03:14Z")
Hi @ulan, I plan to run performance tests over the weekend with my LLMs. Is that still on time for your blog post, or do you need it sooner? [Previous page](https://forum.dfinity.org/t/technical-working-group-deai/24621.md?page=7) [Next page](https://forum.dfinity.org/t/technical-working-group-deai/24621.md?page=9)