Technical Working Group DeAI

Developers
143

Thanks @icpp for your support and for all the work on icpp-pro.

Thank you @patnorris, it would be my pleasure to share my progress with you.

1 Like
144

awesome, looking forward to it! We have a call every Thursday at 6pm CET in the IC Dev Discord. This is our DeAI WG channel there: ICP Developer Community
Would you be interested to share your work during one of our upcoming calls, e.g. next week Thursday or the Thursday after?

145

Next week would be great. Sent you on discord.

1 Like
146

Hi everyone, thank you for today’s discussion! This is the summary of the call (long summary here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.06.20 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub):
In today’s call of the Decentralized AI Working Group for the Internet Computer (IC), discussions focused on enhancing developer documentation and planning future topics. The group emphasized creating structured AI documentation with sections on overview, workflows, tooling resources, and examples to meet developers’ specific goals. They discussed content suggestions, encouraging developer contributions, and aligning documentation with industry standards and ICP capabilities. Technical considerations included merging machine learning and inference workflows and detailing current and future capabilities to set expectations. Innovative AI concepts and iterative improvements based on feedback were highlighted. Future topics will cover current and future AI capabilities, speculative AI model re-architecting, regulatory impacts, and comparative analysis with similar projects. Action items include revising and posting the documentation outline for feedback, and planning discussions on GPU determinism, LLM sizes on ICP, training data pathways, and global AI governance regulations.

147

And what else do you all think should be included in the new DeAI Dev Docs?
If you get the chance, please provide your feedback on the outline that @Jessie created: AI Documentation - Google Docs would be great to have everyone’s ideas included💪

3 Likes
148

@Jessie ,
The outline of the new DeAI Dev Docs looks great!

Excited to contribute.

2 Likes
149

FYI, regarding the issues that @ktimam ran into, he was able to resolve them after Ulan explained that the wasm contained many exported globals that were unused. @ktimam was then able to correct the wasm, by hacking into it manually, and get it running on mainnet. :tada:

To fix the issue of those unused globals systematically, two things are planned:

  • I plan to update icpp-pro to run a post build wasm optimization tool. Ulan made a recommendation (here) what to run and I am investigating right now how to integrate it.
  • DFINITY team is planning to update the ic-wasm tool to always strip out unused globals (here)

So, many thanks to @ktimam for pushing the boundaries, and helping improve the IC and tooling.

5 Likes
150

Sure, here is the corrected version:

“Hi, this SIMD support doesn’t really work for me, or there are some unsupported features that Candle uses. :(”

image
image805×1267 35.1 KB

151

Hey pu0238,
I’m not sure where the screenshot comes from, but please make sure the dfx version is at least 0.20.2-beta.0 and the SIMD instructions are enabled on the project/function level.

152

Document outline looks promising!

153

I am using 0.20.2-beta.0, and I added SIMD support to rustflags in Cargo as shown in the example below. However, libraries from CanDE still do not compile unfortunately.

154

Push your changes somewhere, and I’ll have a look.

155

:muscle: we’ll get a demo by @ktimam today during our WG call. And @gip will share his progress on modular transformer models on the IC too :slight_smile: See you all then

156

Hi everyone, thank you for today’s call (2024.06.27). This is the generated summary (short version, please find the long version here:
In the DeAI Working Group’s call for the Internet Computer, key points included highlighting AI product success stories via a shared survey to strategize marketing and measure metrics for partnerships, aiming to position ICP prominently in the AI space with data partners like CoinMarketCap. Kiko presented his integration of the Flashlight library into ICP, sharing his experiences and challenges in bringing C++ projects, such as a soccer simulator and a physics engine, to the platform, and demonstrated training a neural network on-chain. The discussion covered the use of SIMD instructions to enhance GPT-2 transformer performance, resulting in a 1,230% efficiency increase, and Jeshli’s plan to publish related code and tokenizers. Efforts to bring the LLaMA model to ICP were detailed by Giles, showcasing modular implementation in Rust and the feasibility of running computations on-chain despite latency issues. Community support, funding for computational tasks, and future directions focused on code optimization and deploying the first open-source LLaMA model on-chain were also discussed.

Special thanks to @ktimam and @gip for sharing their exciting work!

5 Likes
157

As mentioned in the last summary, please participate in the survey to highlight DeAI success stories here
and please see its forum thread:

Looking forward to seeing all our community success stories being covered :slight_smile:

Thank you @jennifertran for this opportunity!

158

Hi everyone, thank you for today’s call (2024.07.04). This is the generated summary (short version, please find the long version here:
In discussing the implementation of larger language models on the Internet Computer (IC), progress was highlighted in creating canisters for tensor computations despite memory constraints. Key discussions included the slow manual reimplementation of models for IC and the suggestion to use tools like PyTorch and TinyGrad for efficiency. Challenges with stable memory due to bugs and inefficiency were addressed, with proposed direct memory management improvements. The benefits of using frameworks like PyTorch and ONNX were emphasized, along with the potential of the Rust library Candle. Horizontal scaling, federated learning, and the IC’s decentralized data ownership were explored as future directions. Action items include experimenting with tools like TinyGrad and federated learning, improving stable memory handling, and fostering developer collaboration.

links shared during the call:

1 Like
159

Would love to learn more about how ICP can support federated AI and crowdsource fine tuning models! Also how to set up ICP-network-owned parallel processing / supercomputing

2 Likes
160

Hi there, yes, super fascinating topic. There is this thread as a starter: Discord
happy to also discuss more on this with you :+1:

161

Hi folks!

The recent dfx 0.22.0-beta.0 comes with the floating point optimization and it also has Wasm SIMD (corresponds to the replica version https://dashboard.internetcomputer.org/proposal/130984)

If you see any performance improvement going from dfx 0.20.1 to dfx 0.22.0-beta.0 with Wasm SIMD and you project is open source, I’d be happy to mention your project and improvements in the upcoming blog post about these optimizations.

4 Likes
162

Hi @ulan,

I plan to run performance tests over the weekend with my LLMs. Is that still on time for your blog post, or do you need it sooner?