Someone linked this critique of KANs on the MLST discord
2 Likes
Great topic! I took some time to read through it and learned a lot about DeAI.
We developed a DeAI canister based on Huggingface’s Candle framework and tried running a 0.5B parameter, 1.2G data LLM.
It can run in a local cluster with adjusted instruction limits, and here are the results:
dfx canister call bwwuq-byaaa-aaaan-qmk4q-cai update_chat '(record {prompt="Nice to chat with you, Please introduce yourself."})'
(
variant {
Ok = record {
instructions = 1_128_679_236_950 : nat64;
tokens = 43 : nat32;
message = "\nHello! I am Panda Oracle, a giant panda with human intelligence. I am here to help you with any questions or concerns you may have. How can I assist you today?\n";
}
},
)
Source code: ic-panda/src/ic_panda_ai at main · ldclabs/ic-panda · GitHub
9 Likes
Thank you for sharing. I have also been experimenting with candle and believe it to be an excellent crate for streamlining ML on the IC.
2 Likes
Awesome accomplishment!!
Can you explain more whay you mean with ‘adjusted instructions limit’ ?
Also, are you doing query or update calls ?
2 Likes
Tool to build local IC replica and dfx:
I’ve tested it, and the query call works:
However, changing the instruction limit for query calls is a bit tricky. You need to modify the max_query_call_graph_instructions parameter in the ic.json5 file generated during runtime.
4 Likes
Thank you for those details!
1 Like
Our team @ RuBaRu is taking baby steps towards developing OnChain AI services. These innovations aim to enhance user experience and bring greater transparency to content distribution and discovery. Additionally, our AI will analyse and tag rich media, such as images and videos, with content types. I believe we have a long road ahead, but it’s a solid start towards keeping the ecosystem fully OnChain. We welcome your ideas and feedback as we advance in the SocialFi space.
Read more: RuBaRu - A Fully On-Chain Regenerative Creators-Consumers Economy - #7 by TusharGuptaMm
3 Likes
Hey folks,
The WebAssembly SIMD is available on the mainnet and the latest dfx:
-
To install the latest
dfxwith SIMD support:dfxvm install 0.20.2-beta.0 -
The Rust SIMD example, which compares naive, optimized, auto-vectorized, and SIMD intrinsic matrix multiplication: examples/rust/simd at master · dfinity/examples · GitHub
-
A bit of SIMD documentation on the IC: https://internetcomputer.org/docs/current/developer-docs/smart-contracts/advanced-features/simd/
Let me know if you have any questions,
and enjoy!
11 Likes
This is amazing! Will give it a try!
2 Likes
That is welcome and much anticipated great news!
Thanks for sharing it here, I will copy it to the DeAI WG Discord channel and start spreading the word!
Many thanks to you and other Dfinity team members who have worked on the SIMD support and pushed it along @berestovskyy
2 Likes
Dear WG members,
please add any missing resources to this page 
3 Likes
A pull request to add ICPanda AI canister: chore: add ICPanda AI to AI docs by zensh · Pull Request #3004 · dfinity/portal · GitHub
4 Likes
Hi everyone, this Thursday we’ll have our joined call with the DFINITY growth team 
We’ll discuss how to best collaborate, so please bring all your ideas 
As usual, it’s on Thursday at 6pm CET and in the ICP Developer Discord. This is a link to the event: Discord
Looking forward to the call!
6 Likes
Hi everyone, thank you for today’s call and the great discussion! Special thanks to @jennifertran for putting it together This is the summary of the call, looking forward to collaborating on the next steps and ideas we had 
Long summary here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.06.06 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub
Short summary:
The DeAI Working Group call for the Internet Computer focused on improving AI project marketing and documentation and was a joined session with DFINITY team members across growth, marketing and adoption. Participants discussed the need for better marketing strategies, comprehensive AI documentation, and practical use cases to demonstrate ICP’s benefits. Collaborative efforts were emphasized, including creating a decentralized AI manifesto, collecting success stories, and exploring monthly public updates. The call concluded with actionable steps to support ecosystem teams, enhance marketing resources, and ensure the successful promotion of decentralized AI on the Internet Computer.
6 Likes
Are the meetings being recorded? or just the summary?
Hi there, the meetings are not recorded, but we have all summaries so far in here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub
2 Likes
Hi everyone, thank you for today’s call! Special thanks to @gip for sharing insights on his exciting work This is the short summary (long summary here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.06.13 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub):
In today’s call of the DeAI Working Group for the Internet Computer, several key initiatives were discussed. The team plans to develop a survey to gather information from ecosystem teams building AI projects, aiming to create a tailored marketing strategy. The importance of tracking usage statistics for decentralized AI applications was emphasized to demonstrate activity and traction to potential partners. Updates were provided on the scheduling of DeAI live streams and organizing a DeAI panel for the Chain Fusion Day event at EthCC. A proposal to draft a decentralized AI manifesto was made to outline principles and goals. The group considered creating a chart to rate projects on their actual use of AI technology, enhancing transparency. Jessie presented a draft outline for new documentation pages to improve decentralized AI documentation. Metrics for tracking developer engagement were discussed.
Giles shared his challenges in fine-tuning a large language model (LLM) on the Internet Computer, highlighting the approach of dividing the model into smaller segments. Technical issues like efficient data loading across canisters and handling large data files were discussed, with suggestions to use separate endpoints to load weights. The discussion emphasized the importance of data privacy in AI applications and the potential of the Internet Computer in collaborative AI development. It was noted that specialized tasks might not require extremely large models, with more compact models being sufficient for many use cases. The group touched on current capabilities of the IC in running LLMs and the need for efficiency improvements. Discussions included optimizing Rust code using SIMD, creating a reusable library in Rust, and managing large models by splitting them into canisters. The challenges and potential of utilizing GPUs for AI on the Internet Computer were briefly discussed. The call concluded with a reminder of the upcoming scalability and performance working group meeting, encouraging participants to share topics and initiatives for future sessions.
Please do record the future discussions @jennifertran, thanks.
@patnorris ,
I like to bring awareness to the project of @ktimam , who is porting Flashlight to the IC. This is a C++ machine learning library, and it includes a port of openblas, fftw & arrayfire. A truly amazing effort.
We adapted iccp-pro to support static libraries and C++17, and he is now able to build the wasm that should be able to run on the IC, but he is running again into the dreaded maximum on allowed globals.
He already entered a request to get it increased, here, but I like to also give this issue visibility through communication from our DeAI working group with DFINITY scaling team.
6 Likes
Thank you @icpp and sounds exciting @ktimam!
Cool, happy to talk more about this in the group and see how we can support communication and progress on this
And @ktimam if you’re interested in discussing your work with or maybe giving a demo to the DeAI group, please let me know and we’ll arrange 
happy to learn more