Technical Working Group DeAI

Developers
82

Hi everyone, thank you for today’s call (2024.02.15). This is the generated summary (short version, please find a link to the comprehensive version below):
The Decentralized AI Technical Working Group for the Internet Computer’s call focused on two main areas: establishing benchmarks for AI on the IC and presenting a Vector DB implementation by the ELNA team.

Benchmarking and AI Deployment on the IC:

  • The session opened with introductions to foster community connections, followed by discussions on deploying TensorFlow models using Azle and TensorFlow JS, highlighting the IC’s potential for decentralized AI services.
  • @icpp detailed the memory requirements for large language models (LLMs), emphasizing the need for accurate benchmarking to understand computational demands and optimization strategies, including half-precision computation to reduce resource requirements.
  • Technical challenges of AI deployment were discussed, focusing on memory limitations and the search for decentralized alternatives to centralized cloud services.
  • The group explored technical solutions to these challenges, such as the potential shift to WASM 64 support for improved efficiency and the importance of collaborative tool development for model deployment.

Vector DB Implementation and Future Directions:

  • The ELNA team (@branbuilder) introduced Vector DB, emphasizing its role in managing high-dimensional vectors for AI models and facilitating fast, accurate similarity searches essential for tasks like semantic search.
  • A technical overview explained Vector DB’s operation, including the use of the HNSW algorithm for efficient indexing and cosine similarity for measuring vector closeness.
  • Future enhancements for Vector DB were discussed, including supporting various similarity measures and integrating more efficient vector embedding models to improve search capabilities.
  • Challenges of deploying AI models on the IC due to computational constraints were acknowledged, with strategies discussed for optimizing deployments through smaller models or task segmentation.
  • The session concluded with a focus on collaborative opportunities, the development of educational resources, and the potential for community-driven efforts to advance decentralized AI on the IC.

Long summary: 2024.02.15_ICDeAITWG_CallSummary - Google Docs

5 Likes
83

Hi everyone, thank you for today’s call (2024.02.22). This is the generated summary:
A new participant introduced themselves as a newcomer to the ICP ecosystem, expressing curiosity about ICP’s developments in AI. The group’s purpose was clarified as facilitating exchanges between projects working on decentralized AI on the ICP and as a resource for anyone interested in the topic. The discussion included the establishment of a repository to share resources, projects, and facilitate collaboration within the ecosystem (see the DeAI GitHub repo: GitHub - DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer: All things DeAI on the Internet Computer. The DeAI Working Group collects relevant resources in this repo.).

The conversation then shifted to the potential applications and advantages of decentralized AI on the ICP (also see DSCVR), including privacy concerns (also see this link on consent receipts as mentioned around privacy regulations: What is Consent Receipt?), the ability to run AI components like language models on-chain, and the interest in community or co-ownership models. The importance of keeping training data confidential and the potential for zero-knowledge machine learning AI was also highlighted.

The group discussed the concept of AI-powered DAOs, the potential for AI to manage financial transactions and assets, and the integration of Bitcoin and Ethereum networks with the ICP for enhanced financial services. The idea of AI participating in decision-making processes, especially in crucial areas like finance and public good decisions, was explored. The call also touched on the importance of transparency and hack resistance in decentralized AI systems.

The conversation also included technical details about the ICP’s upcoming features, such as end-to-end encryption for data privacy (VETKeys Primer | Internet Computer) and the development of AI models to detect DAO vote anomalies and secure online transactions.

The participants discussed the limitations and potential of AI models on the Internet Computer (IC), including the challenges posed by instruction limits and the efficiency of using different programming languages for AI development on the IC. There was an acknowledgment of the need for slimmed-down models to work within the IC’s current limitations, with a specific mention of the performance and usability constraints when working with GPT-2 models due to instruction limits.

A significant part of the discussion revolved around two main project ideas related to blockchain security and how these could be integrated into the upcoming hackathon. The first project idea involved creating public voting records on DAOs (Decentralized Autonomous Organizations) to facilitate analytics and possibly prevent malicious activities. The second idea focused on developing an AI model to detect blockchain hacks early by analyzing on-chain data and flagging suspicious activities.

The participants also considered the practical aspects of implementing these ideas, such as data generation and preparation of materials for the hackathon. There was a consensus on the importance of making these projects accessible to hackathon participants, possibly through a dedicated repository or as part of the hackathon’s challenge descriptions.

Additionally, the conversation touched on the potential for collaboration with the Scalability and Performance working group to discuss scalable architectures for decentralized AI applications. This collaboration could lead to a joint session focusing on the technical feasibility and efficiency of deploying AI models on the IC, comparing different programming languages and frameworks.

The call also highlighted the value of visual and interactive presentations, such as demos and walkthroughs, to make complex technical topics more accessible and engaging for a wider audience. There was an interest in recording and sharing these presentations to build a knowledge base that could support the broader community of developers interested in decentralized AI on the IC.

5 Likes
84

Hi everyone, thank you for today’s call (2024.02.29.). It was more informal as we were a smaller group. We thus mostly talked about the projects we’re working on.

These are some specific notes relevant for future calls:
Interest was expressed in talking more about how external computing resources (i.e. outside the IC network, like dedicated GPU networks or traditional cloud service providers) could be integrated and complement the infrastructure available to IC devs in their projects.
Concerning the different performance bottlenecks we’ve been talking about as a group (e.g. instruction limits, memory, GPU), it was proposed that we could discuss a divide-and-conquer approach where we collaborate as a group and members contribute by running different experiments around the performance metrics. For benchmarking, we could use a recently released tool for IC canisters (GitHub - dfinity/canbench: A benchmarking framework for canisters on the Internet Computer.), and then put the different experiment results into the benchmarking section in our DeAI repo (DeAIWorkingGroupInternetComputer/BenchmarkingStats at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub)

Please let us know which future topics you’d like to see on the agenda and which type of calls you’d like to have.

4 Likes
85

Hey DeAI folks,
For the next Scalability & Performance WG we’re going to invite you all to come and ask questions to a handful of the Dfinity experts on IC internals and performance. We’ll have @christian, @ielashi, @stefan.schneider, and @ulan so please come with questions/comments/discussion points!

The date will be March 14th at 5:30pm-6:30pm CET (zoom link). Note that the US starts daylight savings the Sunday before this, but not Europe so the time difference might be different than normally for some of you.

8 Likes
86

Hi @abk, thanks a lot for this great initiative, really looking forward to our session together!

1 Like
87

Hi everyone, thank you for today’s call and special thanks to @massimoalbarello who presented his and @ilbert 's work on integrating off-chain infrastructure! This is the generated summary (short version), please find the long version here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.03.07 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub

Massimo showcased the project he and Luca have been working on aimed at enabling DAOs to manage off-chain web3 services through an orchestrator canister on the ICP, focusing on decentralizing control and ensuring transparency by potentially governing it through a DAO. The project, still in early stages, looks to facilitate deployment on networks like Akash.

The discussion extended to integrating AI infrastructure on the ICP, highlighting the technical challenges of creating GPU subnets and decentralizing AI service management to enhance the platform’s capabilities. Key topics included the complexity of managing resources across different networks, the engineering hurdles in GPU design for AI, the significance of a DAO gateway as a bridge to external compute resources, and the importance of confidential computing for off-chain AI models.

Future directions involve exploring GPU-enabled VMs secured by confidential computing, ensuring payments in ICP, and fostering a community-driven approach to development. A forthcoming joint meeting with the Scalability Group aims to delve into scalability issues, indicating an interest in collaborative efforts to address the challenges of decentralized AI and performance on the ICP.

7 Likes
88

Check out the modulus presentation at https://www.youtube.com/watch?v=JZGm1WNU7IE&t=24205s. Might be relevant.

2 Likes
89

Thank you for sharing!

Relevant indeed as we just had a zkML/oppML session the other week :slight_smile:

I found this chart he drew pretty interesting

image
image692×389 36.5 KB

@apotheosis, where do you see the overhead of proving over computing currently and where do you think it’s going (and which factor is realistic to achieve)?

1 Like
90

The techniques for increasing prover speed are well known at this point.

  • Add more provers and parallelize. (Continuations, sharding)
  • Hand-craft a scheme like Modulus does with GKR.
  • Use zk-precompiles.

There are some smaller models that run locally and work fine for inference with ZK. There are some larger models that need to be run through specialized proving machine.

Proving ‘training’ is much further away.

2 Likes
91

where can we follow your status on Motoko ML? I am doing a first dive into ML I am interested in developing a scientific model.

1 Like
92

Thank you for your interest. The best way to follow the progress of Motokolearn is the github project page. Next is the linear algebra library. Hope soon I can dedicate more time and discuss more features.

3 Likes
93

ModClub recently published an article detailing my journey and experiments with AI on the Internet Computer. You can read about my experience here: The Internet Computer’s AI Frontier.

There’s still a considerable path ahead in refining the libraries and enhancing the efficiency of our tools. One of my immediate objectives involves a thorough analysis of the tools at my disposal, using Canbench. This effort will provide the community with insights, which will be shared through in the benchmarks section of the DeAI Working Group Git Repo.

4 Likes
94

Thank you for sharing the article which already has a bunch of interesting stats and insights and your initiative to collect more :muscle: Maybe we can coordinate our benchmark collections around Canbench

2 Likes
95

Hi everyone, thank you for today’s call (2024.03.21). This is the generated summary (short version, please find the long version here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.03.21 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub):
During the call, the DeAI Working Group delved into the potential and challenges of integrating AI models and specialized hardware with the Internet Computer (IC), exploring the utilization of both on-chain and side-chain resources. The discussion covered the significance of adopting common data formats like ONNX for AI model interoperability, and the role of ONNX runtime in facilitating AI deployments on the IC. The group emphasized the need for common tooling and protocols to aid dApp development, and explored federated learning as a method to train models collaboratively while keeping data private. The conversation also ventured into how the IC’s tokenomics could incentivize contributions to decentralized AI training and applications. The importance of interoperability, standards for growth, and the exploration of future AI applications, including the Metaverse, were highlighted. The discussions pointed towards a shared desire for more transparency, better coordination, and clearer communication within the community to harness the IC’s capabilities for complex computational tasks and to push the boundaries of what’s currently possible with GPU acceleration and AI integration.

6 Likes
96

When is the next meeting?

2 Likes
97

5 PM UTC every Thursday. Discord

2 Likes
98

that’s super interesting, we want to implement on a projects we started ckde.fi for automation

1 Like
99

Hi @Samer and @olahmoon , please join us for one of our meetings (the next one will be tomorrow) if you’re available. Would be great to get to know you and your projects and hear about your interests in DeAI :slight_smile: this is our channel in the IC dev’s Discord: ICP Developer Community Maybe talk to you soon

1 Like
100

Hi everyone, thank you for today’s call (2024.04.04). This is the generated summary (short version, please find the long version here: DeAIWorkingGroupInternetComputer/WorkingGroupMeetings/2024.04.04 at main · DeAIWorkingGroupInternetComputer/DeAIWorkingGroupInternetComputer · GitHub):
The DeAI Working Group call for the Internet Computer was a comprehensive discussion that began with an introduction by @SamDrissi , who shared his project’s aim to develop AI applications for intellectual property education and processes on the Internet Computer platform (Crafting Tomorrow's IP Standards Today with bipQuantum on ICP - #3 by SamDrissi). The group explored the role of AI in providing consultations and establishing proof of ownership for intellectual properties, alongside the challenges of data accessibility and the evolving legal landscape for AI-generated content. A significant focus was placed on the potential for improving AI performance through model quantization and the utilization of SIMD instructions, discussing the current capabilities and future possibilities of Wasm and the Internet Computer’s hardware architecture. The conversation also covered the trend towards integrated CPU and GPU architectures for more efficient AI model execution and speculated on future hardware developments that could further enhance the platform’s AI capabilities. The call concluded with an open invitation for future topic suggestions and collaborative contributions, emphasizing the group’s interest in benchmarking and sharing data to better understand and communicate the Internet Computer’s potential for AI applications.

shared during the call:

“Support for CPU acceleration. It will be possible for smart contract AIs to export data for processing directly on the hardware of node machines. This involves optimizations that allow the models to run deterministically, using newly developed fundamental AI science, and security checks that ensure smart contracts can’t cause non-determinism, since this would cause subnets to stop running. (We already have these techniques working).”
SIMD - Single Instruction Multiple Data (simd/proposals/simd/ImplementationStatus.md at main · WebAssembly/simd · GitHub)
Advanced Vector Extensions
GitHub - WebAssembly/wasi-nn: Neural Network proposal for WASI

4 Likes
101

@jeshli, @mnl: Raheel and Dominic pointed me to this thread and the discussion about ONNX runtimes: Tract and Burn. I thought it might be useful for the community to share my findings:

  • I managed to get the upstream Tract working on IC without any changes (i.e. without forking).
  • The approach relies on the wasi2ic community project.
  • I first compile Tract to wasm32-wasi and then use wasi2ic to convert the binary to wasm32-unknown-unknown that is compatible with the IC.
  • Example usage: code
  • Build steps of the example: code

Did anyone manage to compile Burn to IC? I would be curious to compare its performance with Tract.

11 Likes