Technical Working Group DeAI

Developers
204

Hi All,
cross-posting here for awareness. Looking forward to feedback on this pre-release of ICGPT with a llama.cpp backend.

2 Likes
205

I’m getting the following error using TintStories:

ERROR: The canister TinyStories-42M-Raw LLM is not ready...

…and this with the llama.cpp:

As an AI assistant, I’m here to provide information and answer questions related to information technology and technology. I am here to assist you in a variety of ways, including but not limited to: - Answer questions - Give examples - Offer solutions - Provide references In addition to answering questions, I can also provide information about different fields, offer solutions, and offer references.<|im_end|>

great to see a start on this stuff!

206

Hi @paulous ,
thank you for trying it out.

Another user reported the error with the TinyStories canister. It works OK for most users, but some get this error. Not sure yet why, but looking into it. Can you tell me a bit more detail about your test environment. Are you trying it out on laptop or mobile, and what browser are you using?

About the Qwen2.5 giving that canned response. It is very interesting, because it started doing that after a few hours in operation. Also not clear yet why it is happening, and looking into it.

1 Like
207

my pleasure :slightly_smiling_face:
i was using arch based pc and brave, all up to date.
sounds like fascinating work.

1 Like
208

@paulous ,
I fixed the Qwen2.5 behavior.

I turned out that I was not resetting the kv-cache correctly, and it ended up converging on some behavior that produced the same answer all the time.

Please try it again.

A good question to ask is: What is the difference between a chicken and a turkey? :upside_down_face:

209

Again cross posting here for awareness within the DeAI working group:

1 Like
210

Hi everyone, thank you for today’s call (2024.10.17). This is the generated summary (short version, please find the long version here ):

In today’s DeAI Working Group call for the Internet Computer, the focus was on GPU support for AI applications, exploring two approaches: a specific AI inference API and a more generic GPU exposure. The group discussed the challenges of GPU determinism, the potential flexibility of enabling various use cases, and security considerations. Preliminary testing on NVIDIA A100 cards showed promise for determinism, but further research is needed. The group leans toward the more flexible, generic approach despite its complexity, with a potential launch timeline in 2025 depending on technical readiness and hardware procurement.

Links shared during the call:

3 Likes
211

Hi everyone, thank you for today’s call (2024.11.07). This is the generated summary (short version, please find the long version here ):

The DeAI Working Group for the Internet Computer discussed the challenges and opportunities of integrating advanced hardware capabilities, such as high-performance GPUs and PCIe Gen 5 cards, into the ecosystem. The conversation focused on designing a flexible architecture that supports both WebAssembly (WASM) and more optimized computation models like MLIR, enabling efficient execution of AI workloads. Emphasis was placed on ensuring future-proofing and scalability, balancing cost and performance, and leveraging industry-standard protocols like CXL for high-bandwidth memory and CPU interconnects. The group highlighted the importance of collaboration and planning to facilitate a seamless transition to the next generation of node hardware.

Links shared during the call:

4 Likes
212

Hello everyone, thank you for the short call today!

Please see the notes.

At a high-level, we’ll be sharing more takeaways and action items from the Crypto:AI Conference in Lisbon, Portugal, and Palo Alto AI x Web3 Summit in Palo Alto, USA starting next week.

Please also sign the Manifesto for Decentralized AI:

4 Likes
213

Great initiative.
I wonder what AI tools could be built to protect communities from scams!

Very best wishes,

214

That’s great, but how do I join?

215

Hi there, we meet each Thursday in the ICP Developer Community Discord. We use the voice channel for the call:

Looking forward to having you :+1:

216

Hi everyone, thank you for today’s call (2024.11.28) - our one-year anniversary :rocket:This is the generated summary (short version, please find the long version here ): The DeAI community discussed efforts to encourage participation through the DeAI Manifesto and highlighted strategies to improve group effectiveness after a year of calls. Key initiatives include organizing monthly demo sessions, creating educational content on Rust and C++, and leveraging GitHub to support new developers. Peer learning, hackathons, and a proposed biannual symposium were identified as opportunities for collaboration and showcasing progress. Infrastructure-focused sessions and integration with the Scalability and Performance Group were emphasized, alongside a need for effective outreach strategies to educate audiences on decentralized AI. The call underscored innovation, education, and collaboration as priorities for enhancing the DeAI ecosystem on the Internet Computer. Links shared during the call:

1 Like
217

Hi everyone, @icpp and I set up a proper domain for the DeAI Manifesto: https://www.deaimanifesto.com/ :rocket:
Please sign it there and share :+1:

Individuals can sign it right on the webpage.

For orgs/projects, we’ve set up an easy process to sign it and get your logo etc up on the page: ManifestoForDecentralizedAI/README_signAsOrganization.md at main · DeAIWorkingGroupInternetComputer/ManifestoForDecentralizedAI · GitHub

1 Like
218

Hi everyone, thank you for today’s call (2024.12.05). This is the generated summary (short version, please find the long version here ): The DeAI Working Group discussed enhancing NFTs with AI and AR, organizing fixed monthly sessions with specific themes, and boosting participation through new tools like Discord and live streams. Key focus areas included promoting the DeAI Manifesto via outreach, live events, and collaboration with other ecosystems like Fetch AI. Plans for exploring tools such as Eliza and integrating them into the ICP ecosystem were highlighted. Action items included creating an AR NFT demo, finalizing a live stream schedule, and preparing for structured sessions starting January to advance decentralized AI initiatives and expand community engagement globally.

Links shared during the call:

1 Like
219

DeAI from NVIDIA:

1 Like
220

The decentralized AI generation platform envisioned by Super Protocol seems to overlap with the DeAI concept proposed by Dfinity, doesn’t it? While there may be differences, such as consensus across multiple GPUs, aspects like privacy protection, dataset verification, and consensus mechanisms to prevent fraud appear to be quite similar. Nvidia seems to be backing Super Protocol, but wouldn’t it also be necessary for Dfinity to consider purchasing GPUs capable of confidential computing?

2 Likes
221

The Super Protocol looks interesting indeed. And agreed, ICP will either need to integrate similar confidential computing GPUs into the network nodes or integrate with a system like the Super Protocol to provide the AI computation part of the functionality (and orchestrate the rest on ICP directly).