Technical Working Group DeAI

Developers
1

Hi everyone, @icpp and I have been talking and thought it would be great to have a Technical Working Group around DeAI where we can e.g. exchange ideas, initiatives and updates on relevant IC features. So we’re reaching out with this post to get your feedback on this idea; Who would be interested in joining such a group? What kind of content would you like to see and discuss there?
Happy to hear everyone’s feedback!
Have a great week, Pat

36 Likes
3

I’d be interested in joining for sure

9 Likes
4

That’d be awesome, cheers! Let’s see how much initial interest there is but we could also start with a smaller group of us at first :slight_smile:

3 Likes
5

bout time… count me in!

4 Likes
6

I’d like to join too, very interested in discussing what Kybra needs to support AI use cases.

8 Likes
7

I want to join this tech group. I am an AI researcher currently working on the second milestone of a grant to develope a machine learning motoko library to train models on chain.

12 Likes
8

I’m interested.

Would be helpful to have a community push for getting information about the DeAI subnets Dom suggests are under development.

My current focus is also on migrating to an on-chain vector databases, which needs community discussion.

5 Likes
9

Great, thank you for your responses! These sound like super interesting projects, I’m looking forward to learning more :slight_smile:

4 Likes
10

For our first meeting, I’d like to propose this Thursday, Nov 23 at 5pm UTC.

Agenda draft:

  • Round of introductions to get some context on each of our projects/work and goals in terms of AI on IC
  • Capture an overview of the tech each of us is using and any lessons learnt, best practices and current challenges
  • Start a “wish list” of features/ressources/… that would help the projects’ development
  • @mnl, if you can join and are open to sharing it, I think it’d be interesting to hear about DFINITY’s current work around DeAI and the roadmap
  • Decide on initial frame for the working group (e.g. cadence, meeting day/time, structure, typical agenda items, organization tools)
  • Time to discuss specific topics of interest

What do you think and do you have any other items you’d like to see on the agenda?

I’d schedule us for an hour and if anyone needs to leave earlier, they can let me know and we’ll make sure to capture their perspective and ideas before they need to head out.

How does this sound to you?

12 Likes
11

I’m interested too. Sign me up

4 Likes
12

That’s good idea
:smiling_face_with_three_hearts: :heart_eyes: :star_struck: :kissing_heart: :partying_face:

2 Likes
13

Hi everyone, I’ll also post the link to the video meeting (Google Meet) for Thursday here into the group but if anyone wants to get a direct invite and have it on their calendar, please message me with your email address and I’m happy to invite you.

We’re thinking to record the meeting (and future ones) and also publish some meeting notes to build up a knowledge base and allow anyone who can’t join to quickly catch up. Would this be fine?

Looking forward to meeting you all!

6 Likes
14

Hi everyone, this is the link for our call tomorrow (Thu Nov 23 5pm UTC): https://meet.google.com/ddh-jnpq-dji
Looking forward to meeting you all!

6 Likes
15

See you all in an hour :slight_smile:

2 Likes
16

Sorry the end got cut; first, thank you all for joining, it was great to meet you and hear about your projects and ideas!

I will put out another post about how we could organize ourselves going forward. If you have any ideas for this (or anything else), please post them here or message @icpp or me.

I’m looking forward to continue the conversation :slight_smile:

and for everyone in the US, happy Thanksgiving!

8 Likes
17

Hi everyone, as mentioned, it’d be great to have your input on how we’d like to organize ourselves going forward.

First, I’d like to invite feedback around our first meeting yesterday; which aspects should we keep, which ones should we reduce, which new ones should we add?

We had planned to also discuss some administrative things yesterday but ran out of time to do so (I hope you enjoyed the lively discussion though :slight_smile: ).
@icpp and I had a few ideas around this that I’ll put here as a starting point. It’d be great to have your views and ideas here as well.

  • Weekly call on Thursdays at 5pm UTC for 30-60min (initially, and we can see how we’d like to adapt this)
  • Record the meetings, automatically extract the script and a summary from it
  • Manually write down any important decisions or action items
  • Tools: OpenChat community with channels for topic-specific discussions and to keep the recordings/notes

Common agenda items:

  • Summary of news (related to DeAI on IC and beyond)
  • Project updates: what’s done, plans/milestones, blockers
  • Action items
    • Living doc of “wish list” of features/resources/… that would help the projects’ development (IC and beyond)
    • Dashboard for IC DeAI projects (e.g. projects overview, stats)
    • Standards (e.g. APIs)
    • Documentation/education, e.g. how to get started (incl. programming languages, tech stack, resources, examples)

What are your thoughts around these? What else do you think is important?

If you like, please also share your priorities and which goals you think the WG should have.

Thanks and have a great weekend!

5 Likes
18

Hi everyone, I’m looking forward to our second meeting this Thursday, 11/30, 5pm UTC. I will post the link to the video call here in the group. If you like to get a direct invite, please message me with your email address.

Agenda-wise, it’d be great to decide on a few administrative things (see previous post) and discuss which action items and goals/initiatives we’d like to tackle within the group. What else would you like to see on the agenda?

Looking forward to seeing everyone,
Pat

19

Hi everyone, this is the video call link to our meeting later at 5pm UTC: https://meet.google.com/ddh-jnpq-dji

Looking forward to talking to you all!

1 Like
20

Thank you for joining today, it was great talking to you again!

These are some of the things we discussed:

6 Likes
21

Thanks for the discussion. Re-sharing some of the links to the points discussed earlier.

Dapps:
ICGPT - Canister LLM CPU-inferencing
DeVinci App - Canister LLM WebGPU-inferencing

ML Programming:
C++ CDK - C++ dependencies and cpp models
Rust Burn - ML framework in Rust
Tinygrad - ML framework in pure Python

Distributing AI compute:
RingAttention - Distributing long sequences across multiple devices
PetalsML - BitTorrent style ML computation
Stable Horde - Distributed cluster of text and image generation workers

Lightweight AI models:
TinyLlama 1.1B - Small but coherent LLM
Qwen 1.8B Chat - Small but coherent LLM
SSD-1B - Distilled version of SDXL (can use LCM version too)
Gorilla LLM - LLM trained to perform API calls (agentic software)

LLM structured output:
LMQL, Guidance, Guardrails - Enabling structured query output for LLMs consistently (ex. JSON)

10 Likes