Behind the Scenes: The Technical Backbone of ELNA's AI Ecosystem

Developers Programs & Applications
1

The response to our initial announcement about ELNA on the Dfinity Forum has been nothing short of inspiring. The enthusiasm and interest that the ELNA project has garnered within the ICP community are a true testament to the potential of democratized AI. We are deeply appreciative of the hundreds of requests pouring in for the initial beta-testing waitlist, as well as the numerous messages of support and engagement on Twitter and the forum. We extend our heartfelt gratitude to the ICP community for the warm welcome and encouragement. Your enthusiasm fuels our determination to explore the technical intricacies that drive ELNA’s AI ecosystem.

image
image1400×1400 202 KB

As we delve into the core of ELNA’s technical architecture, we aim to address some common queries that have emerged:

Building the Foundation

ELNA’s inception was rooted in a vision of democratizing AI, enabling users to craft personalized chatbots tailored to their distinct requirements. To manifest this vision, we knew that a robust technical foundation was essential - one that facilitated seamless customization, secure deployment, and efficient data management. It’s the decentralized attributes of the Internet Computer that offered the ideal foundation upon which to construct this revolutionary platform. However, we have encountered some hurdles on our path:

  • Developing AI within Canister Web Assembly Environment: One of the challenges we’ve tackled is developing AI technologies optimized for the canister’s web assembly environment.
  • Instruction and Memory Limitations: The instruction and memory limitations of canisters pose constraints, particularly when dealing with the inference of larger models.
  • Balancing Speed, Scale, Privacy, and Transparency: We are working on maintaining a delicate balance between speed, scale, privacy, and transparency while implementing AI functionality on a decentralized infrastructure.

Advancements on the Horizon

Moving forward, we are optimistic about the advancements that are being made within the Dfinity ecosystem to address these challenges. Some notable improvements include:

  • Latest WebAssembly Stable Memory: The incorporation of the latest wasm-native stable memory provides a solid foundation for AI development within the canister environment.
  • Advanced Quantization and Distillation Techniques: These techniques are poised to minimize the size and instructions required for fine-tuning and inferencing, without compromising performance.
  • Enhanced Data Architectures: Ongoing efforts in data architecture and design are aimed at optimizing vector embedding and data handling.
  • Continual Evolution of LLM Models: The ever-evolving landscape of larger language models (LLMs) will continue to provide improved AI capabilities for ELNA.
    elna-live.drawio
    elna-live.drawio772×741 39.2 KB

The Path Forward

Our roadmap reflects our commitment to refining and expanding ELNA’s capabilities:

Elna-chatbot-001
Elna-chatbot-0011898×924 202 KB

  • Phase 1: Wallet integration and vector embedding for user data on the canister.
  • Phase 2: Full inference engine deployment within canisters.
  • Phase 3: Introducing fine-tuning capabilities within canisters, including methods like QloRA for efficient storage of weights.
  • Phase 4: Exploring standalone Large Language Models (LLMs) akin to advanced AI agents within the Internet Computer ecosystem.

Join Us in Shaping the Future

The journey ahead for ELNA is a collaborative one. We invite developers, researchers, enthusiasts, and the entire ICP community to actively participate in shaping the future of AI on the Internet Computer. Your insights, feedback, and engagement will be pivotal as we continue to innovate, refine, and expand ELNA’s capabilities.

Elna-chatbot-003
Elna-chatbot-0031898×924 65.5 KB

Stay connected with us on Twitter and visit our website to stay informed and engaged with ELNA’s evolution. As we embark on this transformative journey, we look forward to your contributions and ideas, propelling us closer to a future where AI is harnessed by and for the community.

Feel free to reach out with any questions, thoughts, or suggestions. Together, we’re pioneering the next era of AI on the Internet Computer.

Join the initial waitlist for alpha access to ELNA : https://forms.gle/SRP4PoM5AnSERZet7

6 Likes
3

Thank you for your enthusiastic support and kind words! We’re thrilled that you share our vision for personalized LLMs on the IC. :star2:

Your project Llama2.c for running LLM in a canister is a wonderful demonstration of the possibilities of canisters.

We’re absolutely planning to make our project open and collaborative. Stay tuned for updates as we prepare to share the GitHub repos. Your contribution and collaboration will play a pivotal role in shaping the future of ELNA’s AI ecosystem. Together, let’s usher in a new era of community-driven AI innovation!

1 Like
4

I’m the creator of UncensoredGreats.com (chatbots based on collections of Ebooks), for which I’m currently looking to migrating from my web2 stack with ICP. To get this working 100% on-chain, I need to ditch my vectordb and model hosting providers.

Is your intended model as an independent D-app, or Vector DB and AI as a service (i.e., will you encourage ‘user canisters’ from 3rd party dapps)? For me this could either either mean the niche is filled, or a big problem solved.

3 Likes
6

@branbuilder Exciting to see the IC get the first AI project with LLMs on a blockchain, actually I do not think any other blockchain can do this.

I’ve been using desktop LLMs, and they are becoming quite good, especially the ones based on the latest Llama code from Facebook.

Since Georgi Gerganov from Bulgaria took Facebook’s code and forked it by using C++ instead of Python we have been able to run pretty good LLMs on our desktop without a Nvidia card.

So it was a question of time until this would reach the IC, I am glad it has.

For those interested, you can start here and run your own LLM in your Mac:
Georgi Gerganov C++ port of Llama is here.

And here is the main app that uses it for your desktop:

Search and you will see many decentralized implementations that run now in your desktop with good results.

Best of luck folks at ELNA, it’s great to see this next step.

2 Likes
9

I can speak for myself on this one.

My project focuses on books as training data, with the actual e-reader part next to the chatbot part. This way folks can see hallucination/AI bias on opinionated issues, in real time.

The issue I ran into though is that this that my desired level of transparency opens me up to copyright issues. Even translations of the works of Socrates on my site are owned by big publishers >100 years later. And good luck getting permissions for such things.

OpenAI and friends seem to bypass copyright law by centralizing/closing everything, which makes me very sad. So I’d like to bypass copyright law by opening/decentralizing everything. The way to do this ethically, I hope, is with a DAO model where the rightful author can always claim rewards/ownership of their work.

This opens up doors for training data that is the otherwise contentious/philosophical stuff chatgpt never gives straight answers to. Hence the name “UncensoredGreats”, but uncensored really needs to be dao-censored for obvious reasons.

So for me decentralization is required to do cool chatbots. If your question about the backend was for me before, you’re right, no long-term memory because of the extreme emphasis on true-to-the-source origin tracking. It’s also all open-source, and I have an unpublished lightpaper if you were curious about the details.

5 Likes
10

@evanmcfarland unless you have vast amounts of resources, including money I would advise to go the route of Satoshi Nakamoto if you plan to target people who can sue you for years.

Become anonymous, and make an anonymous DAO.

Otherwise refrain from being an easy target of lawsuits.

Best of luck, you do have a noble vision translating the great works is a good thing.

P.S. If you can DM me your lightpaper, AI is a topic I follow, and this is a great way to use it.

2 Likes
11

Good to know.

Please let us know on this Forum if Llama CPP can run on a canister, I am very curious since I see some possible usage for future projects myself.

Best wishes with the project!

1 Like
12

Definitely interested in the lightpaper.

Is there a demo/beta version where you can test a chatbot with a book?

2 Likes
13

@evanmcfarland , UncensoredGreats.com certainly piques our interest. ELNA’s vision aligns perfectly with endeavors like yours, aiming to provide a platform and tools for individuals to bring their LLM-powered ideas to the ICP ecosystem. This entails leveraging Vector DB, an inference engine, and instruction fine-tuning.

We’re currently in the early alpha testing phase with our inference engine and operational DB. Although there are some limitations, we’re confident in our ability to overcome them and ensure a seamless experience.

lets stay in touch contact us on contact@elna.live or in twitter https://twitter.com/elna_live fro any support

more than happy to support you in implementing the same

2 Likes
14

@icpp , your insightful comments and contributions are greatly appreciated. Our initial discussions have revealed substantial synergies, and we’re excited to explore the potential of a decentralized LLM infrastructure within the Internet Computer ecosystem. The journey ahead holds promising opportunities for collaboration and growth. WAGMI indeed!

15

@josephgranata, your dedication and engagement serve as a motivating force for our continued progress. It’s inspiring to know that you’re exploring AI initiatives like GPT4all on your Mac. In fact, such experiences were one of our early inspirations to embark on a similar endeavor within the ICP domain. Your support is invaluable as we move forward.

Collectively, these interactions and insights underscore the vibrant and dynamic nature of the ICP community. We’re eager to contribute to this ecosystem and pioneer new horizons for AI on the Internet Computer. Your engagement fuels our determination, and we look forward to sharing more updates with you all.

#ELNA #ICPCommunity #AIInnovation

1 Like
16

Its working in canister.

Currently we have some limitations with memory and instructions to run a larger model, hopefully @icpp and our team ELNA should be able to solve it for the IC community

1 Like
17

Can you tell us a little bit about the background of the team, the number of people and the main developers?

2 Likes
18

Yes, it’s live on the site (Web2 version). And sent ya the litepaper :slight_smile:

2 Likes
19

Thank you for the reply, Evan!

Ever since ChatGPT came out, the use case of conversing with books or documents was what I was waiting for the most - yet nobody (to my knowledge) was tackling it the right way. Until now!

In my opinion, if eventually the UG DAO can fulfill your vision, UncensoredGreats can become one of the most important projects on the IC!

P.S. I’d like to point out that the vision and scale of what UG can be is greater than most if not all SNS funded projects to date!

P.S.2 I sense that the team behind UG are either crypto veterans or backed by ones.

1 Like