# Behind the Scenes: The Technical Backbone of ELNA's AI Ecosystem

**URL:** <https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412>\
**Category:** Programs & Applications\
**Created:** [August 27, 2023, 1:02pm UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412 "2023-08-27T13:02:45Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![branbuilder](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/branbuilder/32/14889_2.png) [@branbuilder](https://forum.dfinity.org/u/branbuilder)\
**Post date:** [August 27, 2023, 1:02pm UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/1 "2023-08-27T13:02:45Z")

</div>

_The response to our initial announcement about ELNA on the Dfinity Forum has been nothing short of inspiring. The enthusiasm and interest that the ELNA project has garnered within the ICP community are a true testament to the potential of democratized AI. We are deeply appreciative of the hundreds of requests pouring in for the initial beta-testing waitlist, as well as the numerous messages of support and engagement on Twitter and the forum. We extend our heartfelt gratitude to the ICP community for the warm welcome and encouragement. Your enthusiasm fuels our determination to explore the technical intricacies that drive ELNA’s AI ecosystem._

 ![image](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/2X/e/ed841193b4563ea548c00c42c33929998186a35d.jpeg)

As we delve into the core of ELNA’s technical architecture, we aim to address some common queries that have emerged:

**Building the Foundation**

ELNA’s inception was rooted in a vision of democratizing AI, enabling users to craft personalized chatbots tailored to their distinct requirements. To manifest this vision, we knew that a robust technical foundation was essential - one that facilitated seamless customization, secure deployment, and efficient data management. It’s the decentralized attributes of the Internet Computer that offered the ideal foundation upon which to construct this revolutionary platform. However, we have encountered some hurdles on our path:

- Developing AI within Canister Web Assembly Environment: One of the challenges we’ve tackled is developing AI technologies optimized for the canister’s web assembly environment.
- Instruction and Memory Limitations: The instruction and memory limitations of canisters pose constraints, particularly when dealing with the inference of larger models.
- Balancing Speed, Scale, Privacy, and Transparency: We are working on maintaining a delicate balance between speed, scale, privacy, and transparency while implementing AI functionality on a decentralized infrastructure.

![](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/2X/5/5758dffb0aa17bd7826880a27f6657058f12c0cc.gif)

**Advancements on the Horizon**

Moving forward, we are optimistic about the advancements that are being made within the Dfinity ecosystem to address these challenges. Some notable improvements include:

- Latest WebAssembly Stable Memory: The incorporation of the latest wasm-native stable memory provides a solid foundation for AI development within the canister environment.
- Advanced Quantization and Distillation Techniques: These techniques are poised to minimize the size and instructions required for fine-tuning and inferencing, without compromising performance.
- Enhanced Data Architectures: Ongoing efforts in data architecture and design are aimed at optimizing vector embedding and data handling.
- Continual Evolution of LLM Models: The ever-evolving landscape of larger language models (LLMs) will continue to provide improved AI capabilities for ELNA.  
 ![elna-live.drawio](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/2X/2/20bc1a4f7639ece677d7e835b72ba6f8251910e7.png)

**The Path Forward**

Our roadmap reflects our commitment to refining and expanding ELNA’s capabilities:

 ![Elna-chatbot-001](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/2X/9/92ff813d977822b839c48855d5b237555949fe16.jpeg)

- Phase 1: Wallet integration and vector embedding for user data on the canister.
- Phase 2: Full inference engine deployment within canisters.
- Phase 3: Introducing fine-tuning capabilities within canisters, including methods like QloRA for efficient storage of weights.
- Phase 4: Exploring standalone Large Language Models (LLMs) akin to advanced AI agents within the Internet Computer ecosystem.

**Join Us in Shaping the Future**

The journey ahead for ELNA is a collaborative one. We invite developers, researchers, enthusiasts, and the entire ICP community to actively participate in shaping the future of AI on the Internet Computer. Your insights, feedback, and engagement will be pivotal as we continue to innovate, refine, and expand ELNA’s capabilities.

 ![Elna-chatbot-003](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/2X/4/424b772f7168c43712a26c4f8f5cc1f025b3d4b5.png)

Stay connected with us on [Twitter](https://twitter.com/elna_live) and visit our [website](https://elna.live/) to stay informed and engaged with ELNA’s evolution. As we embark on this transformative journey, we look forward to your contributions and ideas, propelling us closer to a future where AI is harnessed by and for the community.

Feel free to reach out with any questions, thoughts, or suggestions. Together, we’re pioneering the next era of AI on the Internet Computer.

Join the initial waitlist for alpha access to ELNA : [https://forms.gle/SRP4PoM5AnSERZet7](https://forms.gle/SRP4PoM5AnSERZet7)

---

<div class="post-metadata">

**Author:** ![branbuilder](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/branbuilder/32/14889_2.png) [@branbuilder](https://forum.dfinity.org/u/branbuilder)\
**Post date:** [August 28, 2023, 11:43pm UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/3 "2023-08-28T23:43:57Z")

</div>

Thank you for your enthusiastic support and kind words! We’re thrilled that you share our vision for personalized LLMs on the IC. 🌟

Your project Llama2.c for running LLM in a canister is a wonderful demonstration of the possibilities of canisters.

We’re absolutely planning to make our project open and collaborative. Stay tuned for updates as we prepare to share the GitHub repos. Your contribution and collaboration will play a pivotal role in shaping the future of ELNA’s AI ecosystem. Together, let’s usher in a new era of community-driven AI innovation!

---

<div class="post-metadata">

**Author:** ![evanmcfarland](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/evanmcfarland/32/1744_2.png) [@evanmcfarland](https://forum.dfinity.org/u/evanmcfarland)\
**Post date:** [August 29, 2023, 3:18pm UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/4 "2023-08-29T15:18:32Z")

</div>

I’m the creator of [UncensoredGreats.com](http://UncensoredGreats.com) (chatbots based on collections of Ebooks), for which I’m currently looking to migrating from my web2 stack with ICP. To get this working 100% on-chain, I need to ditch my vectordb and model hosting providers.

Is your intended model as an independent D-app, or Vector DB and AI as a service (i.e., will you encourage ‘user canisters’ from 3rd party dapps)? For me this could either either mean the niche is filled, or a big problem solved.

---

<div class="post-metadata">

**Author:** ![josephgranata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/josephgranata/32/21930_2.png) [@josephgranata](https://forum.dfinity.org/u/josephgranata)\
**Post date:** [August 29, 2023, 10:46pm UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/6 "2023-08-29T22:46:46Z")

</div>

@branbuilder Exciting to see the IC get the first AI project with LLMs on a blockchain, actually I do not think any other blockchain can do this.

I’ve been using desktop LLMs, and they are becoming quite good, especially the ones based on the latest Llama code from Facebook.

Since Georgi Gerganov from Bulgaria took Facebook’s code and forked it by using C++ instead of Python we have been able to run pretty good LLMs on our desktop without a Nvidia card.

So it was a question of time until this would reach the IC, I am glad it has.

For those interested, you can start here and run your own LLM in your Mac:  
[Georgi Gerganov C++ port of Llama is here.](https://github.com/ggerganov/llama.cpp)

And here is the main app that uses it for your desktop:

> **[GPT4All](https://gpt4all.io/index.html)**
>
> Free, local and privacy-aware chatbots

Search and you will see many decentralized implementations that run now in your desktop with good results.

Best of luck folks at ELNA, it’s great to see this next step.

---

<div class="post-metadata">

**Author:** ![evanmcfarland](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/evanmcfarland/32/1744_2.png) [@evanmcfarland](https://forum.dfinity.org/u/evanmcfarland)\
**Post date:** [August 30, 2023, 2:10am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/9 "2023-08-30T02:10:13Z")

</div>

I can speak for myself on this one.

My project focuses on books as training data, with the actual e-reader part next to the chatbot part. This way folks can see hallucination/AI bias on opinionated issues, in real time.

The issue I ran into though is that this that my desired level of transparency opens me up to copyright issues. Even translations of the works of Socrates on my site are owned by big publishers \>100 years later. And good luck getting permissions for such things.

OpenAI and friends seem to bypass copyright law by centralizing/closing everything, which makes me very sad. So I’d like to bypass copyright law by opening/decentralizing everything. The way to do this ethically, I hope, is with a DAO model where the rightful author can always claim rewards/ownership of their work.

This opens up doors for training data that is the otherwise contentious/philosophical stuff chatgpt never gives straight answers to. Hence the name “UncensoredGreats”, but uncensored really needs to be dao-censored for obvious reasons.

So for me decentralization is required to do cool chatbots. If your question about the backend was for me before, you’re right, no long-term memory because of the extreme emphasis on true-to-the-source origin tracking. It’s also all open-source, and I have an unpublished lightpaper if you were curious about the details.

---

<div class="post-metadata">

**Author:** ![josephgranata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/josephgranata/32/21930_2.png) [@josephgranata](https://forum.dfinity.org/u/josephgranata)\
**Post date:** [August 30, 2023, 2:50am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/10 "2023-08-30T02:50:07Z")

</div>

@evanmcfarland unless you have vast amounts of resources, including money I would advise to go the route of Satoshi Nakamoto if you plan to target people who can sue you for years.

Become anonymous, and make an anonymous DAO.

Otherwise refrain from being an easy target of lawsuits.

Best of luck, you do have a noble vision translating the great works is a good thing.

P.S. If you can DM me your lightpaper, AI is a topic I follow, and this is a great way to use it.

---

<div class="post-metadata">

**Author:** ![josephgranata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/josephgranata/32/21930_2.png) [@josephgranata](https://forum.dfinity.org/u/josephgranata)\
**Post date:** [August 30, 2023, 2:50am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/11 "2023-08-30T02:50:51Z")

</div>

Good to know.

Please let us know on this Forum if Llama CPP can run on a canister, I am very curious since I see some possible usage for future projects myself.

Best wishes with the project!

---

<div class="post-metadata">

**Author:** ![Berg](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/berg/32/6678_2.png) [@Berg](https://forum.dfinity.org/u/Berg)\
**Post date:** [August 30, 2023, 4:46am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/12 "2023-08-30T04:46:43Z")

</div>

Definitely interested in the lightpaper.

Is there a demo/beta version where you can test a chatbot with a book?

---

<div class="post-metadata">

**Author:** ![branbuilder](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/branbuilder/32/14889_2.png) [@branbuilder](https://forum.dfinity.org/u/branbuilder)\
**Post date:** [August 30, 2023, 6:13am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/13 "2023-08-30T06:13:15Z")

</div>

**@evanmcfarland** , [UncensoredGreats.com](http://uncensoredgreats.com/) certainly piques our interest. ELNA’s vision aligns perfectly with endeavors like yours, aiming to provide a platform and tools for individuals to bring their LLM-powered ideas to the ICP ecosystem. This entails leveraging Vector DB, an inference engine, and instruction fine-tuning.

We’re currently in the early alpha testing phase with our inference engine and operational DB. Although there are some limitations, we’re confident in our ability to overcome them and ensure a seamless experience.

lets stay in touch contact us on [contact@elna.live](mailto:contact@elna.live) or in twitter [https://twitter.com/elna\_live](https://twitter.com/elna_live) fro any support

more than happy to support you in implementing the same

---

<div class="post-metadata">

**Author:** ![branbuilder](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/branbuilder/32/14889_2.png) [@branbuilder](https://forum.dfinity.org/u/branbuilder)\
**Post date:** [August 30, 2023, 6:14am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/14 "2023-08-30T06:14:25Z")

</div>

**@icpp** , your insightful comments and contributions are greatly appreciated. Our initial discussions have revealed substantial synergies, and we’re excited to explore the potential of a decentralized LLM infrastructure within the Internet Computer ecosystem. The journey ahead holds promising opportunities for collaboration and growth. WAGMI indeed!

---

<div class="post-metadata">

**Author:** ![branbuilder](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/branbuilder/32/14889_2.png) [@branbuilder](https://forum.dfinity.org/u/branbuilder)\
**Post date:** [August 30, 2023, 6:15am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/15 "2023-08-30T06:15:22Z")

</div>

**@josephgranata** , your dedication and engagement serve as a motivating force for our continued progress. It’s inspiring to know that you’re exploring AI initiatives like GPT4all on your Mac. In fact, such experiences were one of our early inspirations to embark on a similar endeavor within the ICP domain. Your support is invaluable as we move forward.

Collectively, these interactions and insights underscore the vibrant and dynamic nature of the ICP community. We’re eager to contribute to this ecosystem and pioneer new horizons for AI on the Internet Computer. Your engagement fuels our determination, and we look forward to sharing more updates with you all.

#ELNA #ICPCommunity #AIInnovation

---

<div class="post-metadata">

**Author:** ![branbuilder](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/branbuilder/32/14889_2.png) [@branbuilder](https://forum.dfinity.org/u/branbuilder)\
**Post date:** [August 30, 2023, 6:17am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/16 "2023-08-30T06:17:58Z")

</div>

Its working in canister.

Currently we have some limitations with memory and instructions to run a larger model, hopefully @icpp and our team ELNA should be able to solve it for the IC community

---

<div class="post-metadata">

**Author:** ![MillionMiles](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/millionmiles/32/7014_2.png) [@MillionMiles](https://forum.dfinity.org/u/MillionMiles)\
**Post date:** [August 30, 2023, 8:24am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/17 "2023-08-30T08:24:34Z")

</div>

Can you tell us a little bit about the background of the team, the number of people and the main developers?

---

<div class="post-metadata">

**Author:** ![evanmcfarland](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/evanmcfarland/32/1744_2.png) [@evanmcfarland](https://forum.dfinity.org/u/evanmcfarland)\
**Post date:** [August 30, 2023, 11:16am UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/18 "2023-08-30T11:16:45Z")

</div>

Yes, it’s live on the site (Web2 version). And sent ya the litepaper 🙂

---

<div class="post-metadata">

**Author:** ![Berg](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/berg/32/6678_2.png) [@Berg](https://forum.dfinity.org/u/Berg)\
**Post date:** [August 30, 2023, 2:34pm UTC](https://forum.dfinity.org/t/behind-the-scenes-the-technical-backbone-of-elnas-ai-ecosystem/22412/19 "2023-08-30T14:34:56Z")

</div>

Thank you for the reply, Evan!

Ever since ChatGPT came out, the use case of conversing with books or documents was what I was waiting for the most - yet nobody (to my knowledge) was tackling it the right way. Until now!

In my opinion, if eventually the UG DAO can fulfill your vision, UncensoredGreats can become one of the most important projects on the IC!

P.S. I’d like to point out that the vision and scale of what UG can be is greater than most if not all SNS funded projects to date!

P.S.2 I sense that the team behind UG are either crypto veterans or backed by ones.
