ArcMind AI - Autonomous AI Agent and Vector DB

kinwo · February 11, 2024, 8:39am

Project highlights

The objective of ArcMind AI is to bring applied AI to the Internet Computer and ignite new ideas from developers to make use of the foundation components we developed for their projects.

We start with a LLM-powered (GPT3/4) Autonomous AI Agent that uses Chain of Thoughts for reasoning, taking actions and completing goals. As part of the AI Agent, we also developed a Vector Database with Rust running on IC. It is initially used as long-term memory for Autonomous Agent for semantic search (multiple dimensions similarity search) and adding context to the chat prompt. However, its potential use cases can extend to AI-powered applications like recommendation engine and Retrieval Augmented Generation (RAG).

Our Main App Website: https://arcmindai.app

What is Autonomous AI Agent?

A typical AI agent acts on behalf of humans to achieve a goal. Unlike ChatGPT or AI chatbots, given a goal, ArcMind AI can break down the goal into multiple subtasks, plan, reasoning, self-criticism, suggest the next action, and act on it until the goal is reached. It can interact with the real world in real time to explore for new information (not in the LLM training data). The more tools we provide, the more variety of goals it can achieve. At the moment, ArcMind AI can perform “Start LLM Agent”, “Browse Website”, “Google Search” and “Smart Contract Call” via “BeamFi Plugin” for streaming payment.

This process is called “Chain of Thoughts”. It is a novel idea in AI research but has become increasingly popular as “Chain of Thoughts” can increase the faithfulness of LLM and its accuracy. By letting LLM to “speak things out loud”, it usually performs better than typical AI chatbots. Also, it creates a more serious use case in the future by incorporating Autonomous AI Agents in enterprise corporate systems as human’s AI workforce.

Chain of Thoughts Flow Diagram

Chain of thoughts Sample Screenshots

Read the full sample chain of thoughts result screenshots here

How is it built

ArcMind AI and Vector DB are developed with Rust Language:

It is composed of 4 canisters.

Controller
Brain
Tools
Vector DB

Controller acts as the Main Loop canister who has the sole ownership and access to Brain, Tools and Vector DB. When a new instance of ArcMind AI is deployed to IC, our system would deploy all 4 canisters and assign controller’s ownership to the NFID user through our web app. If you deploy it locally, your dfx identity would become the ultimate owner of the whole suite of canisters.

We designed the system in this way such that one user will have the sole control over the ArcMind instance, not sharing with other users. However, it means there is an upfront cost of deploying all 4 canisters. We believe that it is a better approach than sharing canisters among users as we would like to focus on a privacy-centric approach.

To fit for the purpose, ArcMind AI adopts two LLM models. Brain canister uses GPT-4 while Tools canister uses gpt-3.5-turbo-1106 for post browse-website summarization process as gpt-3.5-turbo-1106 allows a bigger context window. GPT-4 performs better at the Chain of Thoughts process.

Architecture Diagram

Internet Computer superpowers

ArcMind AI leverages IC Timer to drive the whole Chain of Thoughts process when a new goal is sent by the user from the frontend. To reach out to LLM models, it uses HTTP Outcall with consensus. To workaround IPV6 limitation, we developed a HTTP Proxy running in Google Cloud so that Tools canister can browse any type of websites.

ArcMind strives to provide privacy-centric and decentralized Autonomous AI Agents. We achieve that by enabling users to maintain full control over their agents and data through NFID (Internet Identity) integration. Our ArcMind Vector DB is a privacy-first data store for long-term memory storage when users interact with ArcMind AI Agent. It is further secured by IC’s blockchain consensus.

Internet Computer’s multi-chain interoperability empowers ArcMind AI to make Smart Contract Call to other smart contracts in IC or other chain e.g Ethereum in an autonomous way. We developed a simple BeamFi Plugin to demonstrate streaming payment via AI Agent.

Open Source

ArcMind AI adopts an open source MIT license.

It is intended to encourage innovative thinking among developers.
By open-sourcing Vector DB components, other teams can integrate semantic similarity search or recommendation into their applications, strengthening the future of AI applications.

We have also open sourced the main ArcMind AI components comprising - Main Loop Controller Canister, Brain Canister, Tools Canister and BeamFi Plugin. We believe that it has a place in enterprise system integration with AI applications. It would most likely become a background autonomous agent as AI companion of your existing workspace tools like Slack, Discord etc. Its use cases are limitless.

Status of the project

ArcMind AI has reached beta status which means it can be used for general users with some limitations.

Occasionally, ArcMind AI agent could be looping by itself in the Chain of Thoughts process. In that case, we have built a safe-guard to allow users to pause and restart the agent. Internally, it has a maximum number of thoughts to process. Also, browsing websites may sometimes return invalid response, which the system would pass the failure status to ArcMind AI. With its Chain of Thoughts process, it can usually understand and seek workarounds.

ArcMind AI has a Long-Term memory store. If you ask it with reference to existing knowledge, it will understand and find the relevant information from memory or past experience. It will then determine if it should conduct a new Google Search to find new information or simply use past experience for response.

Our Web UI is designed to allow users to read the thoughts process clearly with easily collapse/expand button.

There is a limitation of processing one goal at a time. Please wait until the goal has finished before entering a new goal. This is more of frontend limitation as the backend canisters are capable to store and process multiple goals.

To keep the early beta version simple, when users submit a new goal, it clears the past goals. This can change in the future.

We have integrated a monthly payment subscription model to ArcMind AI with a Starter Plan USD $18.99. That is to cover the Canister provisioning fees and OpenAI fees.

Target User

General users who seek to improve their work productivity and creativity using AI but have concerns about privacy in other tools

Future Plans

There are multiple different paths we can take.

Doubling down on ArcMind AI by introducing self-hosted LLM models with better performance reducing network round trips
Integrating with Zapier / IFTTT to open up a whole new type of integration to other services, allowing users to define their own commands for the Chain of Thoughts process
Improve UI to show goals history and add ability to resume conversation from it
Add user feedback commands to allow ArcMind AI agent to proactively seek user feedback
Create a supervisor AI agent to oversee main Autonomous Agent to further improve faithfulness and safety

Read Full Roadmap

Questions?

Reach out to us at https://twitter.com/arcmindai or Email hello@arcmindai.app

Early Adopters Giveaway

Thanks for reading this article to the end. As a big thank you, we would like to offer a free first month subscription of ArcMind AI to 5 lucky users.

How to claim:

Visit https://arcmindai.app/
Sign up with NFID (create a NFID account first if you haven’t)
Proceed to Starter Plan subscription
Enter promotion code ARCMINDFREE to claim first month free subscription
It will then provision a new instance of ArcMind to your NFID principal account.
You can cancel the subscription anytime using My Plan page at the top right menu if you don’t want to pay next month but we would be very grateful if you can continue to support us.
We only have limited available instance for provisioning, if all available ArcMind AI instances have been subscribed, please join the waitlist.

Resources

Main App

Github ArcMind AI

Github ArcMind Vector DB

icarus · February 15, 2024, 1:05pm

Hello @kinwo and it is great to see the ArcMind AI project introduced to the Internet Computer community! I have registered an account and had a play with the AI agent (I should say “my AI agent” ) and it is impressive work. I look forward to trying it out further.

@kinwo you would be very welcome to join the Decentralized AI Working Group (DeAI WG) which has an informal structure but an active group of members, the group community coordinator is @patnorris.
There is a dedicated developer forum thread for the working group, a summary of each meeting is posted there along with other topical discussions: Technical Working Group DeAI - #80 by patnorris
We also converse in this dedicated ICP Developer Community Discord channel Discord
We also meet weekly on Thursday for a voice meeting, the next one is in only 4 hours, so if you can we would all appreciate you dropping in to tell us about ArcMind and your IC vector db implementation (vector dbs have been a live topic of discussion in the WG past two meetings)
ICP Developer Community

patnorris · February 15, 2024, 1:53pm

Thank you @icarus ! It would be great indeed to have you @kinwo (and any team members from ArcMind) in the DeAI working group. Our next meeting is today (Feb 15) at 5pm UTC in the voice channel on the ICP Developer Discord as linked to by icarus. Maybe see you then

evanmcfarland · February 16, 2024, 11:12am

What an interesting setup!

I subscribed and tested, except the ‘manage subscription’ page causes an internal server error. I’d like to remove my card and cancel my subscription through you if possible.

And some first experience impressions: I’ve used autogpt before and personally don’t see the use for autonomous agents at this stage, much less on blockchains. But I’m super impressed with the VectorDB implementation. It seems like the first open IC-native one, and a lot of folks from the DeAI Working Group need one, including myself. If it can be easily adapted to scale into a single production VectorDB, I’d lean into that and would love to collaborate.

Forreal · February 16, 2024, 8:50pm

Would having GPU subnets help extend the functionalities and UX of your project?

kinwo · February 16, 2024, 10:24pm

Thanks for the feedback and trying out Arcmind AI. Appreciated it. Also, my apologies for the manage subscription link issue. I have manually cancelled your subscription and removed your credit card details.

For Vector DB, indeed, I believe lot of other teams would find ArcMind Vector DB an easy drop-in to their own project.

That is the reason we made ArcMind Vector DB a completely separated Github project, independent of core ArcMind AI.

Feel free to jump to the Github:

Good starting point is the Canid file.

github.com

arcmindai/arcmindvector/blob/main/src/arcmindvectordb/arcmindvectordb.did

type PlainDoc = record { content : text };
type VecDoc = record { content : text; embeddings : vec float32 };
type VecQuery = variant { Embeddings : vec float32 };
service : (opt principal) -> {
  add : (VecDoc) -> (text);
  delete : (VecDoc) -> ();
  get_owner : () -> (opt principal) query;
  search : (VecQuery, nat64) -> (opt vec PlainDoc) query;
  size : () -> (nat64) query;
  update_owner : (principal) -> ();
}

For most use cases, all you need is add and search service.

To provision it:
You can use provision.sh and set the owner principal id as argument.
That makes sure only the owner principal can access the canister.

github.com

arcmindai/arcmindvector/blob/main/scripts/provision.sh

#!/bin/bash

# Validate required env vars
if [[ -z "${CONTROLLER_PRINCIPAL}" ]]; then
  echo "CONTROLLER_PRINCIPAL is unset."
  exit 1
fi

# To deplopy locally, update IC_NETWORK to local. To deploy to ic, update IC_NETWORK to ic.
IC_NETWORK=${IC_NETWORK:-local}

# Deploy vectordb canister 
dfx deploy --network $IC_NETWORK arcmindvectordb --argument "(opt principal \"$CONTROLLER_PRINCIPAL\")" > arcmindvectordb_deploy.txt

VECTOR_PRINCIPAL=$(dfx canister --network $IC_NETWORK id arcmindvectordb)
echo $VECTOR_PRINCIPAL

As with any Vector DB, make sure you use the same embedding model for add and search.
In my case, I use OpenAI text-embedding-ada-002 for text embeddings.

Apart from text, ArcMind Vector DB can work with image and audio embeddings.

For images, it would need a CNN (Convolution Neural Network) image classification model and extract the last layer (just before the classification) of the network as embedding.

Some pointer here with CLIP.

I used to do it with VGG16 CNN 6 years ago with image similarity search.
It should be a lot easier now.

One thing to note is when you add a VecDoc for image, the content could be a image ID so that you can link it back to your main DB to original image. For text, the context could be your text doc ID or full text or partial text depending on your use case.

Also, we normalize the input vector size to 768. You may want to experiment with this number to work out the best performance for your use case.

github.com

arcmindai/arcmindvector/blob/main/src/arcmindvectordb/src/embeddings.rs

use crate::datatype::Embeddings;

pub const EMBEDDING_SIZE: usize = 768;

pub fn normalize_embeddings(embeddings: Embeddings) -> Embeddings {
    let mut embeddings = embeddings.clone();
    embeddings.resize(EMBEDDING_SIZE, 0.0);
    return embeddings;
}

Hope that helps.

Cheers

kinwo · February 16, 2024, 10:37pm

Yes, absolutely. We are kind of in the chicken-and-egg problem. ArcMind AI currently uses OpenAI API GPT-4 for LLM inferencing. It takes a lot of network round-trip for chain of thoughts.

If we can host LLM directly on Internet Computer some ways, that would be great. It would significantly improve the overall performance and user experience.

Also GPU and GPU clusters is getting increasingly important for LLM inferencing/serving stage not just machine learning, as new LLM is growing bigger. LLM works best if it can fit in the weights to high speed memory.

A good reference here.

One of the future improvement we are considering is hosting some kind of light-weight open source LLM in IC to start with and see how it performs.

Also, we use text embedding model to generate vector for long-term memory Vector DB. If we can host it directly in IC, it would be really great for any other teams who need Vector DB for features similarity search, to power AI applications.

kinwo · February 16, 2024, 10:50pm

Thanks @ icarus, patnorris.

It is great to learn about DeAI WG. When I started working on ArcMind AI last year July, there were not much IC AI projects. Great to see this space is growing and getting traction especially Vector DB.

Since I live in Australia, I may not be able to join the late mid-night meeting but I will find other ways to contribute.

: )

q2333gh · April 18, 2024, 3:22pm

Really impressive work!
Learn a lot from the code and arch and design goals !
Thank you so much for your work!

Such a pity not too many people got passion on AI on ic

Cause AI is so useful in many aspect of life.

After i learn the course : Generative AI for Everyone - DeepLearning.AI

BTW, ELNA on ic also have some ai product ,
just wanna share some info here : Repositories · GitHub might help guys

Topic		Replies	Views
Introducing Cipher AI Vault - A fully sandboxed AI demo w/ memory Showcase	9	222	September 12, 2024
AI player on ICP - IChess Showcase Functional-Programming , Discussing , community-consideration	9	400	October 9, 2024
The Current State of the Art of AI and Blockchains General	7	148	November 14, 2024
Browser-based AI Chatbot Served From The IC General	45	2530	December 11, 2024
AI LLM via Canister/Cycles? Programs & Applications	6	740	February 12, 2024

Project highlights

What is Autonomous AI Agent?

How is it built

Internet Computer superpowers

Open Source

Status of the project

Target User

Future Plans

Questions?

Early Adopters Giveaway

Resources

Related topics