Hi All,
Some of you reached out asking why I made icpp-llm repository private.
The main reason is that I wanted to bring some sanity to the topic of LLMs running in canisters of the IC.
One has to understand that it is a very interesting and promising field of research. But that really is what the state-of-the-art is. It is R&D, and a lot of hard work still has to be done to make it a reality.
To enable sharing of my research in LLMs running in canisters, I created a dApp that allows collaborators and partners to exercise the available & deployed LLMs.
The name of my dApp is ICGPT Labs.
- ICGPT = Internet Computer Generative Pre-trained Transformers
- Labs = it is research! We are making amazing strides, but the models today are still small
The Beta Preview of the frontend has now been deployed. It is still a little rough around the edges, but I think it is already very cool to see a live token stream coming back from the LLM canister.
Feel free to try it out and give me feedback. You must login with your Internet Identity, and then you can build stories. All real, all live on main-net !.
You can find here: https://icgpt.icpp.world/
A few screenshots to wet your appetite:
The login screen :
The New Chat page allowing you to select the LLM you want to try and an input area to provide the prompt. Right now, we only have one LLM deployed, the Tiny Stories model with 15M parameters which has been the workhorse of my R&D so far:
Once you click Submit, the frontend connects to the selected LLM and starts the inference.
The tokens stream in progress!
Once the initial part of the story has been build, you can continue to build it out, or start over.