Technical Working Group DeAI

[icpp]

Again cross posting here for awareness within the DeAI working group:

Technical Working Group: Scalability & Performance

Hi scalability team, I am really happy with the fact that the 0.5billion parameter LLM is running stable now in a 32bit canister, and it is giving very good answers. You can try it out at https://icgpt.icpp.world/ You will notice right away that the token generation is slow, and the main reason for that is the instructions limit on update calls. This LLM can generate 10 tokens within the instructions limit. The way we work around it is by doing a sequence of update calls, and using the prom…