AI LLM via Canister/Cycles?

I see a lot of AI projects out there working on LLM solutions built on the internet computer. I was wondering if anyone was aware of any project that is working on one that allows for payment of generation with cycles.
I’m thinking the context of a DAO controlled application where I do not want to create an account or have API keys or billing outside of the application but rather pay in cycles.


I have some ideas for a set of icrc standards for describing, providing and tracking this kind of payment. Would be happy to put our heads together on it.

I was thinking of just sending cycles for the request that would pay for the computation/offset costs directly, but I guess since things wont be on-chain like GPUs, then it wouldnt work natively, unless they take cycles that would be an equivalent and do it indirectly.

So are you suggesting more of a payment model where the DAO could send ICP or other crypto to pay for certain amount/subscription? Is there anything right now that does something like this (not just AI)?

I was more thinking discovery and registration/proof of cycles. Cycles ledger is going to be a thing. How many cycles do I send to call do_a_thing()? Is it per byte sent? Can I pre send cycles so I’m authorized? If I’m a canister can I include them with the call? Basically a menu and methods of payment.


Sounds cool. Id be down to help


That would be cool.

I think that the main challenge would be : how to charge; given that the token count is variable out of the LLM and the call charges are based on the token count. Of course the call can be interrupted if one reaches a limit.

Additionally the call should be made with a “transaction id” from IC so that the recieving application makes the call once per transaction for the first replica and caches the result for all subsequent replicas.

This has been an ongoing discussion topic in the DeAI working group this year about a mechanism to pay in cycles for query calls above the call instruction limit, for running LLMs in canister.
The forum conversation starts here after our early Jan meeting:

This then to further discussion in the Explore Query Charging thread here:

The Omnia Network team ( @ilbert and @massimoalbarello ) have been working on a project to enable paying (with ICP) for compute & web services external to the IC, based on their WebSocket Gateway work and following on from this proposal thread they started:

I think you are across some or most of this but may be useful for others coming reading this thread.