@ulan Thank you for clarifying the instruction limits for composite queries on the Internet Computer. I am currently developing an open-source repository for AI inference, aiming to facilitate the deployment of PyTorch/Tensorflow AI models on the IC. The primary challenge is the instruction limit for single queries: models that are within the RAM capacity of a canister often exceed the available instructions.
My initial strategy involved splitting the AI model across multiple canisters and linking them through composite queries. However, I’ve encountered the instruction limit even when testing composite queries with two individually functional queries. Since AI inference and learning processes like backpropagation don’t require updates, running them as updates seems inefficient in terms of cost, energy, and time.
Interestingly, I recently proposed the idea of allowing canisters to pay for increased query execution limits in the DeAI Technical Working Group. This approach could potentially resolve the instruction limit issue for intensive tasks like AI inference.
Moving forward, I plan to evaluate the trade-offs in terms of model speed, capability, and cost when setting the master script to run as an update as compared to a query. I have some follow-up questions: Are query calls made within an update call counted towards the instruction limit of the update? Would calling an update from an update count against the instruction limits of the outer update? Your insights on this would be greatly appreciated.