In one of our projects, we have a task processor that operates every 5 seconds to handle tasks from a pending list. To simplify, the process can be outlined as follows:
Retrieve the first task from the pending list.
Execute the task.
Remove the processed task from the list.
Commit the state changes.
Repeat from step 1.
Step 4, which involves committing state changes, is crucial to prevent the reprocessing of all tasks in the event of a panic during task execution. Presently, we accomplish this by initiating a dummy intercanister call to a do_nothing() query endpoint.
Is there an alternative method to ensure the state is committed more efficiently?
What you’re doing is essentially the trick to have the commit happen at the point you’re interested. It sounds like you could also use timers to sort of get that out of the box. Timers in the ic-cdk library internally schedule a self-call to perform the actual work – this will achieve what you want which is to commit after step 4 and before scheduling the next timer to go through the list again. The idea would be that you schedule a periodic timer that runs every 5 secs and executes 1-4 (1-3 explicitly, 4 implicitly once the message execution ends) every time it runs.
The task processor is already triggered every 5 seconds by a timer.
If I understood correctly, you propose to handle only one task on each execution, however, this is not feasible as there could be hundreds of pending tasks
Ok, I did not realize you wanted to handle multiple tasks in a single timer execution. Then doing a dummy inter-canister call is your main option at the moment. You could also consider having an “outer” timer that triggers every 5 seconds and then for each task trigger a 0-delay one-off timer to process it (so you still get the commit point you want). I’m not sure how well that would scale though, so you’d need to test it out.
Taking a step back, rephrasing your initial question to “can I arbitrarily commit state changes in the middle of message execution” is not something that we’ve been asked to provide before so I haven’t really thought through it. It could be possible in theory, although I imagine it could involve quite some effort to do properly since it would change some basic assumptions around the execution model of the IC (e.g. each message execution is a single atomic unit but with multiple commit points that would change).