IC Cron - let's schedule some tasks bois

Ever wanted to make your canister work automagically, without triggering it from the outside?
Here you go.

This is a rust library which is intended to be used by canister developers to make their canisters proactive for a reasonable price.

Check the readme and let me know what you think.

Relates to Cron jobs on ICP

12 Likes

Fun fact: The NNS itself was using the “hack” of self-calls at some point in the past. Yes, it seems to work, but I consider it hack because of the reasons you mention in the readme:

  • You are consuming lots of cycles.
  • Upgrades are prevented.
  • If you the self-call ever fails (low on cycles, other trap), suddenly the cron stops.
  • I expect a future version of the IC will handle update calls to canister on the same subnet (so yourself included) immediately. Once that long-awaited improvement comes, your code will suddenly consume lots and lots of cycles, up to the per-round cycle limit. Or, maybe even worse, your library will be popular and the IC will have to make pragmatic compromises such as detecting a cron-like self-calling and artificially deferring it to not burn your cycles fast.

So please keep that in mind when using this.

I also know that there are plans to expose the “heartbeat” mechanism used by the NNS on all subnets, which would allow a much safer and reliable implementation of cron.

That said, the great think about a library like yours is that it hides the implementation from the user. So people who use your library to schedule events can, once the IC provides that feature and once your library use that feature instead of the self-call, upgrade your library and redeploy and benefit from the new system.

5 Likes

Oh, and if I read

        #[allow(unused_must_use)]
        fn _call_cron_pulse() {
            if get_cron_state().is_running {
                ic_cdk::block_on(async {
                    ic_cdk::call::<(), ()>(ic_cdk::id(), "_cron_pulse", ())
                        .await
                        .unwrap();
                });
            };
        }

        #[ic_cdk_macros::update]
        fn _cron_pulse() {
            for task in get_cron_state().iterate(ic_cdk::api::time()) {
                _cron_task_handler(task);
            }

            _call_cron_pulse();
        }

correctly then you ran into the same mistake that others ran before: Your pulse function calls itself recursively. This means you are accumulating call contexts without bound, which will cause issues eventually.

The better way to do this (in pseudo-code) is roughly

            ic_cdk::block_on(async {
                while get_cron_state().is_running {
                    ic_cdk::call::<(), ()>(ic_cdk::id(), "_cron_ping", ())
                        .await
                        .unwrap();
                    for task in get_cron_state().iterate(ic_cdk::api::time()) {
                        _cron_task_handler(task);
                   }
               }}

In other words: don’t use recursion, use loops, as otherwise the stack grows.

3 Likes

Thanks for the response.

There are three main goals of this library:

  1. As you said, it abstracts away the implementation, so devs could just think of it as of “task scheduler” and focus on their business logic entirely.
  2. Developers can use this new feature early, not relying on any estimates from the IC-team. Yes, it would be great, if one day we’ll have out-of-the box support for such functionality, but right now there is none.
  3. I need this functionality for some of my other projects.

About the code, you’ve mentioned. Hm… Thanks a lot!
But now I’m confused. I asked this question specifically UDP-like flow for inter-canister calls and the answer seemed to me like “use block_on when you want to send a request, but don’t care for a response - the system will handle it automatically”.
UPD: since I don’t care about the response (I don’t need the continuation of this promise) and I tell the system exactly that, there should be no stored context and no stack overflow.

It would be very cool to have some in-depth docs on this APIs.

1 Like

Maybe I was wrong, I don’t know the Rust CDK very well. It all depends on what “the system will handle it automatically” means. But I thin it only means “set things up to ignore there response message (reply or reject)”, then we have the problem that I described above, and there isn’t really anything the CDK can do here.

We once had the plan for “proper” fire and forget messages on the platform level, but it wouldn’t have been trivial and less often asked for than expectd, so we never added them.

I agree that in-depths docs are missing. But at least we have the forum :slight_smile: Maybe @roman-kashitsyn can double-check my analysis.

1 Like

I believe @nomeata’s analysis is correct, this pattern of self-calls opens new call contexts that will never be closed on each iteration. It’s a very common mistake with self-calls, I made it myself a couple of times :slight_smile:

Self-calls are indeed very fragile and I’d not recommend using them in production.

My colleagues from the Execution team informed me that canister heartbeat feature should now be available on all subnets. I’m not sure if the replica shipped with the latest DFX also supports this feature. You can add something like this to your canister to check:

#[export_name = "canister_heartbeat"]
fn heartbeat() {
  ic_cdk::api::print("Hello from the heartbeat handler!");
}

This is how Governance uses it: ic/canister.rs at e362530172c44679313b1b7fca1e90d8967545d8 · dfinity/ic · GitHub

5 Likes

Oh, that’s very cool!
I will update the library ASAP.

Thanks a lot!

This is great! Is there any documentation on this?

I tried this locally with dfx 0.8.0 - it doesn’t work yet.

Does Motoko need func heartbeat() or func canister_heartbeat? Does it need to be public?

Eventually, surely The Internet Computer Interface Specification :: Internet Computer will show it. I guess Roman leaked it before it is officially there :slight_smile:

This would likely become a system func heartbeat(), like preupgrade and postupgrade. But it needs compiler support (just created an issue).

3 Likes

Just do this in rust

#[export_name = "canister_heartbeat"]
fn canister_heartbeat() {
    ic_cdk::print("we are running cron");
}
1 Like

Hey there, @botch!
Thanks for your response c:

The code snipped you’ve provided is only 50% of what a cron implementation needs to do. To be useful for developers it needs a couple more things:

  1. Scheduling - you need to be able to do something not each consensus tick , but, for example, each month or each minute.
  2. Tasks - you need a way to define subprograms that will be run by the scheduler, once their time has come.

This is exactly what you can use this library for.

Have a great day!

2 Likes

I talked to the execution team earlier and got some more details on heartbeat:

  • Called at most once per consensus round
  • Cannot be customized
  • Is called first before other functions
  • No special restrictions - can do awaits, standard cycles limit
6 Likes

What does Cannot be customized mean?

Not possible to set an interval, I believe.

1 Like

This is oddly vague. It either does run or doesn’t run each round. If it runs each round the check for if you need it to run or not could be a cycle sink.

Timer hashtables would have to be used similar to multiplayer games would use. Each tick you see what subscriber needs to be triggered and trigger it.

2 Likes