IC Cron - let's schedule some tasks bois

Developers
1

Ever wanted to make your canister work automagically, without triggering it from the outside?
Here you go.

This is a rust library which is intended to be used by canister developers to make their canisters proactive for a reasonable price.

Check the readme and let me know what you think.

Relates to Cron jobs on ICP

14 Likes
2

Fun fact: The NNS itself was using the “hack” of self-calls at some point in the past. Yes, it seems to work, but I consider it hack because of the reasons you mention in the readme:

  • You are consuming lots of cycles.
  • Upgrades are prevented.
  • If you the self-call ever fails (low on cycles, other trap), suddenly the cron stops.
  • I expect a future version of the IC will handle update calls to canister on the same subnet (so yourself included) immediately. Once that long-awaited improvement comes, your code will suddenly consume lots and lots of cycles, up to the per-round cycle limit. Or, maybe even worse, your library will be popular and the IC will have to make pragmatic compromises such as detecting a cron-like self-calling and artificially deferring it to not burn your cycles fast.

So please keep that in mind when using this.

I also know that there are plans to expose the “heartbeat” mechanism used by the NNS on all subnets, which would allow a much safer and reliable implementation of cron.

That said, the great think about a library like yours is that it hides the implementation from the user. So people who use your library to schedule events can, once the IC provides that feature and once your library use that feature instead of the self-call, upgrade your library and redeploy and benefit from the new system.

6 Likes
3

Oh, and if I read

        #[allow(unused_must_use)]
        fn _call_cron_pulse() {
            if get_cron_state().is_running {
                ic_cdk::block_on(async {
                    ic_cdk::call::<(), ()>(ic_cdk::id(), "_cron_pulse", ())
                        .await
                        .unwrap();
                });
            };
        }

        #[ic_cdk_macros::update]
        fn _cron_pulse() {
            for task in get_cron_state().iterate(ic_cdk::api::time()) {
                _cron_task_handler(task);
            }

            _call_cron_pulse();
        }

correctly then you ran into the same mistake that others ran before: Your pulse function calls itself recursively. This means you are accumulating call contexts without bound, which will cause issues eventually.

The better way to do this (in pseudo-code) is roughly

            ic_cdk::block_on(async {
                while get_cron_state().is_running {
                    ic_cdk::call::<(), ()>(ic_cdk::id(), "_cron_ping", ())
                        .await
                        .unwrap();
                    for task in get_cron_state().iterate(ic_cdk::api::time()) {
                        _cron_task_handler(task);
                   }
               }}

In other words: don’t use recursion, use loops, as otherwise the stack grows.

3 Likes
4

Thanks for the response.

There are three main goals of this library:

  1. As you said, it abstracts away the implementation, so devs could just think of it as of “task scheduler” and focus on their business logic entirely.
  2. Developers can use this new feature early, not relying on any estimates from the IC-team. Yes, it would be great, if one day we’ll have out-of-the box support for such functionality, but right now there is none.
  3. I need this functionality for some of my other projects.

About the code, you’ve mentioned. Hm… Thanks a lot!
But now I’m confused. I asked this question specifically UDP-like flow for inter-canister calls and the answer seemed to me like “use block_on when you want to send a request, but don’t care for a response - the system will handle it automatically”.
UPD: since I don’t care about the response (I don’t need the continuation of this promise) and I tell the system exactly that, there should be no stored context and no stack overflow.

It would be very cool to have some in-depth docs on this APIs.

1 Like
5

Maybe I was wrong, I don’t know the Rust CDK very well. It all depends on what “the system will handle it automatically” means. But I thin it only means “set things up to ignore there response message (reply or reject)”, then we have the problem that I described above, and there isn’t really anything the CDK can do here.

We once had the plan for “proper” fire and forget messages on the platform level, but it wouldn’t have been trivial and less often asked for than expectd, so we never added them.

I agree that in-depths docs are missing. But at least we have the forum :slight_smile: Maybe @roman-kashitsyn can double-check my analysis.

1 Like
6

I believe @nomeata’s analysis is correct, this pattern of self-calls opens new call contexts that will never be closed on each iteration. It’s a very common mistake with self-calls, I made it myself a couple of times :slight_smile:

Self-calls are indeed very fragile and I’d not recommend using them in production.

My colleagues from the Execution team informed me that canister heartbeat feature should now be available on all subnets. I’m not sure if the replica shipped with the latest DFX also supports this feature. You can add something like this to your canister to check:

#[export_name = "canister_heartbeat"]
fn heartbeat() {
  ic_cdk::api::print("Hello from the heartbeat handler!");
}

This is how Governance uses it: ic/canister.rs at e362530172c44679313b1b7fca1e90d8967545d8 · dfinity/ic · GitHub

5 Likes
7

Oh, that’s very cool!
I will update the library ASAP.

Thanks a lot!

8

This is great! Is there any documentation on this?

9

I tried this locally with dfx 0.8.0 - it doesn’t work yet.

10

Does Motoko need func heartbeat() or func canister_heartbeat? Does it need to be public?

11

Eventually, surely https://sdk.dfinity.org/docs/interface-spec/index.html will show it. I guess Roman leaked it before it is officially there :slight_smile:

This would likely become a system func heartbeat(), like preupgrade and postupgrade. But it needs compiler support (just created an issue).

3 Likes
12

Just do this in rust

#[export_name = "canister_heartbeat"]
fn canister_heartbeat() {
    ic_cdk::print("we are running cron");
}
1 Like
13

Hey there, @botch!
Thanks for your response c:

The code snipped you’ve provided is only 50% of what a cron implementation needs to do. To be useful for developers it needs a couple more things:

  1. Scheduling - you need to be able to do something not each consensus tick , but, for example, each month or each minute.
  2. Tasks - you need a way to define subprograms that will be run by the scheduler, once their time has come.

This is exactly what you can use this library for.

Have a great day!

3 Likes
14

I talked to the execution team earlier and got some more details on heartbeat:

  • Called at most once per consensus round
  • Cannot be customized
  • Is called first before other functions
  • No special restrictions - can do awaits, standard cycles limit
6 Likes
15

What does Cannot be customized mean?

16

Not possible to set an interval, I believe.

1 Like
17

This is oddly vague. It either does run or doesn’t run each round. If it runs each round the check for if you need it to run or not could be a cycle sink.

18

Timer hashtables would have to be used similar to multiplayer games would use. Each tick you see what subscriber needs to be triggered and trigger it.

2 Likes
19

IC cron is now promoted to 0.4.0 adding heartbeat support.
Please, consider helping with testing and provide feedback.

9 Likes
20

Thanks! Can you please add license to the library?