Motoko Canister Timers are in the code review, as the target for the feature is Q1 2023. Hopefully, it will be finished sooner, but let’s not rush and allow the guys finish their job…
We’ll update this thread once there are news.
7 Likes
mparikh
December 22, 2022, 1:11am
84
Is there a way to call async functions (i.e. raw_rand()) within the function invoked by set_timer/set_timer_interval?
1 Like
domwoe
December 22, 2022, 9:15am
85
2 Likes
It might make sense to make the handler async, WDYT @AdamS ?
1 Like
AdamS
December 23, 2022, 1:50pm
87
It might make things more ergonomic if it was, but since most of the time you won’t need to, I elected to remove the overhead for users who don’t need it. You can always ic_cdk::spawn a future from a sync context.
1 Like
I finally waited for the release of timers feature in the motoko 0.7.5, but after updating the version of moc in vessel I still get an error.
I use this command for updating vessel moc compiler:
vessel verify --version 0.7.5
I also updated motoko-base in package-set.dhall:
{ name = "base"
, repo = "https://github.com/dfinity/motoko-base"
, version = "moc-0.7.5"
, dependencies = [] : List Text
},
Despite these steps, I can’t import functions with
import { cancelTimer; setTimer } = "mo:⛔";
What else should I do to take advantage of the timer functionality?
@rabbithole Haven’t tried it yet but here is an example
dfinity:master ← dfinity:gabor/timers
opened 12:20PM - 02 Nov 22 UTC
This PR adds the necessary primitives for the base library to rely on.
``` Moto… ko
setTimer : (delayNanos : Nat64, recurring : Bool, job : () -> async ()) -> (id : Nat)
cancelTimer : (id : Nat) -> ()
```
It also adds a mechanism `system func timer`, so that the user can choose to provide a custom implementation.
A new compiler flag `-no-timer` is also provided to disable all timer-related functionality (e.g. for critical applications).
-----------
# How it works
By default the timer mechanism is enabled, and the `setTimer`/`cancelTimer` utilities can be used. If the `system func timer` is present, the user is responsible for the timer implementation, i.e. she can supply a very simple or very elaborate mechanism. When `moc` is invoked with `-no-timer` then the whole mechanism is disabled. This is useful for minimising the code footprint.
## Canister-global timer discipline
The IC's underlying timer mechanism is one timer per canister. The canister infrastructure is responsible for providing the abstraction of several (and recurring) timers. Timer expiration is best-effort, i.e. the timer's action will never be executed before the desired expiration time, but there is no guarantee of execution _at_ expiration time. It will usually be later.
## The default mechanism
When the user doesn't choose to implement `system func timer` (and doesn't suppress it by `-no-timer`), a default mechanism gets generated. The default mechanism is basically a call to the builtin `@timer_helper`, which will be provided by the compiler.
In the default mechanism the user calls the `setTimer` call from (Motoko-base). This will register an expiration relative to the current time in form of a timer node that `@timer_helper` will visit. Correspondingly a `cancelTimer` call will expunge a timer node. Canister-global timer expirations will also do this, as well as pruning the node tree (a priority-queue).
### The `Node` structure
`Node`s will track the expiration time, the timer Id, and optionally a duration for recurring timers. For administrative reasons they also keep pointers to the earlier and later trees. Each timer node further has a thunk (a function of type `() -> async ()`) which (when called) returns a _future_ (concurrent action).
The most important invariant for nodes is that the expiration time divides the earlier and later trees temporally:
- every expiration on a `Node` in the earlier-tree is `<=` the expiration of the parent node, and analogously
- every expiration on a `Node` in the later-tree is `>=` the expiration of the parent node.
There is a twist to the expiration of the `Nodes` though, as they are mutable, and are set to 0 when the timer expires or gets cancelled. So the above invariant is strictly only valid for nodes (with their child trees) where no node therein is expunged.
This allows for some neat optimisations. E.g. expiration is caused by time marching from early to later, so if a node's expiration is 0 then we can assume that the earlier-tree is completely expired. _Pruning_ (the cleaning of a timer tree from expired nodes) becomes a later-tree-only operation in this case.
The node expired _implies_ earlier-tree expired is a _corollary of the fundamental invariant_ with expungable nodes.
### Insertion
Timer Ids are given out as `1..` and never reused (but begins after upgrade from 1 again).
The _initial_ insertion of a timer (`setTimer`) respects the invariant that a higher Ids will execute after lower Ids when they have the same expiration time. For _recurrent_ expirations this is also the case if the delays are identical.
### Cancellation
Each timer node carries the Id, and this allows expunging the node when `cancelTimer` is called. To maintain the above corollary, each `cancelTimer` must be followed by a compacting operation (basically pruning, but considering the earlier-trees too). Alternatively a grafting operation can be used, appending the later-tree onto the latest slot of the earlier-tree.
It is important to note that while code is running that expunges nodes as a result of canister global timer expiration, there must not be calls to `cancelTimer` (or similar) since that would likely interfere with the corollary. This is only potentially a problem for self-implemented `system func timer`, as the default mechanism will use `@timer_helper`, which is not doing such things.
### Recurrence
When a timer expires that was set up being recurrent, the timer node needs to be re-added to the timer tree but with an expiration shifted by the timer's delay. This will happen in the loop that expunges timers that are already expired. Thus it may happen that the loop will detect the same timer and expunge it again. For a delay of 0 this might cause an infinite loop, so recurrent timers with 0 delay will be treated as non-recurring. 5b9b62faa implements this by avoiding re-adding.
### Tree walking and reconstruction
Several operations walk (traverse) the timer tree. `setTimer` in particular prunes the expired nodes while adding the new node and similarly the reinsertion of recurrent timers.
Since expiration times are kept in mutable ~~fields~~ array elements, node reinsertion due to recurrence will not cause duplicated expirations when tree nodes get copied. This is due to mutable ~~cell~~ array aliasing in the tree walk code. ~~If we decide to get rid of this feature we have to come up with a different scheme to suppress duplicated firing of timers.~~
### Setting the canister-global timer
There are three triggers to call the system API `global_timer_set`:
- connected to `setTimer` (with the expiration time of the earliest active timer)
- when the timer expiration callback is being invoked
- and there is another active timer
- no active timer (`global_timer_set(0)`)
- `cancelTimer` is called, and there is no active timer remaining.
### Trap avoidance
A timed action trapping must not impede the execution of other actions, and thus we use the type `() -> async ()` for registering such actions with `setTimer`. This ensures that
- each action has its own future (even for each recurrence)
- each future gets run in its own `async` context and as such traps and `throw`s are isolated.
### Reclaiming the continuation slots
_Note_: originally the futures were `awaited`, but this seems redundant and has been removed in 204d87a39. Also, an upgrade test is now run to check that executed timer actions don't cause stuck canisters.
~~Upon timer expiration the callback is responsible for installing the workers into the continuation table (this is the `async` part), but it is also important to `await` those futures, so they don't fill up the continuation table. I expect that this can be done by a trailing `await` of an `async forall <futures> try await <it> catch {}`, since the `canister_global_timer` endpoint is called in an `async` context.~~
### The `@timer_helper` builtin
When no user-written `system func timer` is present, `moc` will insert a call to an internal function `@timer_helper`. This
works by creating a worklist of expired timers relative to the current time, then expunging the corresponding nodes and re-inserting recurring ones with respective delays added.
The worklist is then transformed into an array of futures, each in its trap-proof compartment. Finally all the pending futures are `await`ed.
### The upgrade story
<a name="upgrades"></a>
Easy. The global timer gets jettisoned on upgrade, and the timers need to be set up in the post-upgrade hook. Stable variables can be used to remember the timers if they don't have a rigid structure.
## Opting out
To opt-out of the default mechanism the user just declares `system func timer`. By using the passed in function argument to set the global timer, full control is given to either implement a very basic wakeup scheme or an even more elaborate mechanism than the one presented above.
### The initial expiration
After the canister starts running, `system func timer` gets invoked automatically. This also happens after an upgrade.
### An example of a periodic wakeup
The simplest way to achieve a periodic wakeup of a canister with fixed intervals can be coded as
``` Motoko
import Time "mo:base/Time";
import { fromIntWrap } = "mo:base/Nat64";
actor {
system func timer(set : Nat64 -> ()) : async () {
set(fromIntWrap(Time.now()) + 60_000_000_000); // 60 seconds from now
doSomething();
};
func doSomething() {
// whatever
}
}
```
### An example of a exponential approximation
Imagine you are at a hackathon, and there is a deadline to submit the hash of your solution. In the last hour you want to be reminded 60, 30, 15, 7.5, etc. minutes before the deadline. Following implementation could serve the purpose:
``` Motoko
import { now } = "mo:base/Time";
import { fromIntWrap } = "mo:base/Nat64";
import { print } = "mo:base/Debug";
actor {
let hour = 60 * 60_000_000_000;
let deadline = now() + hour; // nanos
system func timer(set : Nat64 -> ()) : async () {
let time = now();
let toGo = deadline - time;
if (toGo < 0) { set 0; return };
debug { print("Still to go: " # debug_show toGo) };
set(fromIntWrap(time + toGo / 2));
}
}
```
# Open question(s)
- ~~Should the recurring timer be re-inserted with expiration relative to old expiration time or current time (of callback)?~~ — to avoid time skew, using the planned expiration time as the base
- Can we use `async*` to avoid the context switch when calling into `@timer_helper`? — probably yes, but in another PR
------------
TODO:
- [x] add `-no-timer` option to `test/perf` sources — dbfaf545b
- [x] release notes
- [x] check that `setTimer` documented as taking nanoseconds
- [ ] debug deactivated tests (compacting GC in `ic-ref` are a possible indication of GC bug! Or maybe starvation.)
- [x] stabilisation (needed? — CWV: no)
- [ ] tests
- [ ] after upgrade, still on? ("the timers get canceled on upgrade")
- [x] when defining `system func timer`, after upgrade it also gets called — 8df4d6f79d67877b43e386b52cdc9f28b9ecad82
- [x] can upgrade after a timer has run
- [x] test recurrent delay 0 — See 5b9b62faa and c440b3e8d.
- [ ] does `actor class` behaves correctly?
- [x] docs: `doc/md/heartbeats.md`
- [ ] what more is needed?
- [x] run with `ic-ref` too (currently no support, but see https://github.com/dfinity/interface-spec/pull/111)
- [x] `var` aliasing is being used to good effect (can we remove it?) — replaced in #3617
- [ ] check the invariant: no more expiration <=> global timer is 0
- [x] add `test/fail` for `-no-timer`
- [x] cancelling of a timer must be followed by compaction (to not break the corollary)
- [x] duration 0 with recurring timers should be excluded
- [x] traps in jobs, `catch`
- [x] add the low-level API `setGlobalTimer` (but see also: #3614)
- [x] try it on the Playground — _works!_
There are no problems with examples, I looked at different examples, including in tests. The problem is that I can’t compile the code with the latest motoko 0.7.5. What am I doing wrong?
claudio
December 28, 2022, 12:46am
91
I’m not sure what is going on with 0.7.5. Either there is a bug and the feature was not fully enabled, or the changelog is slightly misleading because the release only includes the internal compiler support, with the matching base library exposing the feature to follow in the next release.
@ggreif would know the answer but might be on holiday still.
I’ll try to investigate tomorrow.
If you are using dfx to build the sources, you may still need to tell dfx to actually use moc 0.7.5 using environment variable DFX_MOC_PATH (or similar, I may be misremembering the name)
DFX_MOC_PATH="$(vessel bin)/moc" dfx deploy
1 Like
ggreif
December 28, 2022, 9:38am
92
This should work with the latest released moc. What compile error do you get for this?
Please note also that this is merely accessing the compiler internals, and there is still a pending PR for the base library: feat: user-facing API for timers by ggreif · Pull Request #474 · dfinity/motoko-base · GitHub . Your updated package-set.dhall won’t change anything.
@ggreif I don’t see cancelTimer and setTimer listed here
/*
The primitive definitions.
This module should contain everything that cannot be implemented in plain
Motoko. It is available via `import Prim "mo:⛔"`. Normal user code would
usually not import that module directly, but through `base`, which takes
care of providing a proper module structure, e.g. exposing `Array_tabulate`
through `Array.tabulate`.
Therefore, the prim module does not need to provide a stable interface, as its
only supported consumer is the `base` library, and that is bundled with the
compiler.
Nevertheless, it shoud be _safe_ to import prim, i.e. the definitions here
should not break type safety or other guarantees of the language.
*/
module Types = {
public type Any = prim "Any";
public type None = prim "None";
show original
ggreif
December 28, 2022, 12:10pm
94
Oh, I see. Just checked setTimer, works fine for me, no compiler errors
2 Likes
Caused by: Failed while trying to build all canisters.
The build step failed for canister 'ryjl3-tyaaa-aaaaa-aaaba-cai' (journal) with an embedded error: Failed to build Motoko canister 'journal'.: Failed to compile Motoko.: Failed to run 'moc'.: The command '"/Users/khalik/.cache/dfinity/versions/0.12.2-beta.0/moc" "/Users/khalik/Documents/projects/ng-storage/src/backend/journal/main.mo" "-o" "/Users/khalik/Documents/projects/ng-storage/.dfx/local/canisters/journal/journal.wasm" "-c" "--debug" "--idl" "--stable-types" "--public-metadata" "candid:service" "--actor-idl" "/Users/khalik/Documents/projects/ng-storage/.dfx/local/canisters/idl/" "--actor-alias" "journal" "ryjl3-tyaaa-aaaaa-aaaba-cai" "--package" "array" ".vessel/array/v0.2.1/src" "--package" "base" ".vessel/base/moc-0.7.5/src" "--package" "base-0.7.3" ".vessel/base-0.7.3/aafcdee0c8328087aeed506e64aa2ff4ed329b47/src" "--package" "bimap" ".vessel/bimap/main/src" "--package" "encoding" ".vessel/encoding/v0.4.1/src" "--package" "io" ".vessel/io/v0.3.1/src" "--package" "uuid" ".vessel/uuid/main/src"' failed with exit status 'exit status: 1'.
...
type error [M0119], object field cancelTimer is not contained in expected type
module {}
@claudio are right, in my error you can see that the compilation uses moc from .cache/dfinity/versions/0.12.2-beta.0/moc, not the one that comes with vesselDFX_MOC_PATH="$(vessel bin)/moc" works! But I don’t know how to tell the VS Code linter also the compiler version.@ggreif I should wait for the public api with PR #474
I saw the description of the compiler release and decided that the timers would be hidden in primitives without a public API, but I’m glad I was wrong.
P.S. I’m embarrassed that I drew the attention of the team during the holidays, thank you!
ggreif
December 28, 2022, 2:14pm
97
No worries! Let the questions come!
1 Like
Iceypee
December 28, 2022, 11:29pm
98
Does awaiting pause the timer or mess with the intervals like with the heartbeat?
rvanasa
December 29, 2022, 1:36am
99
This should work out-of-the-box in the latest version of the VS Code extension (updated today).
Happy holidays!
5 Likes
ggreif
December 29, 2022, 10:51am
100
It shouldn’t. And I doubt that heartbeat does get paused when there are futures being awaited either. Did you read that somewhere or is it empirical (by you)?
ggreif
December 29, 2022, 12:09pm
101
feat: user-facing API for timers by ggreif · Pull Request #474 · dfinity/motoko-base · GitHub is merged now. Feel free to drop Timer.mo into your base folder and test. The upcoming release (no ETA yet) will contain it. Happy new year and have fun!
1 Like
You are magicians, this is great, thank you! Happy New Year!
1 Like