Canister Logging Support [Community Consideration]

Developers
21

@claudio
I’m running on-chain testing for the Motoko Bootcamp (https://www.motokobootcamp.com/) - I essentially perform inter-canister calls on submitted projects by students. Occasionally, when students modify the interface or send unexpected types, my canister trap.Having access to this call_on_cleanup would have been great to handle those cases.

I’ve found some workaround (checking the Candid interfaces in advance through the frontend and WASM metadata, as well as isolating tests in a separate canister and using try/catch) so it’s not an important item for me anymore.

1 Like
22

Hey it seems dfx 0.18.0 has some sort of log reading functionality? Can we get an update?

1 Like
23

It’s unfortunately not working yet because there was a mixup in the type of call used when dfx is trying to retrieve logs. That’s why we didn’t advertise it yet. We’ll get it fixed in the next dfx version and there’s also going to be a presentation/demo in one of the following weeks global R&D meetings.

Stay tuned, we’ll give you updates and make more noise once it’s available in dfx for beta testing! We’re definitely going to need you guys to test it out and give us feeback!

3 Likes
24

Good news, everyone!
The first canister logging implementation is now available in dfx 0.19.0 release.
Please try it out and let us know about your experience!

5 Likes
25

Hey,

Thanks for all the hard work.

Is the logging enabled on all subnets? Because I get Failed to query call function 'fetch_canister_logs' regarding canister 'xcgns-iqaaa-aaaae-aai7a-cai'. Query call (without wallet) failed. The replica returned a rejection error: reject code CanisterError, reject message IC0504: fetch_canister_logs API is not enabled on this subnet, error code Some("IC0504")

3 Likes
26

Hi Gabriel.
Canister logging is not enabled on mainnet subnets yet.
We wanted to make it available locally in DFX first.
There is no firm ETA on rolling out this feature on mainnet subnets. We would like to see the first feedback on local DFX to see if canister developers are happy with the current look and feel before rolling out it to mainnet.

4 Likes
27

Hello @maksym . Is there any update on this? (Just a periodic ping; I am sure there are many people waiting for this feature. :slight_smile: )

6 Likes
28

I literally searched this post to ask the same question as @bitdivine. Wen mainnet?

3 Likes
29

Hello everyone!

We’re in the final stages of implementing canister logging.
We still need to add some metrics and feature flag handling to ensure a smooth rollout on the mainnet.
I expect it to be available in the next few weeks.

8 Likes
30

Hey @maksym could you confirm if this will be released with the next DFX version?

1 Like
31

hi @Gabriel , if you mean enabling the feature on mainnet then it should happen with the next replica release (see releases page). Here is the commit to keep an eye for.

4 Likes
32

I’m guessing that’s the one:

2 Likes
33

I’m guessing that’s the one:

You are right, that’s the release that enables canister logging on mainnet.

3 Likes
34

Is there anything that needs to be configured to activate the canister logging in a local replica?

I upgraded my Docker image with the proposed release (thread, PR) but, not logs are collected. Neither ic_cdk::print nor trap are delivered when querying the IC mgmt end point fetch_canister_logs, I just get an empty array.

pub fn list_docs(collection: CollectionKey, filter: ListParams) -> ListResults<Doc> {
    let caller = caller();

    print("test"); // <------ This is not provided

    let result = list_docs_store(caller, collection, &filter);

    match result {
        Ok(value) => value,
        Err(error) => trap(&error), // <------ This is not provider neither
    }
}
35

If you start local replica the same way it’s done in DFX (using ic-starter) then canister logging should be enabled since a month or two. Here’s ic-starter related code.

I have recently added rust & motoko examples that are using local DFX replica (without the feature enabled on mainnet), here is a related test.

There should be no extra configurations to make it work.
Things to keep an eye on though:

  • it’ll reject a call to fetch_canister_logs if mainnet subnet does not have the feature enabled with API is not enabled on this subnet
  • it’ll reject a call if the caller does not fit log_visibility setting (try calling it as controller first) with Caller X is not allowed to query ic00 method...
  • it’ll reject an update call to fetch_canister_logs, it must be a query call (related code)
  • make sure to use the correct canister_id when fetching canister logs, otherwise it can return a valid response with an empty array in canister_log_records field.
36

Thank you for your answer. Have you test with the foundation agent-js library as well?

I might be missing something but, as far as I can tell, it doesn’t work.

Here’s a sample with dfx v0.20.0 and agent-js v1.3.0 which also returns an empty array.

Capture d’écran 2024-05-25 à 17.17.09
Capture d’écran 2024-05-25 à 17.17.093072×2062 531 KB

37 
type log_visibility = variant {
    controllers;
    public;
};

@maksym @dfx-json

Did the team think about extending this variant beyond just controllers and public (i.e. to allow a specific non-controller to pull logs).

Something like this:

type log_visibility = variant {
    controllers;
    allowed_viewers : opt vec principal;
    public;
};

It would be really helpful to be able to allowlist a specific canister/dfx generated principal to pull logs within an automated job without needing to explicitly assign it as a controller of the canister.

Especially for DAOs or other high value canisters, it’s harder to use this if only the root/governance canister are able to read the logs, or for 3rd party monitoring services to integrate without using a blackhole monitoring approach.

38

Yes, we considered adding allowed viewers, but decided not to add it on the first iteration.
It can be added later in the process.

1 Like
39

The issue I mentioned above has been resolved and requires no further action.

The root cause of the problem was a misunderstanding. Logs can only be collected if the functions that generated them were called in replicated mode – for example, when an update call is executed.

I did not understand this and inadvertently used query calls in my tests, but it wasn’t communicated clearly neither, so let’s say it was a shared effort that led to the issue :wink:.

Documentation, particularly the sample code, has now been updated to make this more obvious.

Thanks to @maksym and @kpeacock for the support.

3 Likes
40

Logs are live in Juno! :partying_face:

frame
frame1920×1344 127 KB

9 Likes