Is it possible to check when a subnet went down for maintenance in the past?

Developers
,
1

I’m looking for timestamps of when subnets went down and back up again for maintenance. I have a normal server that calls some canisters and every now and again we’re getting a CallFailed response. I’m curious if the times of the failed calls match the times for when the subnet went down for maintenance but i’m not sure where to find such information.
The subnet in question is : https://dashboard.internetcomputer.org/subnet/opn46-zyspe-hhmyp-4zu6u-7sbrh-dok77-m7dch-im62f-vyimr-a3n2c-4ae

err: CallFailed { 
         /// ... other details
        details: TransportError(reqwest::Error { kind: Decode, source: reqwest::Error 
                { kind: Body, source: hyper::Error(Body, Error { kind: Reset(StreamId(601), INTERNAL_ERROR, Remote) }) } }) } }
2

Hey @frederico02

There are not accurate timestamps, but you can look at past proposals on the IC dashboard: you want to look at proposals of type IC-OS Version Deployment.

For example, subnet lspz2-jx4pu-k3e7p-znm7j-q4yum-ork6e-6w4q6-pijwq-znehu-4jabe-kqe was upgraded today at 1pm UTC (proposal 136296). This means the proposal was executed around that time, the actual upgrade usually happens around 15mins later.

1 Like
3

Thanks for the reply @rbirkner , very helpful. Do you happen to know if there is a way to know when the subnet went back online?

4

I can’t think of something right now. Usually an upgrade takes 2-3 minutes. For subnets with a larger state, it can take a bit longer, but never more than 5 minutes.

1 Like
5

Thanks @rbirkner . One final question about when the actual upgrade happens. Is there a canister or dfinity server that has a cron job or scheduled task every 15 minutes to check for upgrades to act on? I ask because our failed call happened at 13:20:37 UTC. If I’m correct about the 15 minute cron job then its also possible that the upgrade actually happened at around 13:15:00 because it w as not picked up at the cron timer of 13:00.

6

The upgrades happen autonomously: the nodes itself run a process called the orchestrator. The orchestrator constantly checks in the registry which version it is supposed to run. If it sees that it should run a different version that it is currently on, it will trigger an upgrade. The time from triggering the upgrade to actually upgrading is about the 15 minuted I mentioned before. This time is mostly needed to fetch the upgrade and to synchronize across all nodes within a subnet as they should upgrade at the same time. You can read more about that in our learnhub under the topic “Chain Evolution”.

1 Like