Pjljw is completely down. 0 blocks/s

Just wanted to let everyone know that subnet pjljw is completely down. Internet Computer Network Status

Finalization rate of 0 blocks/s. :slight_smile:

Can we just unplug it and plug it back in to make it work again? haha

6 Likes

Fwiw, I don’t have any details other than I know some engineers are looking at it and they are too busy to give me details.

1 Like

Apparently another one is also down mpubz with a finalization rate of 0

1 Like

From the status page:

Partial outage: Subnets mpubz and brlsh are currently down

Update - We are continuing to work on a fix for this issue.
Oct 15, 05:38 UTC

Looks query works now, but update still not working.

Even though the incident is still displayed on the status page, it seems alright now to me. I was able to deploy my canisters and perform query and update through my web app.

Just out of curiosity I used the IC dashboard to check the distribution of the canisters on the different subnets and here are some interesting facts:

  • Currently total number of subnets is 22.
  • 23% of subnets are serving 88% of the total canisters.
  • There are 12 subnets that have less than 33 canisters.
  • Most crowded subnet award goes to pjljw with more than 4k canisters and come in second place 5kdm2 with less than 1500 canisters.
  • Most of subnets have no more than 13 Nodes except System subnet (34).

While number of canisters doesn’t give the full picture of the load on the subnets it certainly can give some indication.

4 Likes

Hey folks, wanted to let you know the issue is resolved and we are working on an incident report: Subnets `mpubz`, `brlsh`, and `pjljw` Incident Retrospective - Friday October 15, 2021

2 Likes

incident report updated: Subnets `mpubz`, `brlsh`, and `pjljw` Incident Retrospective - Friday October 15, 2021

Thank you everyone in this thread for being so vigilant!

Hope the incident report above helps clarify what happened.

1 Like