Nns login issue- shown internal server error

[daijianlin]

What is wrong with nns, can the technical guy fix this?

[Severin]

Thanks for the report, the team is taking a look

[Severin]

Internet Identity offline

[Juna]

I have the same trouble.

[Severin]

Subnet uzr34 outage during replica rollout, affecting Internet Identity

Dear IC community: During this week’s rollout of the latest replica version, we experienced a problem with subnet uzr34. At this point, II and the exchange rate canister (XRC) are affected. We are working to resolve the issue shortly — please be patient and keep checking this page for more information as the situation develops. Thanks.

[willguest]

Some cursory investigations:

On May 13th, 2024, at 11:37, the Internet Identity went down, along with its entire subnet:

Subnet

Id: uzr34-akd3s-xrdag-3ql62-ocgoh-ld2ao-tamcv-54e7j-krwgb-2gm4z-oqe

The subnet contains 5 canisters. At the time of the outage, they collectively hold 10.8 GiB of state information and the block height is 44,958,403.

ID & Name	ID & Name	Controllers	Subnet ID	Module Hash
NNS Internet Identity	rdmx6-jaaaa-aaaaa-aaadq-cai	NNS Root	uzr34-akd3s-xrdag-3ql62-ocgoh-ld2ao-tamcv-54e7j-krwgb-2gm4z-oqe	20103f72d31f20fb08e1fa3c50778c25c7a2dc115c92981f4bb54e61cd9f4538
“Canister”	uc7f6-kaaaa-aaaaq-qaaaa-cai	NNS Internet Identity	uzr34-akd3s-xrdag-3ql62-ocgoh-ld2ao-tamcv-54e7j-krwgb-2gm4z-oqe	3c789fe8a257917c2a7ffee5cc9515fa108a631a394a525f2057811f48fc98f5
Exchange Rate	uf6dk-hyaaa-aaaaq-qaaaq-cai	NNS Root	uzr34-akd3s-xrdag-3ql62-ocgoh-ld2ao-tamcv-54e7j-krwgb-2gm4z-oqe	dcc42fd617ce8382a28941678b6299cc4c23f1bfc29bfb1eec517e1e84b9128a
Cycles Ledger Index	ul4oc-4iaaa-aaaaq-qaabq-cai	b6f5g-tu…by-fae	uzr34-akd3s-xrdag-3ql62-ocgoh-ld2ao-tamcv-54e7j-krwgb-2gm4z-oqe	b804edfe58b37032a009ec21aeeb8527087eec8c25c26c12636a41139d517054
Cycles Ledger	um5iw-rqaaa-aaaaq-qaaba-cai	b6f5g-tu…by-fae	uzr34-akd3s-xrdag-3ql62-ocgoh-ld2ao-tamcv-54e7j-krwgb-2gm4z-oqe	d7ef61a1aaa13177a0bb3b17fe855c30f00171de0dc946cc463c9f1c4d835a89

Two of these canisters provide information about their upgrades (on the dashboard):

Exchange Rate Canister Upgrades (showing all 6 items)

Proposal ID	Module Hash	Date Executed
129034	dcc42fd617ce8382a28941678b6299cc4c23f1bfc29bfb1eec517e1e84b9128a	2024-04-08, 13:02:40 UTC
127671	4bfbaa72726f0395383bf5241e77516c3e1b727da04484e728d8e496e24be569	2024-02-05, 14:02:50 UTC
127097	785ad7825857116528fe17d2b286b488bff6a533999d8e46d505f85f3c46bc3f	2024-01-22, 16:30:04 UTC
126300	fe91ed906ba7f7b3ac506680d68ab50fc815dba5718154539fe9df032e222553	2023-12-04, 14:49:57 UTC
126130	45e306e083590365adef0feca43b1f1bc13e8ab8da7b1547800317a6f0e441ef	2023-11-28, 18:59:44 UTC
123717	513bd326c827881278810c09327573a95aac84976a9a97c769847179223ffc27	2023-07-24, 14:24:32 UTC

NNS Internet Identity Canister Upgrades (showing most recent 6 of 114 items)

Proposal ID	Module Hash	Date Executed
129486	20103f72d31f20fb08e1fa3c50778c25c7a2dc115c92981f4bb54e61cd9f4538	2024-04-29, 12:23:14 UTC
129098	1dcbf01453de8e7690bf9de2029f3e1993dbfd11e7ca49a9f87606bbef2da135	2024-04-16, 09:38:26 UTC
129087	aea0f5a35c7950da7c1a30f4665cb9789350a7ebf5792b6a7c423d2f3f5bc824	2024-04-15, 08:07:59 UTC
129083	93f7d87eee96cdf01e32e6144020ef87ecb44fc6d424632811fba8b87c28feea	2024-04-15, 07:44:16 UTC
129031	a191b5cd0423223f08218b46c40b88850cc087ac985a88b7c5232ddcaf8bbfd3	2024-04-08, 08:18:23 UTC
128769	c3c6c9d6687dc6054b7dfa4ce2a91127fd4d60b14ee305a121a83d340da30c48	2024-03-25, 10:33:28 UTC
…	…	…

[ajismyid]

How does ICP update a canister that’s causing an error with internet identity through NNS governance voting, if the error prevents voters from accessing their NNS accounts to cast their votes?

[willguest]

Neither of the two shown above (NNS identity and XRC) show any updates made during May, so I am assuming the error relates to one of the others.

So the options are Cycles Ledger, Cycles Index, or “unnamed”, which is my best guess for the source, either because it isn’t meant to be unnamed, or because it is.

[Severin]

@ajismyid

How does ICP update a canister that’s causing an error with internet identity through NNS governance voting, if the error prevents voters from accessing their NNS accounts to cast their votes?

Most of the big, original neurons are actually mainly controlled by CLI tools like quill. And even if your neuron is held by your NNS dapp principal, you can still add a quill-controlled identity as a hotkey so you can vote from the console as well. The third avenue is by following a CLI-controlled neuron

@willguest

Neither of the two shown above (NNS identity and XRC) show any updates made during May, so I am assuming the error relates to one of the others.

It was actually the subnet, not a specific canister. There’s also info over here: Proposal to elect new release rc--2024-05-09_23-02 - #6 by Lorimer

[Agnostic]

For what it’s worth, I was able to login into the NNS around the time this error was reported. So whatever this problem was, it did not affect the entire ICP system.

[Lorimer]

Further information now available as a post-mortem report - Post Mortem: Subnet uzr34 failed upgrade process on May 13, 2024

Thanks Dfinity

[Lorimer]

I get an Internal Server Error when visiting any NNS page at the moment

[Jdcv97]

I’m worried wtf is happening, I can’t access my funds on the nns where i have everything.

Transactions not working according to the dashboard

[jokerswild]

There’s nothing to worry about. I’m sure Dfinity is on top of the issue whatever it may be. Just check back later or tomorrow. They’ll fix it.

[peterparker]

Cross-posting here as it seems related:

NNS Dapp not loading - Internal server error

I’m totally out of the loop, but I noticed some internal threads. Therefore, just in case it’s unclear and as I do not notice an other posts, just wanted to mention that the team is aware of it and is looking into it. Status to track https://status.internetcomputer.org/