I sincerely hope that Dfinity’s communication improves. Currently, the status page only reports frontend issues, even though there is at least a partial outage of the ledger.
Dont cry, this shows that another layer is needed, if 1 has error, than it triggers safe boot like canister with basic token send/resive and voteing.
embarassing what they wrote on their status page, trully
the issue goes way beyond a mere ‘front-end’
I could not:
- enter NNS
- plug wallet shows 0 balance(icpswap tied to it as well)
- dashboard/icrc1_balance_of was showing correct balance
- DeFi unusable
- cannot transfer any ICP even manually(by directly calling icrc2 icp ledger canister methods) - certificate error(date/time)
sad, currently like 5h+ of outage
While it’s not part of my tasks, I collaborate with the foundation and started answering threads on the forum and multiple tweets around 6 pm, more than four hours ago, trying to provide some more context while my colleagues were focused on resolving the issue.
Additionally, the pager on the status page has been reviewed since your first screenshot and now has a better description.
It’s not perfect, and I also welcome improvements in communication, especially on X/Twitter, but claiming it is “truly embarrassing” seems a bit over the top, if I may share my opinion.
You could be right. Would dropping truly help? Joking aside, after the first four hours, the status page still only showed frontend issues despite a much bigger problem.
I hope something will be done in the future, especially with the NNS frontend, which should have an alternative like NNS2 on another subnet. Telling people not to store their ICP on exchanges and to move it to NNS exposes them to the risk of not having access to their assets.
I know you guys are doing your best, but I feel such outages should be called out and never normalized. I also believe that in the future, there will be many more different problems to overcome, which will make the network more resilient.
@peterparker thank you for your help on the forum.
@Ajki Speaking as a full-time employee - you’re correct for calling this out. Our comms here were inadequate and we need to take accountability for this. We should have better processes here, and I’ll make sure the comms and status page are part of our postmortem
Any outage is not good, but at least ICP is not completely down. Some of its dapps still work like OpenChat, Querio. I haven’t tried any of the Defi dapps.
But I’m just thinking that this would have been 10x worse if ICP was like the majority of the other blockchains that could only run and store transactions (like Bitcoin, Solana, Eth). The entire blockchain would be down or useless for those types of chains.
So while this outage is bad, lets also keep a reasonable perspective.
It’s extremely bad.
My users lost thousands.
I would like to see a full post Mortum
There are huge swaths of users whom have tokens stuck in unfinished txs or suffered tangible losses from the network outages.
That is hard to believe.
Why aren’t these users here complaining?
They are? Go twitter discord, telelgram.
You think crypto day traders post on developer forums? This is the exact problem with Icp. Single subnet controls the ledger.
Sure they do if they use ICP system and need help. Plenty people come on here before saying they lost coins in swap.
Checking Dfinity’s Twitter page I see 1 or 2 negative comments and not even about the tx… I bet you still using ICP!
Oooooh, one person! That’s “extremely bad” apparently.
One or two people, which is about the same number of people that complained on average even when ICP was up. So your comment was exaggeration.
Like I said, you and that ONE person will still be using ICP. It’s better than Solana
How often do you interact with Icp canisters?
Seems like not a lot.
Anyways…
Glad ICP is back up and running. Everyone here is still using it like always.
Appreciate all of the Dfinity’s hard work! #ICP
People have lost money? How?
I asked that member the samething and he pointed me to 1 or 2 people while making it seem it was a bunch of users. Most likely these few transactions are not finalized yet, and they are probably already resolved by the tech staff for those swaps.
To mislead people into thinking that it is occurring on a big scale without evidence is misinformation.
I used ICPswap over an hour ago and it works. Worked the same way it did the last time I used it a few days ago - same speed and everything else normal.
title change from: Today’s outage and the lack of communication is truly embarrassing
to
Today’s outage and the lack of communication was bad