For the 2nd time today, I’ve noticed Transaction Spike and degraded performance across most of the dApps, including nns. Location: Europe. Can we please get some info. Thanks.
2 Likes
Hi @vger
Yes, indeed, the boundary nodes came to their limits as the number of open connections increased drastically across all boundary node regions today around 11am UTC.
As reaction to the previous events, the boundary nodes have now infrastructure to ban misbehaving clients for short periods. Until now, this was running in log-only mode to gain experience. During the high load, we enabled it and could immediately see improvement.
In the past weeks, we have rolled out many performance improvements and other measures in the boundary nodes to better handle these types of events. For this week, we have another release planned and we continue improving the boundary nodes. Sorry about the hiccups along the way!
The Internet Computer was not affected by these events and continued operating normally.
Just a comment about the transactions: These are the requests that make it to the replicas and don’t actually reflect all the requests the boundary nodes see. For example, yesterday around 4pm UTC, the sale of the BTC Flower Gen 2.0 started, which lead to a steep increase of requests, which the boundary nodes and the Internet Computer handled without any issues.
7 Likes
Hi @rbirkner, I appreciate your thorough response. Keep up the good work and please keep us updated on the progress. Thank you.
2 Likes