High User Traffic Incident Retrospective - Thursday September 2, 2021

We have made a number of improvements and more are in progress to address this performance issue. We have improved the efficiency and parallelism of canister smart contract execution and we have improved our load shedding at the node level. This will enable us to increase the traffic which we allow through the boundary nodes and ultimately to fully decentralize the boundary nodes without risking the health of individual nodes, subnets and the IC overall. We in the process of adding caching at the boundary nodes and we will be adding caching support in the service worker and we are investigating adding caching at the replica node layer as well.

Safety and stability are of utmost importance and while we are disappointed this subnet cannot handle all the load being thrown at it at this time, we are confident that we can continually improve performance.

7 Likes