StarRez Root Cause Analysis
Switzerland North Outage - 2nd Nov 2022
Summary
On the 2nd Nov 2022, customers within the Switzerland North region experienced an outage of up to 1hr20mins for core services.
The cause of the outage was underlying infrastructure experiencing resource exhaustion which also triggered an underlying bug that StarRez is currently working to resolve with our upstream vendor.
Root Cause
The root cause was determined to be underlying node resource exhaustion which led to services being moved elsewhere within the cluster.
During this move a known bug was encountered which impacts network connectivity, further delaying the startup of customer resources within the cluster.
Resolution
The problematic node was removed from service and the cluster was scaled to handle the increased workload to allow applications to start again.
StarRez engineers will continue to work with our upstream vendor to resolve this ongoing bug within the platform.