StarRez Root Cause Analysis
StarRez's upstream vendor experienced issues within storage and compute infrastructure, resulting in backend database and compute services becoming inaccessible.
Root Cause
At 10:36PM UTC, 18th July 2024, our upstream vendor experience issues within storage clusters and compute resources in the Central US region. This resulted in connectivity to a subset of backend database and compute infrastructure being lost.
Resolution At 11:25PM UTC, 18th July 2024, a subset of customer services were restored as load was moved to functional compute infrastructure by StarRez engineers. The remaining customer base remained impacted due to connectivity issues with the backend database infrastructure. At 2:40AM UTC, 19th July 2024, backend database connectivity was restored by our vendor and all remaining customer services that were impacted came online.
Next Steps
A review of redundancy in the region will occur to determine if any adjustments can be made to improve resilience.