September 3, 2020

Introduction

On the morning of August 30, 2020, a disruption in an internet backbone service (also known as a ‘middle mile’ network) provided by CenturyLink caused outages and significant service degradation to most internet users in the US. This event highlights several often-overlooked factors that affect user experience, including the role of middle-mile providers, and the combined impact of latency and loss.

NetForecast operates an independent Internet benchmarking service that reports critical deviations from normal baseline performance. We collect data nationwide to produce a score that is uniquely suited to reporting this type of event based on testing that always includes the middle mile and the combined contribution of latency and loss into a single score.

Methodology

Scoring is accomplished using the Application Performance Index (Apdex) open standard, based on counting samples that fall within performance bins designated as “satisfied,” “tolerating,” and “frustrated” to the user. The exact threshold values demarking the bins are determined by NetForecast’s statistical models. Intrinsic in this approach is counting loss conditions in the frustrated group.

NetForecast tests using probes located in consumer homes directly connected to major broadband service providers’ routers (no Wi-Fi involved). The tests operate between the probes and reference servers (targets) located near the largest US Internet Exchange Points (IXPs). These tests operate across carefully selected probe/target pairs to provide a comprehensive US internet performance map.

August 30th Event

The following chart shows how the CenturyLink outage adversely affected Apdex scores in major US cities. Apdex scores range from zero (worst performance) to 1.0 (best performance). The Apdex score drops of 0.10 to 0.20 experienced in all the cities shown, capture performance degradation that users noticed and which made national news.

The following chart shows the outage’s impact on major US ISPs.

Unsurprisingly, CenturyLink experienced the most significant impact from the event. As a provider of both local service and middle-mile service, CenturyLink routes more middle-mile traffic through its own network, and thus its users were at the event’s epicenter.

Conclusions

The loss of a single middle-mile provider can have severe and wide-ranging consequences on user performance. While most discussion to date has revolved around the importance of speed testing in measuring internet experience, the effect of latency and loss on the user experience is at least as important, and in many cases more important than speed.

The approach NetForecast has developed incorporates the effect of middle-mile performance and the metrics of latency and loss into a single easily understandable number. The underlying data enable detailed analysis of any combination of these metrics by geography and ISP.