04 June 2018

ThousandEyes says it detected complete loss of connectivity from AWS' US-East-2 region to at least 10 other SaaS and API services.

Last Thursday, Amazon Web Services (AWS) reported a significant internet connectivity outage that knocked out connectivity for a whole host of services including Salesforce.

The incident affected the company’s AWS US-East-2 region, which covers Ohio on 31 May from around 12.11am to 12.45am PDT.

ThousandEyes deployed its Cloud Agent vantage points to map the public internet to show the impact of the outage on all applications hosted by AWS in the region.

The network intelligence specialist says it was able to detect complete loss of connectivity from US-East-2 to at least 10 other services which, as well as Salesforce, also included Signal, Rackspace and Akamai. 

Alex Henthorn-Iwane, VP of product marketing for ThousandEyes, says: “This outage uniquely illustrates how modern apps and micro-services depend on an intensive mesh of inter-service communication between public clouds, CDNs, hosting providers, SaaS platforms, and cloud-based payment API endpoints. 

“With enterprises building so much dependency on communications that cross infrastructures, networks and services that they don’t own or control, it’s more important than ever to understand these dependencies so they can manage the digital experiences they’re delivering to customers and employees.”

He goes on to note that the outage highlights the fact that doing business on the internet is “far more intricate than the binary notion of ‘service X or cloud provider Y is down’, or ‘the internet just broke’.” 

“In this case, the affected service providers didn’t go down per se,” says Henthorn-Iwane. "Rather, the ability for customers who were building application and service workflows that involved AWS-East-2 and one or more of those external hosting, CDN or SaaS service providers would have suffered downtime due to a connectivity issue within the bowels of the internet, most likely due to a problem at a single ISP.”

In addition, he says ThousandEyes saw an illustration of the “incredible sophistication” of how Amazon manages its IP space as part of a complex and powerful service orchestration system.  

“We can see in our visualisation that communications from AWS-East-2 is going to a very large number of IPs. These are not network device interfaces in the traditional sense, because AWS manages its IP address space differently from how it’s typically done in traditional network infrastructures. 

“In the AWS infrastructure, there is a large degree of abstraction and virtualisation of computing and networking infrastructure and connectivity. So IP address space isn’t ordered in a typical network and sub-network hierarchy.

"IP addresses are drawn from a database and assigned based on service and network connectivity instantiation.”