Going the distance: Building Cross-Continent connectivity with AWS Global Network Infrastructure and Transit Gateway (Part One)

This is the first instalment of a multi-part blog posts on AWS Global Infrastructure and Transit Gateway.

Undoubtedly cloud computing has revolutionised how we build and run applications. While we look at the cloud as a convenient, immense global data center with limitless capacity, we often don’t realise the vast physical infrastructure behind the scenes. Specifically, for its network connectivity, AWS relies on a highly available, low-latency fiber-optic private global network built and operated by AWS itself. Colloquially called the “backbone”, this vast interconnected network connects all AWS regions (continents), ensuring our data travels securely and efficiently within the AWS cloud at the speed of light.

AWS Backbone key tenets:

1) Global Fiber Network — AWS has spent over a decade building a massive global fiber network that spans hundreds of thousands of kilometers. This interconnects all AWS regions and availability zones.

2) Private Network — The AWS global network is an advanced private network, not using the public internet. This allows AWS greater control to ensure high availability and maximum security.

3) Submarine Cables — Many inter-region AWS network links use high capacity submarine communications cables on seabeds across oceans. This provides low latency connectivity.

4) Multiple Paths — There are always multiple redundant fiber paths between AWS facilities to prevent any single point of failure.

5) Proprietary Network Gear — AWS designs its own network hardware/software for cloud efficiency at massive scale unachievable with off the shelf gear.

6) Dedicated Autonomous System — The AWS private global network is Amazon’s own Autonomous System (AS16509), managing routing within AWS infrastructure.

7) Encryption — All traffic on the AWS global network is encrypted for security using AES 256-bit encryption by default. Customers can further encrypt their VPC traffic.

Each tenet highlights an important principle, design decision, or capability regarding the AWS global network:

• Global reach
• Private controlled infrastructure
• Low latency focus
• Resilience via redundancy
• Custom hardware/software
• Own autonomous routing
• Encrypted links

Apart from those outlined tenets, the exact infrastructure and technologies powering the AWS backbone network remain largely a well guarded secret. Particularly the route redundancy, the latency optimisation proprietary hardware and the vast system of underwater cables carrying data across oceans.

Rightly, AWS is quite secretive about details of its global network for competitive reasons. For those interested for a sneak peek in AWS’s backbone infrastructure, I highly recommend watching the “Amazon Global Network Overview with James Hamilton” presentation from the AWS re:Invent 2016 event.

For who doesn’t know, James Hamilton is a distinguished engineer at AWS, and is best known for his behind the scenes work overseeing Amazon’s sprawling network of data centers that power AWS cloud-computing services. James Hamilton is considered quite a legend in the field of distributed systems and cloud infrastructure. Before joining Amazon in 2008, Hamilton had 20+ year career at Microsoft and IBM, becoming a widely respected leader in software infrastructure. His 2007 research paper “On Designing and Deploying Internet-Scale Services” is still regarded as the most inspiration reading for anyone building large-scale cloud services.

Throughout the talk, James highliths the “The Hawaiki Trans-Pacific Cable” a significant network infrastructure project that spans approximately 14,000 kilometres reaching depths of up to 6,000 meters below the sea, connecting Australia and New Zealand with the United States west coast. This extraordinary undersea cable is engineered to address the deep sea hars challenges, such as high pressure, tidal movement, abrasion, dragging anchors and last but not least, big sharks biting on the cable.

It’s been almost ten years now, and in my humble opinion, this remains one of the best presentations at re:Invent. James Hamilton showcased is deep passion for networking, performances and innovation whilst keeping the audience captivated and engaged throughout the whole talk. While AWS Transit Gateway was still three years away from being made generally available in 2019, this extraordinary presentation established the fundamental concepts and vision that would ultimately give life to the AWS services and its capability to scale and reach the four corners of the planet.

Stay tuned for the second instalment of this blog post, where we’ll dive deeper into the architectural components and cross continent deployment of two AWS Transit Gateway connecting Australia and the United States utilising Terraform.