reCap: AWS Global Network
AWS has the largest global footprint to help customers deliver better end-user experiences. AWS is rapidly expanding operations to virtually any region or country, and to meet their data locality and sovereignty requirements. See my recap: AWS Regions, AZs and Edge locations notes on AWS presence and location configurability.
Every AWS data center, AZ and AWS Region is interconnected via a purpose-built, highly available, and low-latency private global network infrastructure. We are going to review it’s capabilities and best practices in the notes below.
PS: Can this be one global network ? I don’t think so .. at-least not yet. As a solution architect we have to stitch the fabric & connect the dots.
Use of Global Network
Use Cases for (if any) refactoring your network infrastructure, are
- Scalable connectivity within AWS, namely VPC-to-VPC connectivity i.e., how far AWS Transit Gateway, VPC peering, and AWS PrivateLink can rescue known patterns.
- Connectivity to On-prem and other Cloud provider’s DCs, with DirectConnect, VPN ..etc
- Connectivity to the internet, with presence of various AWS Gateway options.
In doing so, we shall recap into supporting services for security, routing and application reachability using AWS WAF, load balancing with Elastic Load Balancing, Domain Name System (DNS) with Amazon Route 53, global traffic management with AWS Global Accelerator, and content distribution at a local and global scale with Amazon CloudFront, to name the few.
VPC to VPC
Quickly reviewing the basics,
VPC is a regional resource, and application workloads are bounded by Security Groups and network ACLs. Default VPC comes with default security group.
- Network ACLs defined at Subnet level, and are stateless (meaning return traffic/path is not learnt, have to be explicitly specified)
- Security Groups, defined at ENIs, are stateful.
And a default VPC in AWS, comes with, below traffic rules.
- Allow all Inbound and Outbound, at Network ACL list.
- Deny all Inbound and Allow all Outbound, at security group level.
You can share a subnet, across accounts (see Creating a resource share in the AWS RAM User Guide.) And you can also extend your VPC Region by creating a new subnet that has a Local Zone assignment. When you create a subnet in a Local Zone, you extend the VPC to that Local Zone.
Those are all the controls available within a VPC , to control the traffic within, at max you can extend VPC to a Local Zone but not globally then how about traffic need to traverse across, say
- To another VPC in the same or another AWS region, or
- Connecting privately to AWS services or 3rd party SaaS providers
VPC Peering
You can create a VPC peering connection between your own VPCs, or with a VPC in another AWS account.
You can also establish peering relationships between VPCs across different AWS Regions (also called inter-Region VPC peering) for global connectivity.
VPC peering is neither a gateway nor a VPN connection, so no single point of failure. Traffic always stays on the global AWS backbone, stay private, and never traverses the public internet.
Refer to VPC peering scenarios to understand the patterns used.
But there are gotchas in it’s management, as it requires
- Bidirectional peering i.e., routes are to be updated in both directions in both VPCs.
- Security Groups are to be updated to allow traffic from source range.
- N networks then N*(N-1)/2 peering connections are required.
- No transitive routing supported. Can’t use IGW or NAT in peered network and same is true with On-premises connectivity (VPN and/or Direct Connect) and must be made connect to each VPC.
- CIDRs can’t overlap
Further read: https://docs.aws.amazon.com/vpc/latest/peering/invalid-peering-configurations.html
What can rescue these challenges?
AWS Transit Gateway
AWS Transit Gateway provides a hub and spoke design for connecting VPCs and on-premises networks as a fully managed service. Transit Gateway’s routing tables allows us to isolate VPCs wherever needed (when automatic propagation turned off), making it highly configurable gateway.
Transit Gateway is a regional resource and can connect thousands of VPCs within the same AWS Region. For global connectivity use Transit Gateway Inter-Region Peering feature.
- Scale: 5000 attachments per Region & Up to 50 Gbps (burst)/attachment.
- Place your organization’s Transit Gateway instance in its Network Services account, for centralized control.
Use AWS Resource Access Manager (RAM) to securely share a Transit Gateway instance for connecting VPCs across multiple accounts in your AWS Organization within the same Region.
- Use Transit GatewayConnect BGP for dynamic routing and Generic Routing Encapsulation (GRE) tunnel, delivering up to 20 Gbps per Connect attachment (up to 4 peers per Connect attachment). Through on-premises SD-WAN infrastructure or SD-WAN appliances running in the cloud. This makes it Private connectivity.
SD-WANs, or Software Defined Wide Area Networks, have long been used to connect data centers and branch offices over the public internet.
Another solution to VPC peering challenges is Transit VPC, where one VPC connects to another using (overlay) VPN leveraging BGP over IPsec in a hub and spoke design. Hub uses customer Gateway appliance. See this document for differences. Transit Gateway simplify this architecture.
Shared VPC is still a possibility
- Use subnet sharing feature across accounts (using RAM).
- The two shared VPCs are connected via the Transit Gateway.
Cool, how about connecting privately with AWS services and SaaS providers?
VPC Endpoints
A VPC endpoint enables you to privately connect your VPC to supported AWS services. VPC endpoint services powered by AWS PrivateLink, without requiring an internet gateway or public network connection. Two types of End points, Gateway & Interface,
VPC Gateway Endpoints route traffic by adding prefix lists (like s3.* ..etc) within a VPC route table.
- It’s only available for S3 & DynamoDB.
- can’t be transitive, when compared to Interface Endpoint.
- Only supports IPv4, when compared to Interface Endpoint.
- Supported by the Routing table hence no throughput limit.
VPC Interface Endpoint uses PrivateLink to inject into a VPC at the subnet level, via an Elastic Network Interface (ENI). Its transitive hence enabled in Hub VPCs (in hub and spoke model). Throughput limited to 10Gbps.
PrivateLink
PrivateLink is only technology that allows 2 VPC with overlapping CIDR to connect.
It make sense to create all VPC endpoints in one dedicated VPC called Shared VPC, in place of above. Inter-region dependencies (at VPC level) can be overcome with Route53 DNS resolver.
Before going to our next network connectivity pattern i.e., VPC to On-prem connectivity, let’s recap all resources talked about in below slide.
VPCs to Internet
Can’t leave a VPC without talking about Gateways. I wish they could have aggregated into Transit Gateway with public endpoint.
Internet gateway (IGW) provides access to the Internet. Public IP is required by EC2 instance to use IGW.
Virtual gateway (VGW) provides access to the on-premises data centre through either VPN or Direct Connect connections.
A Network Address Translation (NAT) Gateway provides outbound Internet access for EC2 instances in private subnets. It is scoped to a particular AZ. Can burst upto 10 Gbps. It works One Way i.e., like forwarder, so no inbound initiation of traffic.
Key notes are,
- VPC can have only one IGW and VGW
- IGW does not limit the bandwidth of Internet connectivity compared to NAT Instance. NAT instances are hosted on Public subnets.
- IGW applied in VPC level whereas NGW is applied at instance level
- VGW, IGW, VPC peering is all through the route table.
Lastly,
Hybrid Networks
- VGW gave AWS customers the ability to let multiple VPCs, in the same region, on the same account, share a Direct Connect.
- VGW can only be used in a single account.
- Prior to VGW, a Direct Connect Private Virtual Interface (VIF) was required (1:1) for each VPC
Direct connect Gateway
DGW (Direct connect Gateway) builds upon VGW capabilities by adding the ability to connect VPCs across regions.
- CIDR addresses can’t overlap.
- DGW is used across multiple accounts.
In contrast, Transit Gateway provides enhanced routing services over preceding offerings from AWS.
- However, each VPN session is still limited to 1.25 Gbps of throughput. If you want to scale beyond this, you’ll need to add multiple VPN connections
- TGW, coupled with AWS Resource Access Manager, allows you to use a single Transit Gateway across multiple AWS accounts.
- CIDR overlap is also permitted with the addition of multiple route tables.
Site to Site VPN (S2S VPN) is another Hybrid approach with VGW supporting AES 256-bit and AES-128 encryption. One VGW is per one VPC, thus needing multiple VPNs if you have many VPCs. Can’t be transitive routing to another VPC even though it’s peered.
Global Networks won’t be complete without mentioning of Route53 and CloudFront services, here are key takeaways.
Route53
For routing of Name Resolution queries
Get recursive DNS for your Amazon VPC, private zones and on-premises networks. Plus create conditional forwarding rules and DNS endpoints to resolve custom names. With Route 53 Resolver DNS Firewall, you can filter and regulate outbound DNS traffic for your virtual private cloud (VPC).
Global traffic management: route end users to the best endpoint for your application based on geo proximity, latency, health, and other considerations. Routing based on,
- originating IP address
- Weighted Round Robin (WRR) functionality
Failover routing, automatically route your website visitors to an alternate location to avoid site outages.
- health-checking agents will monitor each location (or “endpoint”)
CloudFront
Amazon CloudFront is a content delivery network (CDN) service, with Edge computing services.
- Content Caching (both Static and Dynamic) & Origin Shield
- TLS termination, with integration to AWS Certificate Manager (ACM)
- Access is restricted to content through a number of capabilities. With Signed URLs and Signed Cookies, Token Authentication is supported to restrict access to only authenticated viewers.
- CloudFront’s native origin failover capability automatically serves content from a backup origin when the primary origin is unavailable.
- Protection against Network and Application Layer Attacks
Global Accelerator
Comes with anycast static IP addresses that serve as a fixed entry point to your applications hosted in one or more AWS Regions. This enables traffic to ingress onto the AWS global network as close to your users as possible.
Improving the end to end latency, no round trip to Route53 as no update DNS records is required. By default, AWS Global Accelerator is protected by AWS Shield Standard, so DDoS detection at the edge.
Further references
from AWS Summit: Advanced VPC Connectivity Patterns — Level 400 (United States)
Private NAT Gateway to achieve overlapping CIDR ranges, from
Hybrid DNS resolution with Amazon Route 53 Resolver Endpoints
AWS Firewall Manager
Then comes management, a AWS Firewall Manager to simplify your VPC security groups administration by centrally configure and managing rules across multiple accounts and resources in a AWS Organization. It can enforce a common set of security rules to all existing and newly build applications, from a central administrator account. That apart,
- Roll out AWS WAF rules (protecting your endpoints)
- Associate your VPCs with Amazon Route 53 Resolvers DNS Firewall rules (i.e., filter and regulate even blocking outbound DNS traffic)