Debugging Networking Issues Using VPC Reachability Analyzer

In this post, we’ll see how we can use VPC Reachability Analyzer to debug networking issues in the AWS environment.

Step1: Create 2 VPCs and connect them using VPC peering. Next, launch 1 instance in both the VPCs. In the security group settings of these instances, allow access on port 22 from respective VPC CIDR only.

For both instances, we’ll also deny all traffic at the NACL level.

Step 2: Now go to VPC->Reachability Analyzer->Create and analyze path and specify our source(instance in VPC1), destination(instance in VPC2), and port(22).

Run the analysis. Once it’s completed, you can see the following findings in the result.

Based on the findings, we can see that route tables of both instances don’t have a path to access instances in another VPC.

Step 3: Add entries in the route tables to use our VPC peering connection to connect to other VPCs and run the same check again.

This time, we can see that traffic is getting blocked at the NACL level since we changed our default Allow rule to Deny.

Step 4: Revert the Default Deny rule in NACL to Allow and run the check again.

Now we can see that traffic is still not reaching instance2 as its security group does not allow traffic from outside the VPC.

Step 5: In the security group of instance2, allow access on port 22 from IP on instance1.

Now run the check again and it should be reachable now.

You can also view the entire network path used to reach out to instance2.

As you can see, a Reachability analyzer can be a very handy tool while debugging complex networking issues. Earlier we used to do all these checks manually but now we have a Reachability analyzer to take care of such analysis.

Reference: https://docs.aws.amazon.com/vpc/latest/reachability/what-is-reachability-analyzer.html