Host your web apps on EKS with Nginx Ingress and External DNS

In this guide, we’re trying to expose a web application that is hosted on EKS to the internet (or within your VPC) using Nginx Ingress and Externa DNS. So here’s what we’re going to do.

1. Install Nginx Ingress controller on our Kubernetes cluster.
2. Create a hosted zone and IAM roles.
3. Install External DNS on the cluster.
4. Expose the web app using Ingress with a unique domain name.

For all the installations we’ll be using Helm. Helm is basically a package manager for Kubernetes. It’s like apt, yum, or dpkg for Linux.

Prerequisites: An EKS cluster, a VPC, Private/Public subnet(s) (depending on your requirement), and a Domain.

Install Nginx Ingress controller on our Kubernetes cluster

NOTE: If your EKS cluster was created after version 1.19 was released, you might need to add some tags to your private and public subnets. Please go through this article by AWS and do the needful to make your subnets visible by the Ingress controller. The tags may vary based on whether you want to access your application from the public internet or just within your internal network.

To install the Nginx Ingress controller, you can either get it from their official Helm repo or download the Helm charts from their GitHub repository. We’re going to install it from the Helm repo.

$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
$ helm repo update 

Now we only have to install the charts with a customized value set. For that, we can either provide values directly with the ‘helm install’ command or we can create a values file. Creating a values file is better since we have a few many values to set in this case. The following example value set is for creating an external Nginx Ingress controller. This means that it will create an internet-facing load balancer for the controller in our AWS account.

You can customize the deployment as you like with the above options and many other options that you will find in their official chart’s values file. You can set your own SSL certificate stored in the AWS certificate manager or you can remove that option. If you only want to access the applications within your internal network, you will need to create an internal load balancer instead of an internet-facing one. For that, just uncomment the last annotation and it will automatically pick up the private subnets to create the required load balancer.

After setting up the values file accordingly, you can continue with installing the Helm charts.

helm install <release_name> ingress-nginx/ingress-nginx --namespace <namespace> --values <values_file> --create-namespace

The above command will install the Nginx Ingress controller with whatever release name you set in the specified namespace. The create-namespace option will create the namespace if it doesn’t exist.

If everything goes well, you would have a LoadBalancer type service created in your namespace with the load balancer’s DNS name as the external IP like shown below.

Nginx Ingress controller service

Create a hosted zone and IAM roles

Before installing External DNS, we need to have a Route 53 private or public hosted zone for our domain name. Note that we are going to use subdomains of this domain for our applications. You can either buy a domain directly from AWS or if you bought a domain from another provider like GoDaddy, replace its default nameservers with your Route 53 hosted zone’s nameservers.

After that, we need to create an ‘OpenID Connect’ type identity provider so that we can assign it to the IAM role that we’re going to create for External DNS to manage the DNS records in our hosted zone.

Follow this guide to create an OIDC provider. You can find your cluster’s ‘Provider URL’ under EKS > Clusters > cluster_name > Details. And set the audience as sts.amazonaws.com

When creating the IAM role, set the Trusted entity type as Web Identity and select the Identity Provider that you created in the previous step.

For permissions, create a new policy and set the permissions in JSON format as shown in the example below. Make sure to add all your hosted zones under resources if there are multiple.

{
    "Statement": [
        {
            "Action": "route53:ChangeResourceRecordSets",
            "Effect": "Allow",
            "Resource": [
                "arn:aws:route53:::hostedzone/XXXXXXXXXXXXXXXXX"
            ],
            "Sid": ""
        },
        {
            "Action": [
                "route53:ListResourceRecordSets",
                "route53:ListHostedZones"
            ],
            "Effect": "Allow",
            "Resource": "*",
            "Sid": ""
        }
    ],
    "Version": "2012-10-17"
}

Install External DNS on the cluster

If you try to access the Nginx Ingress controller’s external IP from your browser, it will take you directly to the Nginx Ingress controller. Now we need a way to direct that traffic to the required microservice in a user-friendly way with DNS. That’s where External DNS comes into play. External DNS will detect new ingresses that are created in our cluster and create the relevant DNS records for us in AWS Route 53 hosted zones.

We’re going to install External DNS from its official GitHub repo. Alternatively, you can use Bitnami as well.

helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
helm repo update

Before installing the Helm charts, we need to set up a values file just like we did before.

You need to specify the IAM role that was created in the previous step as an annotation here. The domainFilters value sets the possible target zones by domain suffixes. The sources value defines the Kubernetes resources type to be observed for new DNS entries.

Now we can install the Helm chart with,

helm install <release_name> external-dns/external-dns --namespace <namespace> --values <values_file> --create-namespace

It’s better to use its own namespace for External DNS. No need to use the same namespace as the Ingress controller.

Check the logs of the newly created pod and you should get something like the below if it was deployed successfully.

Expose the web app using Ingress with a unique domain name

Now we can start creating ingresses for our applications. I’m creating a deployment with an Nginx container and exposing it with a unique subdomain as the host.

If you check the address of your ingress, you should see the same external IP as the Ingress controller before.

ingress

If you monitor the logs of your external DNS pod, you’ll notice something similar to this;

It adds an additional TXT record with an OwnerId field that we specified to avoid collision with the other DNS records. And if you try to access the service with your subdomain, you should see the Nginx welcome page as shown below.

Thank you for reading and good luck!