I wanted to move away from certmanager+lets-encrypt based SSL termination on my nginx ingress pods and use the free ACM generated SSL certs and terminate them on the ALB. This free up my nginx ingress pods and let them do what they do best, routing.
Solution: (aws-alb-ingress-controller + nginx-ingress-controller)
I wouldn’t go too much details on implementation as I have created a helm chart wrapping the whole thing together: https://github.com/sajid2045/eks-alb-nginx-ingress
- aws-alb-ingress-controller is still in beta, I didn’t want to risk it dynamically changing route rules based on my ingress , I have seen people troubleshooting the ingress rules not changing properly. Also this is a hard change in aws target group which I wasn’t too comfortable with.
- I didn’t want to use one alb per ingress rule (which adds up in cost)
- nginx-ingress is battle tested
- By combining the two, I get the best of both world. My wildcard based dns ingress is only registered once in aws and only changes when my nginx ingress pods scale up/down
- There is an extra hop involved which I will need to find out the performance cost based on load test results
- by binding the services to the
nginx Ingress, I am now de-coupling the ingress to my cloud environments. I can run the same pods on my onprem
- Ideally I wanted to use nlb but the SSL termination is still not supported on NLB for the k8s generated NLBs , but once that is enabled, i can just create a new ingress to my nginx svc
- This enables me to create nlb / alb / elb and have them used simultaneously without effecting the native ingresses.
With the extra hop, we need to be careful to treat the source ip. This is what it looks like from a echo server: