Amundsen — Installing in an Istio-enabled environment

A couple of months ago, I managed to install Amundsen into my company’s AWS EKS environment where Istio is enabled on top. So I’ve decided to write an article to share & document my learnings throughout the journey, and I hope this will help other engineers in implementing Amundsen as well.

I’d assume that readers of this article have some basic knowledge aroudn the istio product so I won’t spend too much time explaining what it is. If you are new to Istio, you might visit their website here to understand more a bit.

Writing ServiceEntry in the Amundsen helm chart

Looking at the official helm chart in the amundsen repo here, you can’t find any istio-related manifest files, meaning that you’d have to write your own yaml files, save into the templatefolder and run helm commands to deploy them into your k8s environment.

The first type of manifest files we have to write is ServiceEntry. ServiceEntry, in short, allows traffic to reach to the host defined in the manifest file. At my company’s environment, we host amundsen with AWS Opensearch & Neptune as backend database. So in order for amundsen to reach to the 2 DBs, I’ve defined the following ServiceEntry manifests .

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: opensearch
  namespace: amundsen
spec:
  exportTo:
    - "."
  hosts:
    - [Your Opensearch Hostname]
  location: MESH_EXTERNAL
  ports:
    - name: opensearch
      number: 9200
      protocol: TCP
    - name: https
      number: 443
      protocol: TLS
  resolution: NONE
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: neptune
  namespace: amundsen
spec:
  exportTo:
    - "."
  hosts:
    - [Your Neptune Hostname]
  location: MESH_EXTERNAL
  ports:
    - name: neptune
      number: 8182
      protocol: TCP
    - name: https
      number: 443
      protocol: TLS      
  resolution: NONE

There’re a couple more ServiceEntry that I defined for my company but I hope you get the idea — In an istio-enabled k8s environment, you’d need ServiceEntry in order for the traffic to reach the defined host. So depending on the infrastructure environment you are working with, you might want to add extra ServiceEntry definition (For example, I’ve also written ServiceEntry with hosts *.amazonaws.com and *.amazonaws.ap-east-1.s3 in order to reach some other AWS services.

Writing VirtualService in the Amundsen helm chart

The second set of manifest files to write is VirtualService . In an istio-enabled k8s environment, very likely traffic to a k8s service resource is blocked and you’d have to define VirtualService (And potentially, together with DestinationRule ) in order to expose the k8s service you defined. To learn more about how these 2 resources work together to control traffic, visit the official doc here.

In implementing Amundsen, since we’d only need to expose the frontend service to users, we can directly supplement a VirtualService like below without defining a DestinationRule .

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: amundsen-frontend
spec:
  gateways:
    - <Your gateway namespace>/<Your gateway name>
  hosts:
    - <Your hostname>
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: amundsen-frontend
            port:
              number: 5000

Note that the destination host amundsen-frontend is just the default value which matches the name of k8s service resources that matches with the frontend deployments. You can just hardcode it if you didn’t make changes to the service name / release name.

Time to run helm install

After defining all the above manifests files into the chart, we can finally run our helm commands to install it. You should see the below frontend page upon successful deployment :

I hope this article will provide some help in navigating an istio-enabled k8s environment and have Amundsen successfully installed in it. Stay tuned for more contents !