Exposing Apps to a Private Network using Istio Ingress Gateway

5 min readSep 2, 2020

Istio is an open source service mesh platform that provides a way to control how microservices share data with one another. Inbound and outbound traffic to a service mesh can be managed and controlled using Istio Gateways. You can specify the traffic that should be allowed to enter or leave the mesh. Gateways are standalone envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads.

Istio provides a preconfigured Ingress Gateway deployment to manage inbound traffic to the mesh. A public load balancer IP address gets assigned to the ingress gateway service when you setup Istio on any of the cloud providers. Istio ingress gateway service setup in an IBM Cloud Kubernetes Service (IKS) cluster is shown below.

$ kubectl get svc -n istio-system -o=custom-columns='NAME:metadata.name,EXTERNAL-IP:status.loadBalancer.ingress[0].ip'NAME                   EXTERNAL-IP
istio-ingressgateway   168.xx.xxx.xxx
istio-pilot            <none>
istiod                 <none>
prometheus             <none>

IKS creates a Network Load Balancer (NLB) in ibm-system namespace, to provide an external IP address. NLB pods deployed for ingress gateway service are show below.

$ kubectl get pod -n ibm-system -o=custom-columns=NAME:metadata.name,STATUS:status.phase -l ibm-cloud-provider-ip=168-xx-xxx-xxxNAME                                                   STATUS
ibm-cloud-provider-ip-168-xx-xxx-xxx-87567dd5-dw7cs     Running
ibm-cloud-provider-ip-168-xx-xxx-xxx-87567dd5-vs566     Running

In a mixed environment, you may want to expose certain applications on private network — accessible only to internal users or systems. In order to do so, an Istio ingress gateway with private IP address must be setup.

In this article, we will setup an internal ingress gateway with private IP address and expose an app to a private network. We will perform this exercise in an IBM Cloud Kubernetes Service (IKS) cluster. The details of the cluster are given below…

An IKS cluster with 2 edge worker nodes and 3 default worker nodes.
Kubernetes version 1.17
Network Load Balancer (NLB) 1.0
Istio Service Mesh version 1.5.2

1.Create an internal Istio ingress gateway with private IP address

We will use of IstioOperator API to create an additional ingress gateway in istio-system namespace. Update (or create) IstioOperator resource as shown below…

Add a new ingress gateway with name istio-ingressgateway-private under components.ingressGateways
Add the IBM Cloud service annotation for istio-ingressgateway-private under k8s.serviceAnnotations.

If you do not specify this annotation and your worker nodes are connected to public VLANs, a public LoadBalancer service is created. If your worker nodes are connected to private VLANs only, a private LoadBalancer service is created.

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-system
  name: istio-operator
spec:
  profile: default
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
      - namespace: istio-system
        name: istio-ingressgateway-private
        enabled: true
        k8s:
          serviceAnnotations:
            service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: "private"
  values:
    gateways:
      istio-ingressgateway:
        sds:
          enabled: true

Copy the above contents into a file istio-operator.yaml and execute the following command to apply the changes.

$ istioctl manifest apply -f istio-operator.yaml

An Istio ingress gateway with a private load balancer IP address 10.xxx.xxx.xxx gets created in istio-system namespace.

$ kubectl get svc -n istio-system -o=custom-columns=NAME:metadata.name,EXTERNAL-IP:status.loadBalancer.ingress[0].ipNAME                           EXTERNAL-IP
istio-ingressgateway           168.xx.xxx.xxx
istio-ingressgateway-private   10.xxx.xxx.xxx
istio-pilot                    <none>
istiod                         <none>
prometheus                     <none>

Check the NLB pods created for the above private IP address.

$ kubectl get pod -n ibm-system -o=custom-columns=NAME:metadata.name,STATUS:status.phase -l ibm-cloud-provider-ip=10-xxx-xxx-xxxNAME                                                   STATUS
ibm-cloud-provider-ip-10-xxx-xxx-xxx-68766764b-2zrqg   Running
ibm-cloud-provider-ip-10-xxx-xxx-xxx-68766764b-zbf8x   Running

2.Deploy an sample app

We will deploy a sample app that should be made accessible only to internal users or system.

Create a namespace app and enable automating sidecar injection by adding the label istio-injection=enabled.

$ kubectl create ns app
$ kubectl label ns app istio-injection=enabled

Deploy a hello-world application in app namespace.

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: hello-world
  labels:
    app: hello-world
spec:
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  selector:
    app: hello-world
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
      - image: dinup24/hello-world
        imagePullPolicy: IfNotPresent
        name: hello-world
        ports:
        - containerPort: 8080
EOF

3.Expose the app through ingress gateway on a private endpoint

We will create gateway and virtual service resources to expose the app on a private endpoint. In order to apply these configurations on the private ingress gateway, we will specify a label selector istio: ingressgateway-private in gateway resource.

The following command creates gateway and virtual service resources…

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: hello-world-gateway
spec:
  selector:
    istio: ingressgateway-private
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "hello-world.app.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
  - "hello-world.app.com"
  gateways:
  - hello-world-gateway
  http:
  - route:
    - destination:
        host: hello-world
        port:
          number: 8080
EOF

Try accessing the application from your private network…

If you system is not part of the private network, you may consider using a VPN to join the private network.

$ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway-private -o jsonpath='{.status.loadBalancer.ingress[0].ip}')$ curl http://hello-world.dev.com/api/greetings --resolve hello-world.app.com:80:$INGRESS_HOSTHello World!

The below diagram illustrates how separate instances of network load balancers handle requests on public and private endpoints. NLBs forward the request to the corresponding ingress gateways, which in turn routes the requests to apps based on the routing rules configured.

Summary

Inbound and outbound traffic to a service mesh can be managed and controlled using Istio Gateways. Gateways are standalone envoy proxies that are running at the edge of the mesh.
Certain apps running in a service mesh may need to be exposed to a private network, so that its accessible only to internal users and systems. A separate instance of ingress gateway can be setup with a private endpoint to provide restricted access to these apps.
In this article, we saw how to setup a private ingress gateway in IBM Cloud Kubernetes Service (IKS). We also deployed a sample hello-world application in a service mesh and exposed it to private network using the private ingress gateway.

Exposing Apps to a Private Network using Istio Ingress Gateway

Summary

References

Written by Dinup P Pillai