Hybrid Cloud Monitoring with Instana and IBM Secure Tunnel Network

Authors:
Lijian Wang, Guanqin Zhang & Sunny Tian

Introduction

Today, workloads can be deployed across hybrid clouds, so it’s very common to have multiple clusters and virtual machines. Some are hosted in an Enterprise Private Cloud while others are hosted in a Public Cloud like AWS, Azure, AliCloud, Tencent Cloud in China. You might want to have a unified observability platform to monitor all those workloads.

This blog describes an approach to using IBM Observability platform by Instana and a Secure Tunnel network, to monitor Kubernetes clusters for both private cloud and public clouds — for example, China vendors Aliyun and Tencent, OpenShift, IBM Kubernetes Service, and so on.

Architecture

The Instana server and IBM Cloud Paks as managed-from are installed in Enterprise Private Cloud. IBM Secure Tunnel network can be used to set up communication between public clouds as managed-to and the lab private cloud so the Instana agent deployed for workloads from public clouds can connect back to the lab environment.

Prepare environment

• Redhat OCP Cluster in Private Cloud
• Alibaba Cloud ACK Cluster
• Tencent Cloud TKE cluster
• Instana Server

Install Instana

1. Follow the guide here to deploy the Instana server as self-hosted. On the final step, before the install completes, a console URL is generated as https://<hostname>, as well as credentials to access. Note down the url because it will be used when you install the Secure Tunnel network as described in the next section.
2. Install the Instana agent in the public Kubernetes clusters. Follow the guide here.
3. You will see an error in the Instana agent pod log indicating that it cannot connect to the Instana server. This is because Instana agent and server are in different networks and they don’t talk to each other directly. That’s when you need Secure Tunnel network.

Let’s continue.

Setup Secure Tunnel Network

As the number of services, clusters, locations, and cloud providers continually grows, there is a rising need to access resources in the private network from other networks. Secure Tunnel is introduced to achieve this goal. It packages TCP requests and responses, encrypts them with HTTPS, and transports the payloads between endpoints in different networks.

As described in the Prepare environment section, two Tunnel networks can be created respectively for connection(tunnel-instana ) from AliCloud ACK and connection(instana-tencent) from Tencent TKE.

1. Follow the guide here to install the tunnel network server in the private cloud OpenShift cluster. Currently IBM Secure Tunnel is included as part of IBM CloudPak for Multicloud Management. After you install the IBM CloudPak for MultiCloud Management, you can get the IBM Secure Tunnel . For more information, see IBM Documentation.
2. Install the tunnel client in the public cloud clusters. The IBM documentation has not yet covered the steps to install tunnel client on the ACK and TKE. Here are the missing special steps that you’ll need to follow:

a. Remove or comment out the Route creation part in tunnel-client.yaml. This is necessary because the install script provided in Obtaining the tunnel client installation script supports only OpenShift. Specifically, this is what to be removed:

b. Manually create ingress in public clusters. Here is a working yaml example:

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

annotations:

kubernetes.io/ingress.allow-http: ‘false’

nginx.ingress.kubernetes.io/backend-protocol: HTTPS

nginx.ingress.kubernetes.io/secure-backends: ‘true’

nginx.ingress.kubernetes.io/ssl-passthrough: ‘true’

name: sre-tunnel-client

namespace: tunnel-client

spec:

rules:

- host: >-

sre-tunnel-client.cee89c7f4ee204045b00b35acf5967a44.cn-zhangjiakou.alicontainer.com

http:

paths:

- backend:

serviceName: sre-tunnel-3569fac40a0849fa-svc

servicePort: 8080

path: /

pathType: Prefix

c. Run install-openshift.sh to install the tunnel client.

d. Modify the DNS resolution of the public cluster. Follow the optional step #6 to get the tunnel-dns-service-ip, and then update coredns configmap in kube-system namespace by adding an entry like the following example

luban-instana.xxx {

errors

cache 30

forward . 192.168.137.146:1053

}

3. Verification

a. If you enter the Instana agent pod and run curl command against the Instana server URL, it should be able to connect to the server and the Instana agent pod log will indicate that it is connected.

2021–11–09T13:19:24.201+00:00 | INFO | stana.luban.cdl.ibm.com:1444–2–1 | Backend | com.instana.agent — 1.1.602 | Connected using HTTP/2 to luban-instana.<xxx>:1444 with id ‘00:16:3e:ff:fe:12:aa:93’ and key ‘xxxxxx’

b. For each of the tunnel networks, ensure you’ll see Status that is in Ready state and Connections column shows the actual connections count through it.

From now on, Instana agent automatically discovers resources in the two public cloud clusters and they are monitored as part of the hybrid cloud monitoring family!