Introduction to HashiCorp Consul Connect with Kubernetes
I’ve been having a lot of conversations recently about Service Mesh and the updates to Consul which have allowed this. Consul has been around for many years now, and in mid-2018 HashiCorp released their service mesh capability — Consul Connect.
In this guide, I’ll walk you through how to use Kubernetes in GCP with HashiCorp Consul as a service discovery and service mesh tool, both inside and outside of a Kubernetes cluster. We’ll install Consul with just a few commands, then deploy some custom services that will then use Consul to discover each other over encrypted TLS via Consul Connect. We’ll then do the same thing for services outside of Kubernetes. Finally, we’ll leverage Intentions in Consul to tighten down the whole thing.
- Task 1: Prerequisites and setup
- Task 2: Install Helm to the Kubernetes cluster
- Task 3: Enable Stub DNS
- Task 4: Create an application that uses Consul Connect secure service segmentation
- Task 5: Service mesh outside Kubernetes
- Task 6: Leverage Consul Intentions to deny service communication
You can also review the guide in my github account with all the required files to make it work.
Task 1: Prerequisites and setup
Before we get started, I have already setup gcloud command line tool, helm and kubectl:
https://cloud.google.com/sdk/docs/downloads-interactive
gcloud initInstall helm and kubectl with Homebrew.
brew install kubernetes-cli
brew install kubernetes-helmService account authentication
It’s recommended that you create a GCP IAM service account and authenticate with it on the command line.
https://console.cloud.google.com/iam-admin/serviceaccounts
https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account
gcloud auth activate-service-account --key-file="my-consul-service-account.json"Ensure Google APIs are enabled
For your account, you need to ensure the APIs are enabled. If you are starting from scratch, they are most likely not, so you’ll need to enable them:
https://cloud.google.com/endpoints/docs/openapi/enable-api
Enable:
compute.googleapis.com Compute Engine API
container.googleapis.com Kubernetes Engine APICreate a Kubernetes cluster
Assuming your previous steps are setup correctly, then you should be able to create a standard-cluster via the CLI. We’ll call the cluster consul-cluster and deploy it to the australia-southeast1-a zone.
gcloud container clusters create consul-cluster \
--zone australia-southeast1-aConfigure kubectl to talk to your cluster
gcloud container clusters get-credentials consul-cluster \
--zone australia-southeast1-aTask 2: Install Helm to your cluster
Once we have our foundation laid, we can commence the rollout of Consul. The easiest way to deploy Consul to Kubernetes is using helm. First, let's initialise helm:
helm initYou can verify that the tiller-deploy service is running with:
kubectl get pods --namespace kube-systemYou should see several services with tiller-deploy-* somewhere near the bottom.
Next, create permissions for the service account so it can install Helm charts (packages).
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:defaultYou should see output:
clusterrolebinding.rbac.authorization.Kubernetes.io "add-on-cluster-admin" createdCreate a new file named helm-consul-values.yaml. Edit with the following code. Comments inline about what we are doing:
global:
datacenter: dc1
image: "consul:latest"# Expose the Consul UI through this LoadBalancer
ui:
service:
type: "LoadBalancer"# Allow Consul to inject the Connect proxy into Kubernetes containers
connectInject:
enabled: true# Configure a Consul client on Kubernetes nodes. GRPC listener is required for Connect.
client:
enabled: true
grpc: true# Minimal Consul configuration. Not suitable for production.
server:
replicas: 1
bootstrapExpect: 1
disruptionBudget:
enabled: true
maxUnavailable: 0# Sync Kubernetes and Consul services
syncCatalog:
enabled: true
Install Consul to the cluster. We’ll use a clone of the official HashiCorp Consul on Kubernetes GitHub repo and then install it using our values defined previously.
git clone https://github.com/hashicorp/consul-helm.githelm install -f helm-consul-values.yaml ./consul-helm
Note: If you don’t give the cluster a name with --name then the Helm chart will randomly generate one for you.
This step can take a few minutes.
A large amount of text will output to the screen, something like:
NAME: imprecise-albatross
LAST DEPLOYED: Wed Dec 19 12:35:22 2018
NAMESPACE: default
STATUS: DEPLOYEDRESOURCES:
==> v1/ConfigMap
NAME AGE
imprecise-albatross-consul-client-config 1s
imprecise-albatross-consul-server-config 1s==> v1beta1/PodDisruptionBudget
imprecise-albatross-consul-server 0s==> v1/Pod(related)NAME READY STATUS RESTARTS AGE
imprecise-albatross-consul-bsgbn 0/1 ContainerCreating 0 0s
imprecise-albatross-consul-gpwn4 0/1 ContainerCreating 0 0s
imprecise-albatross-consul-hwbjx 0/1 ContainerCreating 0 0s
imprecise-albatross-consul-connect-injector-webhook-deployw9gt4 0/1 ContainerCreating 0 0s
imprecise-albatross-consul-sync-catalog-569746b955-grlbt 0/1 ContainerCreating 0 0s
imprecise-albatross-consul-server-0 0/1 Pending 0 0s==> v1/DeploymentNAME AGE
imprecise-albatross-consul-connect-injector-webhook-deployment 0s
imprecise-albatross-consul-sync-catalog 0s==> v1/ServiceAccount
imprecise-albatross-consul-connect-injector-webhook-svc-account 1s
imprecise-albatross-consul-sync-catalog 1s==> v1beta1/ClusterRole
imprecise-albatross-consul-connect-injector-webhook 1s==> v1/ClusterRole
imprecise-albatross-consul-sync-catalog 1s==> v1beta1/ClusterRoleBinding
imprecise-albatross-consul-connect-injector-webhook-admin-role-binding 1s==> v1/ClusterRoleBinding
imprecise-albatross-consul-sync-catalog 1s==> v1/Service
imprecise-albatross-consul-connect-injector-svc 1s
imprecise-albatross-consul-dns 1s
imprecise-albatross-consul-server 0s
imprecise-albatross-consul-ui 0s==> v1/DaemonSet
imprecise-albatross-consul 0s==> v1/StatefulSet
imprecise-albatross-consul-server 0s==> v1beta1/MutatingWebhookConfiguration
imprecise-albatross-consul-connect-injector-cfg 0s
There are several key values to note here:
- Service Name:
imprecise-albatrossis unique to each install. We’ll reference this later so keep that in mind. ==> v1/Service:We’ll explicitly look at these in more detail, but good to know what has been deployed.- imprecise-albatross-consul-connect-injector-svc
- imprecise-albatross-consul-dns
- imprecise-albatross-consul-server
- imprecise-albatross-consul-ui
Wait another moment and then verify that all services are up:
kubectl get podsNAME READY STATUS RESTARTS AGE
imprecise-albatross-consul-bsgbn 1/1 Running 0 3m
imprecise-albatross-consul-connect-injector-webhook-deployw9gt4 1/1 Running 0 3m
imprecise-albatross-consul-gpwn4 1/1 Running 0 3m
imprecise-albatross-consul-hwbjx 1/1 Running 0 3m
imprecise-albatross-consul-server-0 1/1 Running 0 3m
imprecise-albatross-consul-sync-catalog-569746b955-grlbt 1/1 Running 0 3m
We can also verify that the services defined earlier for imprecise-albatross are running with:
kubectl get svcWe can then verify that Consul is up and running by visiting the exposed LoadBalancer IP address:
get svc imprecise-albatross-consul-uiNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
imprecise-albatross-consul-ui LoadBalancer 10.47.255.168 35.189.28.251 80:32085/TCP 28m
You should see Consul is running and has imported all our services:
Task 3: Enable Stub DNS
In order for Consul service discovery to work smoothly, we need to enable Consul within the Kubernetes DNS system.
Find the name of your DNS service with:
kubectl get svcIt should look something like imprecise-albatross-consul-dns
We’ll then create a quick script in bin called enable-consul-stub-dns.sh
mkdir bin
vi bin/enable-consul-stub-dns.shThe contents of the script:
#!/usr/bin/env sh# Loads a ConfigMap that allows pods to use the `.consul` TLD.
# Call with the name of your DNS service as deployed by the Consul Helm chart.
#
# bin/enable-consul-stub-dns.sh piquant-shark-consul-dns
#
# https://www.consul.io/docs/platform/Kubernetes/dns.htmlDNS_SERVICE_NAME=$1cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
labels:
addonmanager.kubernetes.io/mode: EnsureExists
name: kube-dns
namespace: kube-system
data:
stubDomains: |
{"consul": ["$(kubectl get svc $DNS_SERVICE_NAME -o jsonpath='{.spec.clusterIP}')"]}
EOF
Update the permissions of the script:
chmod +x bin/enable-consul-stub-dns.shPass the service name matching *-consul-dns to the stub DNS script just created (with your cluster name)
bin/enable-consul-stub-dns.sh imprecise-albatross-consul-dnsYou’ll then see something like this:
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
configmap/kube-dns configuredTask 4: Create an application that uses Consul Connect secure service segmentation
In order to test Consul Connect in this new cluster, we need some services to run. To make this guide simpler we have preconfigured two simple services:
- Counting Service: will start an incremental counter
- Dashboard Service: will connect to the Counting Service, retrieve the current count, and display this via a simple web page.
Lets first define both of these services as code and place them in a separate directory:
mkdir services && cd servicesvi counting-service.yaml
The code for the Counting Service:
apiVersion: v1
kind: Pod
metadata:
name: counting
annotations:
"consul.hashicorp.com/connect-inject": "true"
spec:
containers:
- name: counting
image: hashicorp/counting-service:0.0.2
ports:
- containerPort: 9001
name: http
initContainers:
- name: counting-init
image: hashicorp/counting-init:0.0.9
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: HOST_IP
valueFrom:
fieldRef:
fieldPath: status.hostIPvi dashboard-service.yaml
The code for the dashboard service:
apiVersion: v1
kind: Pod
metadata:
name: dashboard
labels:
app: "dashboard"
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/connect-service-upstreams": "counting:9001"
spec:
containers:
- name: dashboard
image: hashicorp/dashboard-service:0.0.3
ports:
- containerPort: 9002
name: http
env:
- name: COUNTING_SERVICE_URL
value: "http://localhost:9001"
initContainers:
- name: dashboard-init
image: hashicorp/dashboard-init:0.0.4
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: HOST_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
---
apiVersion: "v1"
kind: "Service"
metadata:
name: "dashboard-service-load-balancer"
namespace: "default"
labels:
app: "dashboard"
spec:
ports:
- protocol: "TCP"
port: 80
targetPort: 9002
selector:
app: "dashboard"
type: "LoadBalancer"
loadBalancerIP: ""The configuration you created uses several Consul commands to setup proxies between services.
The counting service needs to start an extra container running consul that manually starts its own proxy.
You’ll also notice the dashboard service needs to start an extra container running Consul that manually starts an upstream proxy to the counting service proxy.
Let's apply both of these services to get them running in our Kubernetes cluster:
kubectl create -f services/We can now view the LoadBalancer of the Dashboard service to see the two in action. Lookup the LoadBalancer service and collect the external IP:
kubectl get svc dashboard-service-load-balancerYou should see something like this:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-service-load-balancer LoadBalancer 10.47.242.43 35.197.183.60 80:31657/TCP 1mOpen the external IP in a browser. Observe in the top right-hand corner that the service is connected and counting is incrementing.
If you take another look at the Consul UI — you’ll notice that we have several new services representing the counting and dashboard services. If you drill into these further you’ll be able to see the IPs representing the service and the nodes they are running on.
We can then create some intentions to test the service. In the Consul UI, select Intentions. Create a new Intention:
You’ll see a Success message. Now, visit the Dashboard service URL again. You should see the service has stopped and that counting is unreachable :
You can have a further inspection of these services in the container, within Consul GUI. Some things to note:
- The services can come up at any IP address and using service discovery in Consul will find each other.
- The services are communicating over mutual TLS via side car proxies injected by Consul through Kubernetes.
Task 5: Service Mesh outside of Kubernetes
Right now, you should have witnessed the power of Consul for Service Discovery and Service Mesh within Kubernetes. However, one of the other great features of Consul is that it has been operating as service discovery and service mesh outside of Kubernetes for some time now.
To test this out, we’ll now create a new service running on a standard VM.
Step 1: Create a virtual machine
From the CLI, again we can create a simple instance:
gcloud compute instances create payment-service \
--zone=australia-southeast1-b \
--machine-type=n1-standard-1 \
--image=ubuntu-1604-xenial-v20181004 \
--image-project=ubuntu-os-cloudYou should see the payment instance created.
Step 2: Deploy your application (or mine)
Once the machine is up, connect to it via SSH and provision the server. The easiest way to do this for this guide is to borrow someone else’s code. In this case, I will use a colleague’s payment gateway as the test service. You can find it here.
This script will completely provision the instance with Consul, the payment service, and the Consul Connect integration.
gcloud compute ssh --zone australia-southeast1-b "payment-service"Once connected to the payment-service host:
curl -L https://raw.githubusercontent.com/emojify-app/stack/master/azure-kubernetes/packer/scripts/provision_payment.sh >> provision-payment.sh
chmod +x provision-payment.sh
./provision-payment.shThis will take several minutes to run.
After this completes, set up a directory for the next step:
sudo mkdir /home/ubuntu/.kube
sudo chmod 777 /home/ubuntu/.kubeStep 3: Configure Consul for your cluster
Given that Consul is running as a service in Kubernetes and so are our other services, to allow this new service to connect, we need to be able to authenticate to the Kubernetes API. To do this, we need two things:
- Updated kubeconfig file
- Access permissions
To achieve the first step, download the kube config file locally:
gcloud container clusters get-credentials consul-cluster --zone australia-southeast1-aWhere,
consul-clusteris your Kubernetes cluster name (if you changed it earlier)
You should see kubeconfig entry generated for consul-cluster.
This will download the config file to ~/.kube/config
Copy that file to the new server under /home/ubuntu/.kube/config
gcloud compute scp ~/.kube/config payment-service:/home/ubuntu/.kube/configTo get access permissions you can create a new service account in GCP, and then copy the resulting .json file to /home/ubuntu/.kube/config or you can use the service account you created for yourself as part of the prerequisites (used to connect to the cluster locally - but not recommended for anything other than testing) and copy it to the same directory. I’ll do the second way:
gcloud compute scp ~/Documents/GCP/cameron-huysmans-b45674d4d0bc.json payment-service:/home/ubuntu/.kube/creds.jsonNote: I have called the target file creds.json
Connect to the payment-service instance and update to the Consul startup service to match your files:
gcloud compute ssh --zone australia-southeast1-b "payment-service"Update the Consul service definition with your credentials file:
sudo vi /etc/systemd/system/consul.serviceAdd the following line at the end: Environment=GOOGLE_APPLICATION_CREDENTIALS=/home/ubuntu/.kube/creds.json
E.g.:
[Unit]
Description = "Consul"[Service]
KillSignal=INT
ExecStart=/usr/local/bin/consul agent -retry-join 'provider=k8s label_selector="app=consul,component=server" kubeconfig=/home/ubuntu/.kube/config' -data-dir=/etc/consul.d/data -config-dir=/etc/consul.d
Restart=always
Environment=GOOGLE_APPLICATION_CREDENTIALS=/home/ubuntu/.kube/creds.json
Ensure your .kube/config file hasn’t carried over any local settings. For mine, it had a few lines referencing gcloud which all started with /Users/cam/.../gcloud. You’ll want to ensure all of these just reference gcloud directly:
i.e. from this:
cmd-path: /Users/cam/Downloads/google-cloud-sdk/bin/gcloudto this:
cmd-path: gcloudWhen this is resolved, we can restart our services.
sudo systemctl daemon-reload
sudo service consul restartCheck that the service is ok:
journalctl -u consul.serviceYou should see something like:
Dec 19 06:01:45 payment-service consul[10023]: ==> Consul agent running!
Dec 19 06:01:45 payment-service consul[10023]: Version: 'v1.3.0'
Dec 19 06:01:45 payment-service consul[10023]: Node ID: '119c20f8-31ff-34d0-eb05-a519b224ce62'
Dec 19 06:01:45 payment-service consul[10023]: Node name: 'payment-service'
Dec 19 06:01:45 payment-service consul[10023]: Datacenter: 'dc1' (Segment: '')
Dec 19 06:01:45 payment-service consul[10023]: Server: false (Bootstrap: false)
Dec 19 06:01:45 payment-service consul[10023]: Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 8600)
Dec 19 06:01:45 payment-service consul[10023]: Cluster Addr: 10.152.0.6 (LAN: 8301, WAN: 8302)
Dec 19 06:01:45 payment-service consul[10023]: Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
Dec 19 06:01:45 payment-service consul[10023]: ==> Log data will now stream in as it occurs:
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] serf: EventMemberJoin: payment-service 10.152.0.6
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [WARN] agent/proxy: running as root, will not start managed proxies
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] agent: started state syncer
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce k8s os packet scaleway softlayer triton vsphere
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [INFO] agent: Joining LAN cluster...
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [WARN] manager: No servers available
Dec 19 06:01:45 payment-service consul[10023]: 2018/12/19 06:01:45 [ERR] agent: failed to sync remote state: No known Consul servers
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] agent: Discovered LAN servers: 10.44.2.10
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] agent: (LAN) joining: [10.44.2.10]
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] serf: EventMemberJoin: gke-consul-cluster-default-pool-c8ea6a77-4078 10.44.1.7
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] serf: EventMemberJoin: gke-consul-cluster-default-pool-c8ea6a77-1684 10.44.2.8
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] serf: EventMemberJoin: imprecise-albatross-consul-server-0 10.44.2.10
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] serf: EventMemberJoin: gke-consul-cluster-default-pool-c8ea6a77-g3f1 10.44.0.8
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] agent: (LAN) joined: 1 Err: <nil>
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] agent: Join LAN completed. Synced with 1 initial agents
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] consul: adding server imprecise-albatross-consul-server-0 (Addr: tcp/10.44.2.10:8300) (DC: dc1)
Dec 19 06:01:47 payment-service consul[10023]: 2018/12/19 06:01:47 [INFO] agent: Synced service "payment"Within the Consul UI, you should also see the Payment service and its proxy has registered.
If you are seeing errors for the payment service, it may need to be restarted as well:
sudo service payment restart
sudo service payment status
sudo service payment-proxy restart
sudo service payment-proxy statusYou are looking for active:running for both services.
You can test this from the CLI:
curl -X POST -d '{"name":"Cam"}' -H "content-type:application/json" localhost:8080Return Message:
{"message":"sorry insufficient funds","id":"dad2947e-d21f-4e7f-a649-ed4f3860e7c2"}Notes about the provisioning script:
- It will automatically install Consul
- It will automatically register the service with Consul (assuming your consul DC is
DC1in Task 2) - It will automatically register a Sidecar Proxy and expose localhost ports for the service
- You can test the service locally using
curl -X POST -d '{"name":"Cam"}' -H "content-type:application/json" localhost:8080 - It will always return
sorry insufficient funds - Any connection issues are typically due to Kubernetes permissions. Ensure you have the correct config file and
.jsonservice account permissions. - You can view the payment and Consul logs with
journalctl -f --output=cat -u paymentandjournalctl -f --output=cat -u consulorjournalctl -u payment.serviceand useSHIFT + gto get to the end of log.
Step 4: Open the GCP firewall for Payment service
The payment service has been configured to listen on port 8443.
gcloud compute firewall-rules create payment-service-connect \
--description="Used for the payment service" \
--direction=INGRESS \
--priority=1000 \
--network=default \
--action=ALLOW \
--rules=tcp:8443Task 6: Leverage Consul Intentions to deny service communication
Now that we have the service mesh working, we can experiment with disabling service intentions.
Step 1: Kubernetes to Kubernetes service
First, visit the GUI and Deny all
If we visit the Dashboard service IP address again we can observe that the counting has stopped.
Revisit the Consul GUI and add a new intention. This time create a new rule to allow the Dashboard service to connect to the Counting service.
Revisit the Dashboard service UI and observe that the counting has started again. We have successfully confirmed our service mesh is working in Kubernetes to now test for services outside.
Step 2: Kubernetes to native VM service
We can leverage our existing container for the Dashboard service to connect to our Payment service setup from earlier. However, first, we need to ensure that the upstream connection in the proxy has been configured.
First, let's destroy our Dashboard and Counting service containers because we are going to make some changes:
kubectl delete -f services/pod "counting" deleted
pod "dashboard" deleted
service "dashboard-service-load-balancer" deleted
Then, let’s edit the dashboard-service.yaml file, so that the annotation for upstream services looks like:
"consul.hashicorp.com/connect-service-upstreams": "counting:9001, payment:1234"Where payment:1234 is the new addition to the line.
Re-apply the configuration to Kubernetes:
kubectl create -f services/Wait for the services to come online and ensure they are working by reviewing the dashboard service IP address.
Visit the Consul GUI and ensure that the Deny intention created earlier for all services is either deleted or set to Allow.
Now, we can connect to the Dashboard service container and try to establish a connection to the payment service:
kubectl exec -it dashboard /bin/sh
apk add curl
curl -X POST -d '{"name":"Cam"}' -H "content-type:application/json" localhost:1234We should see the sorry insufficient funds return again.
In addition to manipulating Consul intentions from the GUI, we can also programmatically update them from the payment service machine:
Open a new terminal session and connect to the payment server:
gcloud compute ssh --zone australia-southeast1-b "payment-service"
consul intention create -deny dashboard paymentFrom your existing window connected to the dashboard container, observe the output of the connection string:
curl -X POST -d '{"name":"Cam"}' -H "content-type:application/json" localhost:1234`You should now see an output like this:
curl: (56) Recv failure: Connection reset by peerYou can have a play with more intentions and or learn about them here.
Note:
- We exposed localhost port 1234 as our upstream proxy port for the service, so this is the port we need to use locally to address it.
- If you have issues connecting to the payment service:
- ensure the intentions are correct and there are no blanket
deny * - ensure the GCP firewall is enabled
- reload the containers again with a different upstream port for the payment service.
Tearing it all down
Delete the payment-service instance:
gcloud compute instances delete payment-serviceDelete the Kubernetes cluster (and everything with it):
gcloud container clusters delete consul-cluster \
--zone australia-southeast1-aDelete the firewall rule:
gcloud compute firewall-rules delete payment-service-connectExtra: Debugging
# Connect to a container
$ kubectl exec -it my-pod-name /bin/sh# View logs for a pod
$ kubectl logs my-pod-name# See full configuration for debugging
$ helm template stable/consul
Within a pod (may require Consul pod or extra installation of curl).
# View all environment variables
$ env# Install tools on a container for curl and dig
$ apk add curl
$ apk add bind-tools# Use the Consul HTTP API from any pod
$ curl http://consul.service.consul:8500/v1/catalog/datacenters# Use service discovery
$ ping dashboard.service.consul
Advanced
Scale up deployments to start more counting services.
$ kubectl get deployments
$ kubectl scale deployments/counting-service-deployment --replicas=2Or, in a deployment:
spec:
replicas: 5Health checks:
spec:
containers:
- name: "..."
livenessProbe:
# an http probe
httpGet:
path: /health
port: 9002
# length of time to wait for a pod to initialize
# after pod startup, before applying health checking
initialDelaySeconds: 30
timeoutSeconds: 1
# Other content omittedhttps://kubernetes.io/docs/tutorials/k8s101/
https://kubernetes.io/docs/tutorials/k8s201/
Networking: https://kubernetes.io/docs/tutorials/services/source-ip/
$ kubectl get nodes --out=yamlOther/Random Notes
Run Service
https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/
https://docs.docker.com/get-started/part2/
https://www.consul.io/api/agent/service.html#register-service
Get Bearer Token for K8S Dashboard
gcloud config config-helper --format=json
