No Search Engine Challenge: Kubernetes with Minikube
Kubernetes has awesome tutorials and documents. They were so awesome that I found myself issuing the commands in the Hello Minikube tutorial without fully understanding what they meant. Ah, the perils of labs. To structure my learning, I gave myself a challenge. Using only the content in the Hello Minikube tutorial:
- Can I get my own Python sample application and database up and running?
- Can I gain a rudimentary understanding of how Kubernetes works with Docker?
However, I sternly constrained myself with a few rules:
- You can only reference the content in Hello Minikube tutorial. This excludes links and any search engine queries.
- You can use your existing knowledge of Docker.
- You can use your existing knowledge of container networking and service discovery.
- You can use the
kubectlcommand line help.

Rounds (Exercises/Objectives)
Below outlines some key exercises for this challenge. An italicized challenge indicates that it may only be partially covered by the “Hello Minikube” tutorial.
- Get your application up and running.
- Build a local Docker image to use in your Minikube.
- Run the Docker image on a Kubernetes node.
- Check your application logs. - Get your database up and running.
- Get your application to connect to your database service.
- Get the IP address of the database.
- Get an alias that will resolve to the database.
- Reverse engineer how that alias resolves. - Make an API call to your application (e.g., I run my integration tests).
- Expose your application.
- Check that you can access your application from your laptop.
Round 1 Fight! (The App)
Traditionally, you issue docker build to create your container image locally. Minikube can do the same thing. I decided to try my Python + MongoDB sample application, since it is fairly straightforward. It consists of a Python Flask application and a MongoDB database.
Challenge: Build a local Docker image to use in your Minikube.
This was pretty much covered by the tutorial. A very important command is eval $(minikube docker-env). This sets my local Docker to my Minikube’s Docker. I built my application using docker build -t mysampleapp:0.1 . .
Challenge: Run the Docker image on a Kubernetes node.
This was also covered in the tutorial. After completing the instructions, I checked my deployments and examined my pods.
$ kubectl run mysampleapp --image=mysampleapp:0.1 --port=80
deployment "mysampleapp" created$ kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
mysampleapp 1 1 1 1 55s$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysampleapp-3893302466-pfb12 1/1 Running 0 3s
I remembered that this particular application requires a database name and URI passed as an environment variables. The application will still run without them but the logs should have an error.
Challenge: Check your application logs.
Thanks to command line help pages, I found that kubectl has an option to get the logs for a deployment. Since I only have one container in my application deployment, those are the only logs that I will output.

I do indeed see the error message that tells me I need my DB_NAME as an environment variable. Now, I just need a database name and URI to pass to my application…
Round 2 Fight! (The Database)
I ran my MongoDB database with the same command as my sample application, just updating the port to the default MongoDB port.
$ kubectl run mysampleapp-database --image=mongo:latest --port 27017
deployment "mysampleapp-database" createdThen, I check my deployments and pods for my database.

I nearly worked myself into a panic when I saw that my container was still creating for my database pod. As it turns out, I was just impatient.

Celebrate! The commands were not much different than the application but it was nice to understand how the status reflects the readiness of the containers. I don’t have a database with any sort of high availability configuration — perhaps I will leave that for another challenge.
Round 3 Fight! (Connect App to Database)
I’m starting to get the hang of these commands. Now time to connect my application to my database. One method of doing this is by retrieving the IP address of my database container and passing it to my application.
This is not the ideal way to should do this. However, I was curious.
Challenge: Get the IP address of the database.
I need the IP address of the database. I check my kubectl command line help and settle on the “describe” command. Perhaps that contains some reference to the container’s IP address.

I’ve got an IP address and it looks like my database is running on the default Docker bridge (172.17.0.5). Just for kicks, let me pass that to my application. How do I pass an environment variable to my deployment? I used the command line and the handy grep function to look for an environment variable field.

With that in mind, I passed my environment variables to kubectl. When I check my application’s logs, I notice that the application has started properly with the default Flask start-up sequence.
$ kubectl run mysampleapp --image=mysampleapp:0.1 --port=80 --env="DB_NAME=test" --env="DB_URI=mongodb://172.17.0.5:27017/test"$ kubectl logs deployment/mysampleapp
* Running on http://0.0.0.0:80/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 308-493-950
The excellent news is that it has started. However, I have just hard-coded my database IP address, which is not a good idea because IP addresses are dynamic in the cloud.
Challenge: Get an alias that will resolve to the database.
I wonder if there is an alias for the database containers at the pod level that I can pass instead. In the “Hello Minikube” tutorial, I should be able to expose my database as a service. However, I’m looking to keep my database only resolvable within the cluster. Therefore, there should be a way to resolve to my database pod’s internal address without the need to expose it.
Admittedly, I tried everything. I could access it via the internal Docker IP address, as demonstrated above. However, it looks like no combination of pod, replica set, or container name will resolve to that same IP address.
$ kubectl exec mysampleapp-46257247-708f6 curl 172.17.0.5:27017
It looks like you are trying to access MongoDB over HTTP on the native driver port.$ kubectl exec mysampleapp-46257247-708f6 curl mysampleapp-database-3508969018
curl: (6) Couldn't resolve host 'mysampleapp-database-3508969018'$ kubectl exec mysampleapp-46257247-708f6 curl mysampleapp-database-3508969018:27017
curl: (6) Couldn't resolve host 'mysampleapp-database-3508969018'
$ kubectl exec mysampleapp-46257247-708f6 curl mysampleapp-database:27017
curl: (6) Couldn't resolve host 'mysampleapp-database'
It does say in the tutorial that the pod is only accessible via its internal IP address, so maybe I need to create a service for the database after all. I followed the second part of the tutorial to expose my database service.
$ kubectl expose deployment mysampleapp-database --type=LoadBalancer
service "mysampleapp-database" exposedI completed the instructions with a successful browser window stating the typical Mongo HTTP port access message, which meant the service was created and exposed. Now, what do I put in my environment variables for my application? Maybe with the service exposed, I can just resolve to the service name.
- Let me confirm this is the case by checking the DNS lookup in my application container. Success!
$ kubectl exec mysampleapp-46257247-708f6 nslookup mysampleapp-databaseName: mysampleapp-database
Address 1: 10.0.0.190 mysampleapp-database.default.svc.cluster.local
- Maybe I can access it on the database port. I use
curlto get to the database. Success!
$ kubectl exec mysampleapp-46257247-708f6 curl mysampleapp-database:27017
It looks like you are trying to access MongoDB over HTTP on the native driver port.- I restart my application pod with the new environment variables. No error messages on start-up. Success!
$ kubectl run mysampleapp --image=mysampleapp:0.1 --port=80 --env="DB_NAME=test" --env="DB_URI=mongodb://mysampleapp-database:27017/test"
deployment "mysampleapp" created$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysampleapp-2784377183-x02bl 1/1 Running 0 7s
mysampleapp-database-3508969018-zw07m 1/1 Running 0 1d$ kubectl logs mysampleapp-2784377183-x02bl
* Running on http://0.0.0.0:80/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 308-493-950
Much to my confusion, I noticed that nslookup resolves to a 10.0.0.190 address. It’s an internal IP address but where is the gateway to that router? How does Kubernetes use that address?
Challenge: Reverse engineer how that alias resolves.
What is that 10.0.0.190 address linked to the service name? Some observations:
- It’s routable from my application container.
$ kubectl exec mysampleapp-2784377183-k2lj9 curl 10.0.0.190:27017
It looks like you are trying to access MongoDB over HTTP on the native driver port.- It’s routable from within my Minikube.
$ minikube ssh
$ curl 10.0.0.190:27017
It looks like you are trying to access MongoDB over HTTP on the native driver port.- It’s not in an interface on Minikube.

- When I do
kubectl get services, I notice that there is an entry for a “cluster IP” which is the gateway address for that network.
$ kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.0.0.1 <none> 443/TCP 7d
mysampleapp-database 10.0.0.190 <pending> 27017:31799/TCP 47mThe command provided a few helpful hints:
- The
10.0.0.Xaddresses on all of my services indicates that they are all in the same network. - The
10.0.0.1address for thekubernetesservice seems to serve as a cluster router (similar to Traefik, CloudFoundry’s router, etc.). - The services are port mapped from pod port to host port.
For DNS resolution, let me check my resolv.conf in my application container to figure out the network’s nameserver.
$ kubectl exec mysampleapp-2784377183-k2lj9 cat /etc/resolv.conf
nameserver 10.0.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:What is this magical 10.0.0.10 address? This has to be something internal to Kubernetes and resolvable within the node, based on the observations I outlined above. Maybe if I look at the properties of the node itself with kubectl describe nodes. Surprisingly, I found a lot of information about other system-related pods in my cluster, including a kube-dns pod!

I tried multiple times to access that pod for more information but I could not figure it out through kubectl. I went so far as to use Docker itself to give me more information.

My best guess is that there are a set of services associated with kube-dns that help resolve DNS within the cluster. They appear to leverage dnsmasq to facilitate service name resolution.
Round 4 Fight! (Run My App’s Integration Tests)
That was a tough round! After the challenge, I need to do a bit more research on how Kubernetes resolves its services. However, I still want to be able to connect to my application to run my local integration tests on my laptop.
Challenge: Expose my application.
Similar to the database, I am going to expose a service for my application.
$ kubectl expose deployment mysampleapp --type=LoadBalancer
service "mysampleapp" exposed$ kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-minikube 10.0.0.247 <nodes> 8080:30721/TCP 3d
kubernetes 10.0.0.1 <none> 443/TCP 7d
mysampleapp 10.0.0.67 <pending> 80:31850/TCP 7s
mysampleapp-database 10.0.0.190 <pending> 27017:31799/TCP 13m
Nice! When I check the service through minikube service mysampleapp, my browser comes up with my “Hello World!” page. When I look at my browser, the IP address catches my eye…

The IP address appears to be routable from my laptop. If I call the API from my laptop’s command line, I can reach it through this address.
$ curl http://192.168.99.102:31850/
Hello World!If I look at the network interfaces inside Minikube, I see that IP address is linked to the eth1 interface.

Minikube is running through VirtualBox on my laptop. When I check VirtualBox, the second interface is set to VirtualBox Host-Only Adapter. This means I can effectively treat Minikube as its own host.
Challenge: Check that you can access your application from your laptop.
Now that I’ve confirmed that I can route to my application from my laptop, I can finally run my integration tests to see if everything is working as intended. I have some simple integration tests to confirm some API functionality. I added the Minikube URL to my integration tests.

I run my integration tests, and they pass!

Summary
This was an awesome exercise for me. It solidified what I knew about container networking and enhanced my understanding of Kubernetes under-the-hood. Thanks to some awesome Kubernetes documentation in the command line and an understanding of basic Docker concepts, I felt pretty confident in exploring the technology. I enclosed some of the references to the open questions below. For the next challenge, maybe I can try this on a virtual machine — or even try out some of the network policy functionality!
References
- Get your application/database up and running.
- My Python sample application with database
- Hello Minikube tutorial
- Kubernetes Container Logging - Get your application to connect to your database service.
- Connecting Applications with Services
- Kube-DNS (DNS for Pods & Services)
- Debugging Services
- Cluster Networking - Make an API call to your application (e.g., I run my integration tests).
- Running Kubernetes on Minikube
