Monitoring JVM within a Docker container using Stackdriver

Introduction

Now that you have landed on this post, you must be running your Dockerized Java app inside a Google Compute Engine (GCE) VM instance and obviously want to monitor it. Well this post will show you how you can monitor the JVM using Stackdriver Monitoring agent and JVM plugin. Though there are many other methods available on the web that make use of different monitoring tools but why not use a GCP native tool to accomplish the same? It does make it simpler after all.

As you may already know…

Java applications run inside a virtual machine called Java Virtual Machine (JVM). Java provides a set of tools, for management and networking of Java applications, called Java Management Extensions (JMX). So JMX basically enables monitoring of JVMs out of the box, all we need to do is enable it on a specific port that out agent can read and get all the good stuff like thread count, memory usage, garbage collection, open file descriptors, etc.

Deploying the Java App

If you already have the JMX remote enabled on your containerized app then you may move to the next section otherwise read on…

1. Build Docker image

Within the dockerfile, ensure that the Java command to run the application includes the system properties to enable JMX remoting.

For example, if your java command to start the app is:

java -jar /.app.jar

then you modify it to look like below:

java -Dcom.sun.management.jmxremote \
     -Dcom.sun.management.jmxremote.port=<JMX_PORT> \
     -Dcom.sun.management.jmxremote.ssl=false \
     -Dcom.sun.management.jmxremote.authenticate=false \
     -jar /.app.jar

Some things to note:

• Choose a valid port of your choice instead of JMX_PORT (Example: 9001).
• For this example, we disabled the SSL and authentication to JMX. Please enable and configure them if your use case requires it.

2. Deploy the container inside a GCE VM instance

Create a GCE VM instance running linux, make sure a service account is attached to the VM instance while creating it. Once the instance is created, install Docker on it. Deploy the container using docker run command ensuring the JMX port is listening on the container.

$ docker run ... <app specific options> ... --publish=<JMX_LOCAL_PORT>:<JMX_PORT>

• JMX_LOCAL_PORT is the port mapped on GCE VM.
• JMX_PORT is the same port that was set earlier within Dockerfile while creating the image.

Enable Stackdriver Monitoring

Now that the container is up and running with JMX listener enabled, we will install Stackdriver Monitoring agent and JVM plugin that will query the metrics from container and forward to Stackdriver Monitoring service.

1. Setting up correct IAM access on GCP

For the agent to be able to upload metrics to Stackdriver Monitoring, it must have necessary IAM roles assigned to it at the project level. Grant roles/monitoring.metricWriter to the GCE instance service account at project level.

You can read more on setting up access control for Stackdriver Monitoring here.

2. Install monitoring agent

Install the Stackdriver monitoring agent for linux.

$ curl -sSO https://dl.google.com/cloudagents/install-monitoring-agent.sh
$ sudo bash install-monitoring-agent.sh

You can read more on installing the Stackdriver monitoring agent here.

3. Add JVM Plugin

Enable Stackdriver JVM monitoring plugin by downloading and adding the configuration file to agent configuration.

$ (cd /opt/stackdriver/collectd/etc/collectd.d/ && sudo curl -O https://raw.githubusercontent.com/Stackdriver/stackdriver-agent-service-configs/master/etc/collectd.d/jvm-sun-hotspot.conf)

4. Update JMX Plugin Configuration

Edit the downloaded JMX plugin configuration file (jvm-sun-hotspot.conf) to replace JMX_PORT with JMX_LOCAL_PORT port number that was set earlier while running docker container.

5. Restart the monitoring agent

Restart the monitoring agent to let it pick up JVM plugin configuration.

$ sudo service stackdriver-agent restart

Verify on Stackdriver Monitoring Console

Now that the agent is setup to collect and export JVM metrics, it is time to verify the same on Stackdriver Monitoring console.

1. Select the Project Workspace

Go to Stackdriver Monitoring console and select the workspace of your project. If it is a new project then a new workspace will be created for this project by default, alternatively you can add your project to existing workspace.

Read more on Stackdriver workspaces here.

2. Select Java Virtual Machine Metric Resource

On the left hand panel, navigate through Resources > Java Virtual Machine

Stackdriver Monitoring console showing JVM resource selector.

3. View the different metrics

After you click on Java Virtual Machine and if everything was configured correctly, you should see your GCE instance name and the JVM metric charts.

Stackdriver JVM Monitoring window showing different collected metrics.

Not able to see metrics for your GCE Instance?

If you are unable to see your GCE VM Instance name on JVM monitoring console then follow the different suggestions in Stackdriver Monitoring agent troubleshooting guide.