How to Monitor Your System Dependencies

Nicola Apicella
Jan 3 · 3 min read

In the past few years, the DevOps reports have shown how high-performance teams use monitoring to quickly detect errors, reduce downtime and costs.
Monitoring a complex application is a significant engineering endeavor in and of itself, but thanks to effort of cloud providers, developers can instrument their system with few clicks or configurations.

When it comes to your system health, monitoring your dependencies is just as important as monitoring your system. If your system is experiencing issues, your system dashboards and alarms should highlight if one of your dependency is having a bad time.
In the rest of the article I show a simple pattern to monitor your dependencies. The examples use CloudWatch Embedded Metric Format and golang but the underlying idea can be applied to other cloud platforms or languages.

What’s EMF?

CloudWatch Embedded Metric Format allows to automatically generate metrics from logs. As a developer you need to log the metrics as JSON to stdout and Cloud Watch takes care of publishing the metric on your behalf.
You can use one of the AWS client libraries for NodeJS, Python or Java to build and log an embedded metric structure. Sadly, at the moment AWS has not published a client library for Golang which means you need to create the JSON object yourself following the schema described in the specification. It’s relatively straightforward and not surprisingly a few community maintained projects do exactly that, for example aws-embedded-metrics-golang.

I find EMF very convenient but it’s usage is not strictly necessary to monitor your dependencies. You can publish the metric yourself if that’s what you want to do.

Instrumenting the client

The main idea is to instrument the client you use to call your service. Regardless of the language or the library you use, it’s almost certain that the client you use to call your dependencies allows to plug in custom logic. You can leverage it to log the EMF metric.
For example, assuming you use a golang http client, you can implement the RoundTripper interface:

The code implements the RoundTripper interface with the logic to log details about the http calls. In the example, we log the Duration and the result of the call (client error or server error ). Once I deployed the code (in my case to Lambda) and triggered it a few times, we can see the metrics published to CloudWatch:

cloud watch dashboard

Instrumenting a different client is very similar. For example, here we instrument the golang AWS SDK to log calls made to other AWS services:

Conclusions

With AWS CloudWatch you can easily set up monitoring for your AWS resources and applications. EMF is a very convenient way to plug custom metrics in your system. As I mentioned, the pattern is quite generic and you can easily port it in other languages or even other monitoring systems.

If you develop on AWS take also a look at AWS X-Ray. With AWS X-Ray you can visualize and analyze the health, performance, and availability of your applications in one location, and see an end-to-end view of requests as they travel through your application.

Follow me on Twitter to get new posts in your feed.
Credit for the cover image to Luke Chesser on Unsplash.

The Startup

Get smarter at building your thing. Join The Startup’s +794K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Nicola Apicella

Written by

Software dev engineer at Amazon. Golang, Java and container enthusiast. Love automation in general. Opinions are my own.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +794K followers.

Nicola Apicella

Written by

Software dev engineer at Amazon. Golang, Java and container enthusiast. Love automation in general. Opinions are my own.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +794K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store