MySQL’s SRE Golden Signals

Part of the How to Monitor the SRE Golden Signals Series

Steve Mushero
Nov 10, 2017 · 5 min read
Image for post
Image for post

MySQL is a key element of many, if not most, modern online systems, so its performance is of the utmost interest, including and especially if it’s used as-a-service such as AWS RDS.

The gold standard on MySQL monitoring is Vivid Cortex and their SaaS platform’s ability to get and analyze all these signals and much more. But if you don’t use Vivid Cortex and/or want to include these signals in your own core / integrated monitoring system, here is how.

All these methods require you to have a good MySQL connection, as there is no good way to get the signals without it (though you can get some stuff from the logs, and Vivid Cortex uses a protocol analyzer). Please make sure your monitoring user has very limited permissions.

Getting MySQL’s Golden Signals varies in complexity based on the version you are running, and if you are running MySQL yourself or using RDS. I apologize in advance for all the pieces & parts below.

A tricky thing, this MySQL

Note that MySQL Latency is fundamentally hard to get accurately, and there are tricky issues with some of the data, including how the Performance Schema works. The methods below outline a couple ways; there are several others, all of which have pros and cons.

None are great, but they give general latency you can use to know how things are doing. Be aware the methods below will not capture prepared statements nor stored procedures (both of which are probably rare for most people, depending on your programming language, app server, and framework).

What about RDS?

Everything below should apply to AWS MySQL RDS as long as the Performance Schema is on — you must turn this on via your AWS RDS Instance Parameter Group and then sadly restart your DB.

Mapping our signals to MySQL, we see:

SELECT (avg_timer_wait)/1e9 AS avg_latency_ms FROM
performance_schema.events_statements_summary_global_by_event_name
WHERE event_name = ‘statement/sql/select’;

TRUNCATE TABLE
performance_schema.events_statements_summary_global_by_event_name ;

As you can see, MySQL is challenging and there are many more items you can read, both to get the above signals, and to get a broader sense of DB health for alerting, tuning, and so on.

Next Service: Linux Servers

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store