Linux: inexplicably high IO

When things get stuck without apparent reason


Problem: random notifications from monitoring system (zabbix) about high IO on a Debian Squeeze server.

Steps undertaken to find the cause of the problem involved two utilities: iotop and iostat

The first allows you to analyze IO by process the second by device so after running them both on separate shells:

iotop -od 2
iostat -xm 2

I was able to spot the problem: PostgreSQL autovacuum

Adjusting vacuum_cost_delay and vacuum_cost_limit in postgresql.conf solved the issue.

You can set those values per database or even run vacuum manually once in a while depending on how often and many records are deleted/updated in your DB (aka bloat space).

A good explanation by Heroku: https://devcenter.heroku.com/articles/heroku-postgres-database-tuning

Email me when Dawid Pogorzelski publishes or recommends stories