Continuous testing but with irregular schedule

Testing a network or service availability continuously has the benefit of getting an alert as soon as something changes or breaks. Many networks and services are monitored with for example a cronjob running every hour. We pondered how fast we should test and what should the test interval be.

Our test cycle is almost half an hour. It’s not half an hour (1800 seconds). It’s almost. And here I explain why it is 1789 seconds.

The benefit of this “missing” 11 seconds is that the same test is performed periodically, but on different time. If a test cycle is started on midnight 00:00 and it keeps running, almost 24 hours later (48 times 1789 seconds), the test cycle is started again on 23:51. That’s 9 minutes difference during one day give or take a few seconds.

Our reasoning behind this is to catch things that might be scheduled to happen at specific times. For example a periodic automatic reboot or a configuration run is scheduled to happen on midnight 00:00 and it lasts for five minutes. What if during this window there’s something unexpected happening? If a particular test never hits this five minute window, you’d never know. However, when we run test cycle with 1789 seconds there’s a small change that the test sometime someday hits that five minute window and can catch the unexpected happening.

Magic number 1789?

Why exactly 1789? 1789 is a prime number. In our case a prime number is cool because it is only divisible with one and itself. With a bit of math it can be shown that it takes a test cycle exactly 1789 days (4 years and 10 months) to start over on same time (to hit that midnight 00:00:00). Every other test is ran on different time, be it just a second difference but a difference anyway.

But there’s no magic in 1789. Except that it’s a prime. Other primes close to 1800 are: 1783, 1787, 1801 and 1811. Or if you fancy almost one hour cycle (3600 seconds) there are primes like 3581, 3583, 3607 and 3613. We might reconsider our choice of prime number in the future, but now it is one seven eight nine.