Testing — Automated Testing

I’m going to talk about why your automated tests should be handled with more care than your production code.

First of all, what is the purpose of testing.

Testing, simply put, is a means to build confidence that the software you have built will perform as expected and will have few bugs and fewer regressions of previously fixed bugs.

The impact of confidence on a piece of software determines whether it is released, delayed or canned, and this confidence broadly correlates to the manifestation of live issues and defects.

Suffice to say, testing is one of the most valuable activities in software development.

Testing however is seen as a cost. You see, in Software Development you can see new code being created and new features being created. It’s really easy to assume that writing production code is valuable (this is an incorrect assumption).

Let’s talk about what value is briefly, specifically in the field of Software.

Value is working Software which has a positive impact on the goal that the Software was created for.

More often than not, value is Software which is sold or generates revenue through licence fees. Ultimately it generates money. The nature of Software is that once it is written it can be copied and sold an unlimited number of times for very little cost (packaging and distribution is about all it costs once built, and even then that is negligible in the digital age).

Therefore because writing production code directly results in new features it is easy to associate it with being valuable.

Testing doesn’t create new features. Testing causes delays, it stops some features from being released. Because of this testing has often been seen as an unwelcome companion, an evil that is just required and always get’s in the way.

I want to pause for a moment and I want you to consider the following statement. Once you’ve read it, please stop reading for a few seconds to really let it sink in.

*This scenario is purely fictional* Imagine that your bank is releasing a small change to it’s payment processor, it’s meant to make processing payments significantly faster. It has a defect which deducts 10 times the amount it’s meant to deduct every time a transaction occurs and credits a tenth of the amount it is meant to credit.

Have you stopped and thought about the impact of that statement, what that would mean for you? Can you imagine filling up your car with £20 of fuel and then having £200 deducted from your account? Your weekly shop which usually costs £80 now deducts £800 from your account.

Now let’s assume that there are a series of tests which could have been easily written to test that when someone credits different varying amounts of money then that money is actually credited and that if someone deducts varying amounts of money then the actual amounts of money requiring to be deducted are actually deducted. It’s simple test, a simple test which represents immense value to the 100,000’s to 1,000,000’s of people who bank with that provider.

Simply put, the cost of not having those tests is so astronomically higher than the cost of implementing them in the first place.

I can’t imagine waking up one morning after a routine direct debit goes out only to find that my account is now empty. I can’t fuel my car, I can’t get around, I can’t do anything.

This is a money focused example, but let’s assume it’s a bug in the life support software used in hospitals, or in the software which records prescriptions or a bug in the alarm system in your office. I could go on, and I’m sure that you can imagine the consequences.

Mistakes will always happen, it’s natural and people are not perfect. However, testing is something that can prevent the majority of these issues from recurring and in many cases prevent them from occurring in the first place.

What we have these days are vague attempts at automated tests which are hacky, not entirely stable and still get in the way, they fail without reason, we don’t trust them, we don’t believe them when they pass and we can discredit them when they fail. Why? Because we are still caught up in the belief that testing is a second rate citizen rather than putting it where it should be, front line and centre of everything. It should be treat with more care and attention than the live code, it should be beautiful code, it should be easy to change, easy to add to, easy to remove from and ultimately be the single most trusted part of the entire process. It should be respected and, every time it stops a release it should be applauded, why? Because it has just saved you from a PR disaster damaging your business, your reputation and potentially putting lives at risk.

Treat your automated tests better than your treat your production code. Messy production code slows you down when adding new features. Messy test code means you release defects and bugs that you should have solved long ago. Messy test code won’t slow down your ability to release (you’ll just side step it and work around it), instead it will do something far worse, it will destroy your product, your market share and in the best case scenario send your customers to your competitors.