There are Lies, Damn Lies, and Unit Tests

While Mark Twain famously gave issue with statistics — it was only because Unit Tests had not been invented yet. Several years ago I was introduced to Test-Driven Development. A process where developers create tests to check their code as it is being written. Since then, it has been the source of many coding nightmares.

As Director of Engineering, I have seen developers use the Unit Test as a defense against poorly written code. Then upon inspection realize the test only accounted for a fraction of the requirements. It seems that as test fail during development, the hurried Engineer would remove the offending test instead of correcting the problem with the code. Sadly, this is not a one-time occurrence.

In other scenarios, developers will leave out critical tests as the task of creating them takes time away from feature development. This usually occurs when teams are rushed to complete a project and working weekends or evenings to do so.

The issues described above are not problems with Unit Tests. Instead, they represent problems with management. However, there is one real issue with Unit Testing. That is, the tests are created by the person who is writing the code and therefore creates a conflict of interest. The solution is to have the Unit Tests created by one team member then coded by another.

Dividing the responsibility will improve the effectiveness of the Unit Tests and make management look good. Maybe. The solution is almost never that simple. Instead, it requires one to review the Unit Test as a means to measure it against the requirement. It also requires a developer trained on how to create an effective test.

Some teams covet workmanship and will deliver stellar results regardless of methodology. Others will only do the minimum. The difference is not the team, it is the expectations and guidance given by management. Unit Tests just reveal the quality of technical management.

As a manager, most problems with development are my fault. If the Unit Tests are worthless, it is because I set a precedence to allow it. Not by direct order but by a lack of one. That is, management must set the expectation and provide the training required to create worthwhile Unit Tests along with all other aspects of quality requirements.

Good managers set clear expectations. Great managers ensure those expectations are understood. Just introducing Test-Driven Development will not improve quality. Instead, quality is the result of multiple processes implemented with a team dedicated to following them.

Leading the developers needs to be a person who teaches more than demands. He or she must be able to clearly communicate the requirements and assist the team to best fulfill them. After more than 100 projects, those that were great had management supporting the team with communication, realistic expectation, and appropriate training. Those that failed were missing one or more of this items.

If done properly, Unit Tests can help the team prove the code meets all requirements. Add in automated tests to ensure code quality and proper functionality and the team is much improved. Expand this to include a formal peer-to-peer code review system and the team is better still. However, it all centers on management doing their job to properly support the team.

When done wrong, Unit Tests are the equivalent of lies. Management and most of the team will believe things are good when in fact there are major issues somewhere in the code. However, it is management’s job to ensure the tests are being done as required. It is also their job to ensure the code meets the standards. Anything less and the team and project suffers.