The Resurrection of Software Testing
James A. Whittaker
In 2011, Alberto Savoia took the stage at the prestigious Google Test Automation Conference in full Grim Reaper regalia and declared that “test is dead.” The primary point being that testing no longer needed to be a separate role in the development lifecycle. Software processes and tools had matured to the point that macro quality concerns no longer required large dedicated teams to assure. Instead, testing had evolved into a collection of micro activities that could be spread over other roles, like program managers, developers and even end-users.
He was right. In the trailing months after his dramatic prediction, first Google and then Microsoft discontinued most job titles that had the word ‘test’ in them. Software Engineer in Test, Test Engineer and so forth became very rare and test as a standalone career path all but disappeared. Most other companies followed suit.
The demise of testing as a separate discipline was a long, slow decline from its heyday in the 80s and 90s. Back then, software was poorly understood. Programming languages and the compilers that turned them into executable code were unreliable, little more than advanced prototypes. Teams would write software and have no idea how it might behave when it was installed in a customer environment and exposed to customer input. It took a lot of testing to prevent expensive recalls and still there were expensive recalls.
The discipline of testing thrived in this target-rich environment and research into software testing was a hot topic. My own article on testing was one of the most downloaded articles of the decade in the 1990s. By the time to 2000s ended, tools, processes and training matured to the point that software released after this time was reliable out of the gate and recall-class bugs were few and far between.
This same process is going to have to play out with AI in a similar manner. In 2024 AI is as poorly understood as software was in 1994. Like the software of that time, AI is being trusted with many important tasks, making quality, safety and security concerns very real and present dangers.
In other words, software testing may well have died in 2011, but it’s mindset needs to be resurrected in 2024. The world needs the skills of testers more than ever and researchers need to step up to create similar tools, processes and training as they did for software so long ago. AI is a technology the world will depend on for decades to come. Getting the best quality assurance minds on it now is one of the highest technical imperatives of every company from Big Tech to do-it-yourself AI enthusiasts.
Will the real software testers please come forward?
