Survey: AI’s Impact on Testing

People follow me down the hall of test conferences asking if and when AI will take over their manual testing, or automation job. No one knows, but perhaps together we know. Borrowing a method used by Nick Bostrom (author of SuperIntelligence) to figure out how long it will be before AI takes over the world, I simply surveyed folks in the space to get an idea.

Question: How soon do you believe AI/Machine-Learning will have a significant impact on manual testing?

Answer: 76% of people believe AI will have an impact on manual testing within the next 3 years.

Question: How soon do you believe AI/Machine-Learning will replace all manual testing?

Answer: Almost half of everyone thinks AI won’t replace manual testing. That said, the data seems to be a bit polarized as the next biggest vote was for AI to replace manual testing as soon as 2020, with votes trailing off to 2040. And, of course, no one thinks AI will replace manual testing in the next couple of years. A peek at the types of folks behind the numbers shows that half of manual testers believe AI will replace their job in 2020. Automation Testers on the other hand made up almost all the ‘Never’ votes. Perhaps manual testers are hopeful, or know their job is ripe for machines. Perhaps Automation engineers know how difficult the effort would actually be…

Andrew Ng, perhaps the most respected person in Applied AI these days says “Pretty much anything that a normal person can do in <1 sec, we can now automate with AI.” How many manual testing steps or verifications can be done in about a second?

Question: How soon do you believe AI/Machine-Learning will have a significant impact on Automated testing?

Answer: Everyone believes AI will have a significant impact on automated testing and 78% think it will happen between 2017 and 2020. Test automation engineers better start learning AI or it will probably swamp them.

Question: How soon do you believe AI/Machine-Learning will replace Automated testing?

Answer: 50% of of folks think the job of Test Automation engineer will be replaced in the next 5 to 10 years. Not great odds to keep making career of it :) . A full 86% of folks believe automation jobs will be replaced by AI at some point. I wonder if it will be the automation engineers themselves that build the AI that kills their job — or will it be someone else?

In summary, it looks like these AI bots are actually going to be coming soon. They’ll start helping our manual and automation testing tasks in the near term, and likely replace many of these jobs sometime in the next 10 years.

Survey Details: Granted there aren’t a lot of folks working or thinking about this space today, so the survey was small. It was sent out to about 200 people, and 50 responded. The survey was sent out to folks who had signed up for the AI for Software Testing Association.

Here is a peek at the type of folks that answered the survey.

Thanks to everyone who took the survey. If you want to add your voice to the collective wisdom, please AI for Software Testing Survey yourself!

— Jason Arbon