Getting Started with AI for Testing
Preparing the Human Brain for the Rise of the Machines
Last week I had the honor of giving the opening keynote at StarWest 2017 and PNSQC 2018 on the topic of Artificial Intelligence for Software Testing (AIST). The talk, entitled “Rise of the Machines: Can Artificial Intelligence Terminate Manual Testing?”, created quite a bit of buzz at the conferences. You can check out the video below if you missed it. Aside from the “Do you think AI will take over the world?” type questions, one of the most popular things I was asked about after the talk was: “How do I get started with AIST?”
The short answer to that question was to point delegates to general resources on AI and machine learning. However, it is important to recognize that such a question is loaded and therefore deserves a more detailed answer. First let’s start by revisiting the definition of AIST.
Artificial Intelligence for Software Testing (AIST) is an emerging field aimed at the development of AI systems to test software, methods to test AI systems, and ultimately designing software that is capable of self-testing and self-healing.
— AISTA (www.aitesting.org)
Examining the definition we can see that how to start depends on two things: what you are trying to achieve and where it lies in the intersection of using AI to test software, testing AI-based systems, and building software with intrinsic self-testing and self-healing capabilities. A safe place to start is pursuing knowledge and understanding in the areas of artificial intelligence, software testing, and self-managing systems.
Artificial Intelligence
AI is not new. It has been around since the 1950’s and knowing its historical progression and major advances over the years will help you to position your work in the AI space. Creating the vision for your tool, method, or approach, acquiring the right talent, aligning team members, developing the solution, and communicating the value of your work all start with positioning.
The field of AI is broad and so I recommend that you gain an understanding of that breadth before you do a deep dive into specific areas. Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig is a great reference book for all things AI. When you are ready to go deeper, you’ll want to get your hands on more specialized books. For example, Deep Learning by Ian Goodfellow presents a broad range of topics on hierarchical learning architectures. In terms of freely available self-paced multimedia learning, check out the video courses on YouTube from Andrew Ng (Stanford University) and Lex Fridman (MIT).
Software Testing
For as long as there has been software, there has been software testing. Despite the rise of other approaches such as formal verification and model checking, testing remains the primary means of validation and verification (V&V) used in the software industry today. Like AI, the field of software testing is extremely broad, but chances are that if you are looking into AIST you may already have a background in testing. However, if you don’t and are looking for online resources on software testing, check out this post on 45 Websites Every Software Tester Should Bookmark in 2017. For books, some old favorites of mine are James Whittaker’s series on How to Break Software, Lee Copeland’s Practitioner’s Guide to Software Test Design, and Cem Kaner’s Testing Computer Software.
Self-Managing Systems
In 2001, software giants IBM, Microsoft, and HP all set out on their own initiatives to define the next generation of computing systems. Such systems were envisioned to be self-managing, and able to adapt to changes in order to meet the needs of its users. IBM coined the term autonomic computing and modeled their approach against the human autonomic nervous system (ANS). The ANS regulates critical bodily functions such as homeostasis without conscious involvement. Applying this idea to computing, systems should adapt to their environment through self-configuration, self-optimization, self-protection and self-healing. What? No testing? Although the giants weren’t talking about it, in order for autonomic computing to work in practice, self-testing needs to be an inherent part of the system.
It was for this reason that I decided to pursue this research area during my graduate studies from 2005–2009. Unfortunately to date there is still a general lack of research in the area of self-testable autonomic software. However, the recent advances in AI/ML, Cloud Computing, DevOps, and Microservices, have improved the potential for self-testing to provide a formidable solution to the grand challenges of software testing. We’ll be posting some self-testing and autonomic computing foundation papers on AISTA’s website for those interested in learning to how self-testing systems can be designed and developed.
Other AIST Presentations
In addition to the Rise of the Machines keynote, StarWest this year featured two other talks on AIST. Jason Arbon gave an updated version of his talk “AI and Machine Learning for Testers”, originally presented earlier this year at StarEast. Paul Merrill from Beaufort Fairmont presented a concurrent session on “Machine Learning and How it Affects Testers”.
What’s Next?
As much as you can read books and watch videos about AIST, there is nothing more powerful than getting hands to keyboard. In a future post, we’ll cover some tools that you can play with to get a feel for AI and ML.
Tariq King — Co-Chair (AISTA)
