Although it feels like I’ve been engaged in public speaking in the software industry for a while, looking back I realize that it actually hasn’t been that long. It was seven years ago when I gave my first industry talk at STARWEST 2016 sharing my thoughts on why it seemed that software testing as a discipline was lost, and the perhaps fostering a culture of innovation was a way to bring back testing!

Prior to speaking at STARWEST, I had spoken at academic conferences where most of the focus was on new and novel quality engineering and testing approaches. However, a lot of the academic research was focused on the conceptual or theoretical, with less focus on practical aspects of software testing and quality engineering. Fast forward a few years later and I was embedded in an industry setting with many practical applications of testing, but little to no innovation. It made perfect sense that my messaging to the community back then was around bridging the gap between these two worlds.

It was after giving a few keynotes in the US that I started to get requests to speak around the world, and I have been blessed to have spoken in many locations that I never thought I would visit including Budapest, Kuala Lumpur, Iasi, Chisinau, Tokyo, Dublin, Lisbon and more. However, this post is not about these beautiful cities. Nor is it about innovation in software testing. In fact, it’s not even about testing at all. It’s about community, and in particular, a specific community that I found to be so unique that it’s filled with unicorns :)

A Unique Invitation

For me this all started with an invitation to give a keynote at the Agile Testing Days USA conference which was set to be held in Chicago in the summer of 2020. To be honest, I didn’t think much of it when the invitation first came in. I just glanced at the opportunity and it looked legit, so I said yes and moved on. However, as you probably already noticed, the conference date itself would present a major challenge since that would be the year that we would all go into a global lockdown due to the COVID-19 pandemic. Regarding the conference, the organizers contacted me and said that the conference was being postponed for a year, and that they would contact me again as the new date got closer.

Ok so no surprise, the event was postponed! What I found interesting though was that while most other conferences shifted to a purely online format, or some events were completely canceled, this one seemed to just keep the program in tact and just move the date. This stuck out to me as quite unique, especially when a whole year later, history would repeat itself. Conferences in general were still suffering from the impacts of the pandemic in 2021, but ATD USA insisted on keeping all the speakers on the program if they wished to remain. We were given an option to update our talks and the next new date was set for May 2022. For me this was an early sign that this was a community that not only valued the speakers, but valued their word, in the form of the agreements they had made with all of the speakers two years prior.

A Unique Experience

Once the event actually came around, I was pleasantly surprised. I mean, I have been a part of many conferences, and some amazing software conference communities including STAR, Agile DevOps, PNSQC, Quest for Quality, CAST, and others. However, I would soon come to find out that there is something quite different about ATD.

Unicorns!

So there’s a backstory around ATD unicorns which you can read about in this article: The Story of the Agile Testing Days Unicorns. And yes, there were literally unicorns all around the conference venue, which is actually pretty useful when you’re walking around wondering if you’re headed the right direction. If you bump into a unicorn, the chances are you’re good! However, my main point here isn’t about the unicorn theme they have going on. It’s about the people, those that organized the conference, those that spoke, and those that attended. I met so many unique people and formed lasting friendships. There was definitely a vibe of appreciating the individual personalities and a feeling that you can just be yourself at the conference because once you are there, you are accepted.

More Than Just Talks!

Let’s face it, you are going to a conference to learn. Many a time, the focus on how you learn and grow at conferences is all about the talks. Don’t get me wrong, ATD had a great program that included keynotes, workshops, and track talks. However, as a former professor I always reminded folks that you learn best when you are engaged in learning. What was really cool about the ATD experience were that there were a host of bonus and social sessions blended into the program. Everything from mentoring and coaching, to mindfulness and quiet time, or combo sessions where workshops and talks are mixed together.

The short way to sum it up was that the way the program is laid out made the conference super fun to attend. And why wouldn’t it be, there was literally dinners and food every day and night, a costume party, game nights. It’s like the only words that give it justice are that it was completely off the chain!

Little did I know that this entire vibe has existed for quite some time as part of the ATD events that happen in Germany and that for the last few years prior to COVID, they had just kicked off bringing this experience to the US. I found this out at the speaker dinner. Everyone was telling me that this is even bigger at ATD in Potsdam, Germany, and that there’s another event ATD Open Air that happens on the waterfront in Cologne, Germany. So of course, being my authentic self, I told my table straight that being from the islands, I would certainly love to give a keynote on the waterfront at ATD Open Air… and furthermore, if any of the speakers dropped or couldn’t make it to ATD 2022 in Potsdam, to call me first.

What I didn’t realize was that at my table were several of the organizers, and they immediately told the beloved founder and father of ATD, Jose (Pepe) Diaz, what I said. In that moment, Pepe looked at me and said you’re in for ATD Open Air. Furthermore, a few months later I got a message that one of the speakers couldn’t make it to Potsdam, and again I was in! And that was it, I spoke at both ATD USA 2022 in Chicago, and ATD 2022 in Potsdam all because I was in a safe space to be myself and speak my mind, around folks who were listening. This is how I ended up sharing the stage with a unicorn, twice!

A Unique Ask

Lately I’ve been super busy settling into my role at EPAM, and growing there like crazy, while balancing speaking at internal and external events, publishing articles, and serving on multiple conference boards and program committees in industry and academia, and hopefully being a good father and husband at the same time (you’d have to ask the wife and kids how I’m really doing!). What’s my point here. Well it’s that my time is valuable. And with that ever so valuable time I took the time to write about this particular series of conferences. But why? Well, in a nutshell it is a very different experience and each of the two times I have been to an ATD conference I left with some serious withdrawal, which is something I hadn’t felt before at a conference. So my ask is that if you’ve never checked out one of these events before, do yourself a favor and attend one. You won’t regret it!

In May I’ll be at ATD USA 2023 giving a brand new tutorial on Breaking into AI and Machine Learning, and then in June at ATD Open Air in Cologne, Germany giving a keynote. I’m also serving on the program committee for ATD 2023, which will again be in Potsdam so I also encourage you to submit a proposal so that perhaps you too can share the stage with a unicorn! Here are all the respective links. And no, I don’t get paid to promote this or any other conference for that matter. I just value and appreciate professional software communities and this happens to be an awesome one worth sharing. Not to take away from the others, but definitely make some room to add this one to your list for professional development and training.

Registration Link for ATD USA 2023

Registration Link for ATD Open Air 2023

Paper Submission Link for ATD 2023 (Closes March 19th)

Metaverse: a network of 3D virtual worlds focused on social connection and often described as a hypothetical iteration of the Internet as a single, universal, simulated world that is facilitated by virtual and augmented reality technologies.

What is the Metaverse really?

Although this article began with a fancy definition, no one really agrees on what the metaverse is or isn’t. Conceptually, it’s not hard to grasp the metaverse as an online 3D cyberspace connecting users across the globe in various aspects of their lives. Think of multiple computing devices and platforms connected together via the Internet, allowing each user’s avatar or character to navigate through a virtual space and become immersed in different experiences everywhere from virtual hangouts to sports games, bar mitzvah’s, weddings, and more. In reality though, the metaverse doesn’t exist, well at least not yet.

The idea of the metaverse was first developed in the science-fiction novel Snow Crash by Neal Stephenson. Stephenson had a vision of a virtual-reality (VR) based Internet evolving into a world that resembled a massively multiplayer online (MMO) game. In it, the metaverse would be populated by user-controlled as well as system daemons running in the background. While the idea of a metaverse was once fiction, it is looking more and more likely that it will soon be a reality. As you may know, the social networking giant Facebook has gone as far changing its name to Meta to reflect their belief that the metaverse is the next evolution of social connections, and are committed to bringing the metaverse to life.

Where Can I Experience the Metaverse?

Let’s take a look at some instances where the global coronavirus pandemic pushed quite a few metaverse-like experiences to the forefront of our personal and professional lives. Here, I’ll share a few of my own experiences and encourage you to chime in with any others…

Gather.Town

In the Fall of 2020, I was getting ready to log into the StarWest testing conference. As a testing nerd, this was nothing new for me. After all, I’d attended this and several similar conferences over the last decade or so. However, this particular year, the conference had to be held virtually because of global shutdowns due to the pandemic. StarWest has always had a virtual component for folks who could not attend in-person to be able to watch the keynote sessions for free. However, upon logging in, I wasn’t greeted with the usual interface with web links and embedded video streams. Instead, I was transported into a 2D virtual space called gather.town.

The floor plan for the virtual space was quite similar to the floor plan for the physical conference and, as an attendee, I could select an 8-bit avatar and now instead of jumping from link to link, it was possible to walk my avatar from one session to another, and even bump into other folks for a quick catch-up along the way. As you walk up to another avatar or group, your video feed automatically appears and you can immediately engage in the conversation. My buddy Jason Arbon and I, even held our “Ask Us Anything” couch session virtually in Gather. The above screenshot of me waiting for Jason to show up for our session is living proof that some things probably won’t change, even in the metaverse :)

It was also cool to walk up on the main stage with my avatar to see myself and the other keynote speakers. Although the experience was completely retro with the 2D space and 8-bit characters, this gave me a good glimpse of what the metaverse could be like.

Virtual and Augmented Reality

Once the pandemic hit, I found myself spending a lot more time using my Oculus Quest 2. When it was time for the olympics, I wondered if there was anything available in VR that could get me in close on the action. Thankfully, the Tokyo Olympics presented some live and full event replays on the Quest 2 for pay-TV subscribers in the United States. Set up was easy, as I just had to download an app is called NBC Olympics VR by Xfinity and sign in with my account. And there I was watching the opening and closing ceremonies, and my favorite Track & Field and other sporting events in 3D VR. I even set up a watch party with a few of my Oculus Friends who were also subscribers.

I was also able to get my wife and kids involved in trying out various Quest 2 experiences, ranging from playing Beat Saber to realistic simulations like the Jurassic World: Apatosaurus VR experience. If you’re trying to get an idea of what the metaverse may be like, I’d suggest trying out some of the VR or augmented reality (AR) headsets as the metaverse will likely be powered by these technologies.

Video Games

You can already find elements of the metaverse in popular video games like Second Life, World of Warcraft, and Fortnite. Because of the services they now offer, along with the emphasis on 3D VR, video games are perhaps the closest you can currently get to a metaverse experience. My kids are huge fans of video games like Roblox and Fortnite, which now host virtual events like concerts and meetups. That’s right, gamers aren’t just playing games anymore, they are using gaming platforms for all aspects of their lives in cyberspace. I just had to check out this stuff for myself and so in Fortnite, along with 12.3 million other players, I took part in Travis Scott’s virtual in-game music tour.

It is important to note that many of these “metaverse” elements aren’t new to the gaming world. For instance, it’s been 20 years since a wedding was held in the game Second Life and, since 1996, player avatars have been roaming around the 32-bit meadows of Furcadia, one of the oldest massively multiplayer online role playing games (MMORPGs).

To bring things full circle, a group of developers came up with a novel way to host a DevOps conference within the popular Nintendo game Animal Crossing: New Horizons.

Why We Can’t Wait to Automate the Metaverse?

As the buzz and hype around the metaverse continues, many are raising concerns about the potential risks in an environment where the boundaries between the physical and virtual worlds are blurred. These risks and concerns associated with the metaverse are a large part of the motivation for why we can’t wait to begin investigating automated metaverse testing techniques. Here’s a quick list of some of the key challenges:

Risks and Quality Concerns

• Identity and Reputation— Ensuring an avatar is who they say they are in the metaverse. Along with identity authentication and verification comes the need to protect users in the metaverse against impersonation and activities that may harm their reputation.
• Ownership and Property — People have already started to purchase digital assets like NFT art and virtual properties. The metaverse raises two key questions: how can ownership rights be grated for creators of digital assets? and how can ownership be verified?
• Theft and Fraud — As banking, currency, payment systems, and other forms of commerce migrate to the metaverse, we are likely to see an increase in attacks that result in stealing, scamming and other types of crimes for financial gain.
• Data — Folks are already speculating that the metaverse is yet another ploy by the tech giants to get more data. After all, control of data can help organizations control markets. Whether or not you believe there is a conspiracy to get your data, there’s no doubt that nowadays the abuse of data and misinformation is widespread and needs much attention.
• Invisible Avatar Eavesdropping — Malicious actors may figure out how to make their presences in the metaverse undetectable, and use such an exploit to invisibly join meetings and eavesdrop on private conversations.
• Harassment and Personal Safety — Did you know that a woman was sexually harassed on Meta’s VR social media platform? Harassment and personal safety is not just a physical thing, it can be verbal and now a virtual experience that should be prevented.
• Legislation and Jurisdiction — In a virtual space that is accessible to anyone across the world, it is essential to be able to identify any boundaries of that space and put rules into place to make sure it is safe and secure for everyone.
• User Experience — The ability of the metaverse to become a space where people can connect, form meaningful relationships and become fully immersed in its surroundings, is directly correlated with its visual and graphical fidelity along with other aspects of look and feel. Visual, audio, performance, accessibility and other issues are therefore likely to detract from the overall metaverse user experience.

Promising Directions

Even with such a tall order of risks, concerns, and testing challenges, there’s much more behind our motivation to automate testing of the metaverse. AI and machine learning (ML) has been helping us to design and develop new, more robust, and resilient, automated testing tools, products, and frameworks. The engineering team at test.ai has pioneered many of these innovations and continue to leverage AI to solve some of the hardest testing problems we have today. Here are some examples of where much of our work to date directly correlates and translates into automated testing of metaverse-like experiences.

AI for Testing Digital Avatars

Computer vision unlocks a realm of possibilities for test automation. Bots can be trained to recognize and interact with visual elements, just like humans do. We applied our AI-driven testing technology to the validation of digital personas like those developed by SoulMachines. The engineering team at test.ai trained ML object detection classifiers to recognize scenarios like when the digital person was speaking, waiting for a response, smiling, serious, or confused. Leveraging AI, we developed automated tests to validate conversation-based interactions with the digital avatars on SoulMachines. This included two forms of input actions, one using the on-screen textual chat window, and the other by tapping into the video stream to ‘trick’ the bots into thinking that pre-recorded videos were live interactions with humans. A test engineer could therefore pre-record video questions or responses for the digital person, and the automation could check that the avatar had an appropriate response or reaction.

Even in the early stages of their development, our bots were able to learn how to recognize interactions that were relevant to the application context, but ignore others. Let’s look at a concrete example. The figure below shows the results of running a test where the goal was to validate that the bot was able to respond appropriately to the gesture of smiling. We all know that smiles are contagious and it’s very hard to resist smiling back at someone who smiles at you, and so we wanted to test this visual aspect of the bot interactions. The automation therefore launched the digital person, tapped into the live video stream, and showed the digital avatar a video stream of one of our engineers who after a few moments started to smile. The automation then checked to see the avatar’s response to smiling, and here was the result.

As shown in the figure, if you compare the bot’s current observation of the avatar with the prior observation, you will notice there are two differences. Firstly, the avatar’s eyes are closed at the moment of the capture as indicated by the blue boxes, and it is also smiling broadly enough that its teeth are now visible (red boxes). However, the difference mask generated by our platform only reports one difference — the smile. Can you guess why? Perhaps a bug in the test.ai platform? No, quite the contrary. Here the test.ai bots have learned that blinking is part of the regular animation cycle of the digital avatar. It’s not just trained on a single image, but actually trained on videos of the avatars, which include regular movements. With those animations now recognized as part of the ground truth, the bot distinguishes that the big smile is a deviation from the norm, and so produces an image difference mask highlighting that change and that change only. Just like a human would, AI can notice that the avatar smiled back in response to someone smiling at it, and knows that the eyes blinking at the moment of screen capture is just coincidental.

Want to play around with AI’s ability to detect emotions for yourself? Or maybe even ON yourself? Head over to https://cloud.google.com/vision and scroll down to the ‘Try It Yourself’ widget and upload a photo of yourself or someone else’s face. Under the ‘Faces’ tab of the output, you’ll see how the AI classifies the facial expression in the picture.

Pretty cool huh? At last year’s Conference for the Association for Software Testing (CAST), I gave a workshop on AI for software testing and had the participants play around with the Cloud Vision API. Testing guru Paul Holland took part in the workshop, and as a group we found his result somewhat amusing :) Check it out…

Well what do you think? Does Paul look joyful? Some folks detected a bit of a smile in Paul’s photo but Paul himself wasn’t so convinced :) On the flip side, the bot was 92% confident in its classification of Paul’s emotions!

AI for Testing Video Games

AI plays games, why not test them too?”

This was the tag line for a chapter I wrote in the 2021–22 World Report on the State of AI Applied to Quality Engineering. In fact, much of my day-to-day focus for 2021 was on R&D related to the use of AI for testing modern video games like League of Legends, Fortnite, Call of Duty, and more. Like a good software engineer, I’m going to apply the DRY (Do Not Repeat Yourself) principle to this article, and instead, point you to some of the resources we developed and shared last year on the subject.

Here’s a link to the aforementioned chapter entitled Transforming Game Testing with AI-Driven Automation, which also includes an audio book version for those who’d rather listen :) There were also a number of presentations, but one of my personal favorites was the talk I did for the AutomationGuild conference. Our friend and founder of TestGuild Conferences, Joe Colantonio, has granted special permission to share the presentation video recording as part of this post. Enjoy! Thanks Joe!

After watching the presentation is it should be clear why I can’t wait to automate the metaverse. These techniques certainly have a wow factor and I remember some days just spinning up our tech on some of the latest games, just to watch the bots explore and test the games. Finally, using the powers of AI for the good of testing and also getting paid for it :). Definitely a dream job for a testing nerd like me.

AI for Testing Virtual and Augmented Reality

Of course we couldn’t just stop with 2D and 3D game testing. Huge Kudos to Dionny Santiago for this next one. Dionny leads our core platform team at test.ai and recently extended that platform to handle Virtual Reality (VR) environments. Using a similar approach to how we interact with gaming consoles, controllers, and video streaming devices, we integrated tools and drivers that allow us to manipulate and observe the input-output functions of a VR headset. Once we could control inputs and observe outputs in VR, it was just a matter of tying that API into test.ai’s Game Testing Cortex, an ML brain that combines supervised ML and goal-based reinforcement learning to test video games in real-time.

The final result is that engineers or external programs can make calls to the VR API controller and leverage it to build and execute tests in that environment. Take a look at it in action as we programmatically modify the yaw causing the headset to rotate within the virtual space.

AI for Testing at Scale

AI brings some other key advantages to the test automation world, including accelerated test coverage, reuse of tests across applications, domains, and platforms, among others. When it comes to fruition, the metaverse will undoubtedly be a big place, the bounds of which may not even be fully comprehensible to the human mind. Machines on the other hand, are good at handling large amounts of data and processes at scale, and AI and ML are paving the way forward. This happens to be one of Jason Arbon’s favorite topics, and if you are curious to learn more about this subject, including the future of testing at scale, check out this webinar with Jason and Kevin Pyles. Our DevOps architect Patrick Alt also gave a technical presentation on using Kubernetes, AI, and other techniques for Deploying a Large Scale Army of AI-Driven Testing Bots. After all, our mission at test.ai is to test the worlds apps… how’s that for a testing at scale problem? :)

AI for Non-Functional Testing

AI for performance, accessibility, usability, trustworthiness and security testing are all topics that we have covered over the last year. AI is slowly taking over the testing space. It started with AI being incorporated into specific toolsets and programming languages, but now we’re seeing AI stretching beyond functionality into cross-cutting concerns. You can expect this type of growth until there is eventually a full-stack replacement for test automation using AI. It’s quite exciting to watch and as ideas like the metaverse become a reality, such AI-powered full-stack test automation will prove to be necessary to keep up with the pace of development.

If you’re interested in learning more about each of the above dimensions of testing with AI, check out my e-book on AI-Driven Testing. It’s free if you are an O’Reilly Learning Platform subscriber, and if not, for a limited time, Keysight Technologies has made the book freely accessibly via the following sign up and download page.

AI for Testing AI

So there’s one aspect of the metaverse that I’ve purposely left until last because its both particularly interesting and challenging. Human controlled avatars aren’t likely to be the only entities you can interact with in the metaverse. Yes, there is a great probability that there will be some equivalent to non-player characters within the metaverse. In gaming, a non-player character (NPC), is any character that is not controlled by a human player. For example, in one of my favorite games FIFA soccer, there are several NPC’s ranging from the referee, goalkeeper, fans in the stadium, among others.

To enhance the experience, NPCs in the metaverse may themselves incorporate AI and ML so that they can adapt and evolve to changing user needs and conditions in the virtual environment. The open question therefore is how do we test these AI-controlled NPC’s in the metaverse. Well, my answer couldn’t possibly get anymore meta: use AI to test AI.

Now that I’m 100% confident you’ve now seen the true level of my crazy…

Let me explain :)

AI and ML are enabling highly dynamic behaviors in software today. A static approach to testing is likely to be inadequate in validating these self-adaptive and self-evolving capabilities. As a result, many of the current testing techniques and approaches will only take you so far with such a system. Automation itself will need to evolve and the testing framework itself will need to be adaptive so that it can monitor and regulate the system under test. Such a system would be trained, managed, and monitored by humans, while facilitating gathering and coalescing the vast amounts of data on metaverse interactions and transactions. This would allow us to apply the rules, laws, goals, and policies defined for the virtual space. Furthermore, testing will need to become an inherent part of the NPC’s and other aspects of the metaverse for the initiative to be successful. It is the only foreseeable way to ensure that all the risks and quality concerns mentioned in this article can be addressed. A holistic testing strategy for the metaverse would involve validation and verification activities across multiple disciplines that are triggered both offline, prior to deployment, as well as on-line, continuously, after deployment.

As we’ve been building out the next generation of automated testing at test.ai, I have noticed that there is a need to promote and share our practical experiences and strategies for testing AI-based systems. Furthermore, beyond testing, an end-to-end quality engineering approach should chart the path forward with AI. I recently had the privilege of working with former O’Reilly AI/ML editor, Rebecca Novak, designing a course on Quality Engineering for AI and Machine Learning. A short version course is available for individual subscribers on O’Reilly’s Learning Platform, or if you’d like to have your engineering team go through it, you can register your team for an extended version of the course.

Wrap Up

The metaverse isn’t here yet, but we can’t wait to automate it. Firstly, it’s too risky to wait around, and secondly the challenge itself is being met equally by some pretty cool and awesome technology that is driven by AI. It’s a great time to be in technology, and an even greater time to be in testing, so let’s look forward to testing the metaverse and beyond.

A huge thanks to Yashas Mavinakere, Jonathan Beltran, Justin Phillips, Jason Stredwick, and the entire engineering team at test.ai for all the work they do to make things like this possible.

— Tariq King, Chief Scientist, test.ai

References

Balance Academy. What is the Metaverse? Published Sept. 21, 2021. Last Updated Jan. 13, 2022.

Wired. Cecilia D’Anastasio. Video Games Already Do What the Metaverse Just Promises. Jan, 10, 2022

Facebook. Introducing Meta: A Social Technology Company. Oct. 28, 2021.

Michael Kan. New Use for Animal Crossing: Virtual Tech Conference Venue. Apr. 30, 2020.

MIT Technology Review. Tanya Basu. The Metaverse has a Groping Problem Already. Dec. 16, 2021

Bias refers to prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair.

The World is Filled with Bias

A quick search on bias reveals a list of nearly 200 cognitive biases that psychologists have classified based on human beliefs, decisions, behaviors, social interactions, and memory patterns. Certainly, recent events stemming from racial inequality and social injustice are raising greater awareness of the biases that exist in the world today. Many would argue that our social and economic system is not designed to be fair and is even engineered in a way that marginalizes specific groups and benefits others. However, before we can improve such a system, we first have to be able to identify and measure where and to what degree it is unfairly biased.

Since the world is filled with bias, it follows that any data we collect from it contains biases. If we then take that data and use it to train AI, the machines will reflect those biases. So how then do we start to engineer AI-based systems that are fair and inclusive? Is it even practical to remove bias from AI-based systems, or is it too daunting of a task? In this article, we explore the world of AI bias and take a look at it through the eyes of someone tasked with testing the system. More specifically, we describe a set of techniques and tools for preventing and detecting unwanted bias in AI-based systems and quantifying the risk associated with it.

Not All Bias is Created Equally

While there is definitely some irony in this heading, one of the first things to recognize when designing AI-based systems is that there will be bias, but not all bias necessarily results in unfairness. In fact, if you examine the definition of bias carefully, the phrase “usually in a way considered to be unfair” implies that although bias generally carries a negative connotation, it isn’t always a bad thing. Consider any popular search engine or recommendation system. Such systems typically use AI to predict user preferences. Such predictions can be viewed as a bias in favor of or against some items over others. However, if the problem domain or target audience calls for such a distinction, it represents desired system behavior as opposed to unwanted bias. For example, it is acceptable for a movie recommendation system for toddlers to only display movies rated for children ages 1–3. However, it would not be acceptable for that system to only recommend movies preferred by male toddlers when the viewers could also be female. To avoid confusion, we typically refer to the latter as unwanted or undesired bias.

The AI Bias Cycle

A recent survey on bias and fairness in machine learning by researchers at the University of Southern California’s Information Sciences Institute defines several categories of bias definitions in data, algorithms, and user interactions. These categories of bias are summed up in a cycle depicted in Figure 1, and can be described as follows:

1. Data Bias: The cycle starts with the collection of real-world data that is inherently biased due to cultural, historical, temporal, and other reasons. Sourced data is then sampled for a given application which can introduce further bias depending on the sampling method and size.
2. Algorithmic Bias: The design of the training algorithm itself or the way it is used can also result in bias. These are systematic and repeatable errors that cause unfair outcomes such as privileging one set of users over others. Examples include popularity, ranking, evaluation, and emergent bias.
3. User Interaction Bias: Both the user interface and user can be the source of bias in the system. As such, care should be taken in how user input, output, and feedback loops are designed, presented, and managed. User interactions typically produce new or updated data that contains further bias, and the cycle repeats.

Resources on Bias in AI

Interested in learning more about the unwanted AI bias and the bias cycle? Check out these video resources by Ricardo Baeza-Yates, Director of Graduate Data Science Programs at Northeastern University, and former CTO of NTENT. In the first video, Baeza-Yates does a great job of introducing bias and explaining the bias cycle in less than four minutes. In the second video, he takes a deeper dive into the data and algorithmic bias, providing several real-world examples of the different types of bias. Baeza-Yates is clearly an expert in the field and I highly recommend that you check out his Google Scholar Profile for additional resources and publications on this topic.

During last year’s Quest for Quality conference, I had the pleasure of meeting Davar Ardalan, founder and storyteller in chief of IVOW. Her recent post: “AI Fail: To Popularize and Scale Chatbots We Need Better Data” has a list of resources on different topics related to AI bias.

Unwanted AI Bias: A Testing Perspective

Having spent my career studying and practicing the discipline of software testing, it is evident that the testing community has a role to play in the engineering of AI-based systems. More specifically, on the issue of AI bias, I believe testers have the necessary skills to directly contribute to tackling the problem of unwanted bias in AI systems.

Shortly after my friend and colleague Jason Arbon gave a keynote at PNSQC 2019 on testing AI and bias and released a free e-book on the topic, we started brainstorming about what testers can bring to the table today to help with the challenge of unwanted AI bias. Here are the answers that came — testing heuristics for preventing and detecting AI bias, and a quantitative tool for assessing the risk of unwanted bias in AI systems. As life would have it, nearly a year later we’re only just getting around to putting these ideas out into the community.

AI Bias Testing Heuristics

It’s a myth that testers don’t like shortcuts. Tester’s actually love shortcuts — just not the kind of shortcuts that compromise quality. However, shortcuts that take complex testing problems and reduce them into simpler judgments are welcomed with open arms. That is exactly what testing heuristics are — cognitive shortcuts that help us to solve problems while testing software. We provide three types of heuristic-based artifacts to support testing AI for unwanted bias: a set of mnemonics and a questionnaire checklist.

Mnemonics for Testing AI Bias

If you’ve forgotten them, mnemonics are memory tools! just kidding :) But seriously, mnemonics help our brains package information, store it safely, and retrieve it at the right moment. To this day, I still recall many of the mnemonics I learned in math class such as Never Eat Shredded Wheat for remembering the cardinal points and BODMAS or PEMDAS for the order of mathematical operations.

As a starting point for developing a set of practical techniques for testing AI systems, we’ve created seven mnemonics to help engineers remember the key factors associated with unwanted AI bias. These mnemonics are displayed graphically over the AI bias cycle in Figure 2 and are described as follows:

Mnemonic #1: DAUNTS
The challenge of testing AI for bias can seem daunting and so it is only fitting that DAUNTS is our first mnemonic. It is a reminder of the top-level categories in the AI bias cycle — Data, Algorithm, User iNTeraction, Selection/Sampling

Mnemonic #2: CHAT
Since the 70’s, the real-time transmission of text has been a signature of the Internet. CHATS is meant to remind us of the biases in data sourced from the web — Cultural, Historical, Aggregation, and Temporal.

Mnemonic #3: Culture > Language + Geography
To break the monotony of all the acronyms, this mnemonic is in the form of a math equation. Actually, it’s more of an acronym hidden in an equation when in natural language: Culture is GREATER than Language and Geography. This mnemonic represents all the sub-types of cultural bias which in addition to language and geography include these seven other aspects of humanity — Gender, Race, Economics, Age, Tribe, Education, and Religion.

Mnemonic #4: SMS
We’ve repurposed the well-known mobile acronym SMS to help refine the sampling bias category by indicating the need to check for diversity in data sources and appropriate sampling — Sources, Sampling Method, and Size.

Mnemonic #5: MOV
Reminiscent of both the Quicktime movie file type and the machine instruction that moves data from one location to another, MOV now gives us an easy way to remember the types of selection bias — Measurement, and Omitted Variable.

Mnemonic #6: A PEER
Inspired by the peer-to-peer (P2P) architecture made popular by the music sharing application Napster, A PEER encompasses five key biases — Algorithmic, Popularity, Evaluation, Emergent, and Ranking.

Mnemonic #7: Some People Only Like Buying Cool Products
A set of testing mnemonics would not be complete without a good rhyme that feels a bit random and unscripted. This final mnemonic for the types of user interaction bias is just that — Social, Presentation, Observer, Linking, Behavioral, Cause-Effect, and Production.

It should be noted that the aforementioned sub-categories of bias are heavily intertwined, and do not necessarily fit cleanly into the separate boxes as depicted in Figure 2. The goal is to place them where they have the most impact and relevance in your problem domain or application space.

Questionnaire Checklist for Testing AI Bias

Good testers ask questions, but great testers seem to know the right questions to ask and where they should look for the answer. This is what makes the questionnaire checklist a useful tool for understanding and investigating software quality. Such artifacts provide questions that provoke answers that reveal whether desirable attributes of the product or process have been met.

Based on our experiences testing AI systems, we have created a questionnaire checklist. The goal of the questionnaire is to ensure that people building AI-based systems are aware of unwanted bias, stability, or quality problems. If an engineer cannot answer these questions, it is likely that the system produced contains unwanted, and possibly even liable versions of bias.

AI Bias Risk Assessment Tool

Although the mnemonics and questionnaire are a good start, let’s take it a step further and bring another contribution to the table — one that undeniably spells T-E-S-T-E-R. Surely nothing spells tester better than R-I-S-K. After all, testing is all about risk. One of the main reasons we test software is to identify risks with the release. Furthermore, if a decision is made to not test a system or component, then we’re probably going to want to talk to stakeholders about the risks of not testing.

Risk is anything that threatens the success of a project. As testers, we are constantly trying to measure and communicate quality and testing-related risks. It is clear that unwanted bias poses several risks to the success of AI, and therefore we are happy to contribute a tool for assessing the risk of AI bias.

The idea behind the tool is that, like the checklist questionnaire, it serves as a way to capture responses to questions about characteristics of the data, including its sampling and selection process, machine learning algorithm, and user interaction model. However, as responses are entered into the system, it quantifies the risk of unwanted bias.

Introducing the AI BRAT

Our first version of the AI bias risk calculation tool was a quick and easy Google sheet template. However, Dionny Santiago and the team at test.ai have transformed it from template to tool and launched a mobile-friendly web application codenamed AI BRAT — AI Bias Risk Assessment Tool.

To promote learning and application of the heuristics described in this article, questions in AI BRAT are grouped and ordered according to the mnemonics. Definitions for each type of bias appear below each sub-heading, and tooltips with examples can be viewed by hovering over or tapping on the question mark icon to the right.

Expanding a sub-heading reveals questions associated with the considerations made for detecting and/or mitigating each type of bias and the likelihood of it occurring in the dataset. AI BRAT tracks answered questions from each section to ensure each type of bias is being covered.

Risk Calculation

As questions are answered, values for the severity and likelihood of AI bias are assigned to responses. Responses contribute to severity (impact) and likelihood (probability) values as follows:

Severity/Impact
I do not know or have not considered this type of bias (3 Points)
I have discovered unwanted bias and am unable to mitigate it (3 Points)
I have discovered unwanted bias but implemented bias mitigation (2 Points)
I have not discovered unwanted bias or determined it is acceptable (1 Point)

Likelihood/Probability
Likely (3 Points), Somewhat Likely (2 Points), Not Likely (1 Point)

AI BRAT then calculates a risk score using a 3x3 risk matrix and classifies it into high, medium, or low based on the result of multiplying the likelihood and severity values.

Interpreting, Using and Saving the Results

AI BRAT is initialized with a risk score of 100%. In other words, there is a 0% chance that the system is fair and each section is highlighted red. As users respond to each question, the goal is to drive the risk of bias score down until the section turns green, or until the value is as low as possible (in this case 11% which is 1 out of 9).

Results can also be saved to a report using the Export to PDF button at the top-right. Check out AI BRAT today at https://bias.test.ai and let us know what you think.

Other Resources and Tools on Testing AI Bias

Technology giants Google, Microsoft ,and IBM have all developed tools and guides for testing AI for bias and/or fairness.

Google’s What-If Tool

In this Google AI Blog, James Wexler describes the What-If Tool, a feature of the open-source TensorBoard web application, that facilitates visually probing the behavior of trained machine learning models. Watch the video below for an introduction to the tool, including an overview of its major features.

Microsoft’s Fairlearn Toolkit

Microsoft is tackling bias in machine learning through its new open-source Fairlearn Toolkit. Fairlearn is a Python package that enables ML engineers to assess their system’s fairness and mitigate observed unfairness issues. It contains mitigation algorithms as well as a Jupyter widget for model assessment. Besides the source code, this repository also contains Jupyter notebooks with examples of Fairlearn usage. In the video below, Mehrnoosh Sameki, Senior Product Manager at Azure AI, takes a deep-dive into the latest developments in Fairlearn.

IBM’s AI Fairness 360 Toolkit

The staff members of the Trusted AI group of IBM Research have released AI Fairness 360 — an open-source toolkit that helps you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle. The toolkit contains over 70 fairness metrics and 10 state-of-the-art bias mitigation algorithms that have been developed by the research community. Check out a quick demo of AI Fairness 360 below.

What’s Next?

Wondering how to get involved? Here are some ways we believe folks can have an impact on testing AI and bias:

1. Devising practical testing methods and processes for preventing and detecting unwanted bias in datasets.
2. Developing new coverage models and static/dynamic analysis tools for validating AI and ML systems.
3. Mastering and contributing to the existing open-source toolkits for measuring fairness and detecting/mitigating unwanted AI bias.

References

1. Bias on the Web. R. Baeza-Yates. Comm. of the ACM, Vol. 61, № 6.
2. A Survey on Bias and Fairness in Machine Learning. N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan.
3. Testing AI and Bias. J. Arbon.
4. Software Testing Heuristics: Mind the Gap!. R. Bradshaw, and S. Deery.
5. The Quest for Quality in AI. D. Ardalan, T. M. King, N. Chelvachandran, K. Obring, Y. Sulaiman, J. Farrier, L. Zubyte, J. Jerina, and R. Mugri.
6. 7 Types of Data Bias in Machine Learning, H. Lim.

Quantifying the Risk of AI Bias was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

Last week I had the honor of giving the opening keynote at StarWest 2017 and PNSQC 2018 on the topic of Artificial Intelligence for Software Testing (AIST). The talk, entitled “Rise of the Machines: Can Artificial Intelligence Terminate Manual Testing?”, created quite a bit of buzz at the conferences. You can check out the video below if you missed it. Aside from the “Do you think AI will take over the world?” type questions, one of the most popular things I was asked about after the talk was: “How do I get started with AIST?”

The short answer to that question was to point delegates to general resources on AI and machine learning. However, it is important to recognize that such a question is loaded and therefore deserves a more detailed answer. First let’s start by revisiting the definition of AIST.

Artificial Intelligence for Software Testing (AIST) is an emerging field aimed at the development of AI systems to test software, methods to test AI systems, and ultimately designing software that is capable of self-testing and self-healing.
 — AISTA (www.aitesting.org)

Examining the definition we can see that how to start depends on two things: what you are trying to achieve and where it lies in the intersection of using AI to test software, testing AI-based systems, and building software with intrinsic self-testing and self-healing capabilities. A safe place to start is pursuing knowledge and understanding in the areas of artificial intelligence, software testing, and self-managing systems.

Artificial Intelligence

AI is not new. It has been around since the 1950’s and knowing its historical progression and major advances over the years will help you to position your work in the AI space. Creating the vision for your tool, method, or approach, acquiring the right talent, aligning team members, developing the solution, and communicating the value of your work all start with positioning.

The field of AI is broad and so I recommend that you gain an understanding of that breadth before you do a deep dive into specific areas. Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig is a great reference book for all things AI. When you are ready to go deeper, you’ll want to get your hands on more specialized books. For example, Deep Learning by Ian Goodfellow presents a broad range of topics on hierarchical learning architectures. In terms of freely available self-paced multimedia learning, check out the video courses on YouTube from Andrew Ng (Stanford University) and Lex Fridman (MIT).

Software Testing

For as long as there has been software, there has been software testing. Despite the rise of other approaches such as formal verification and model checking, testing remains the primary means of validation and verification (V&V) used in the software industry today. Like AI, the field of software testing is extremely broad, but chances are that if you are looking into AIST you may already have a background in testing. However, if you don’t and are looking for online resources on software testing, check out this post on 45 Websites Every Software Tester Should Bookmark in 2017. For books, some old favorites of mine are James Whittaker’s series on How to Break Software, Lee Copeland’s Practitioner’s Guide to Software Test Design, and Cem Kaner’s Testing Computer Software.

Self-Managing Systems

In 2001, software giants IBM, Microsoft, and HP all set out on their own initiatives to define the next generation of computing systems. Such systems were envisioned to be self-managing, and able to adapt to changes in order to meet the needs of its users. IBM coined the term autonomic computing and modeled their approach against the human autonomic nervous system (ANS). The ANS regulates critical bodily functions such as homeostasis without conscious involvement. Applying this idea to computing, systems should adapt to their environment through self-configuration, self-optimization, self-protection and self-healing. What? No testing? Although the giants weren’t talking about it, in order for autonomic computing to work in practice, self-testing needs to be an inherent part of the system.

It was for this reason that I decided to pursue this research area during my graduate studies from 2005–2009. Unfortunately to date there is still a general lack of research in the area of self-testable autonomic software. However, the recent advances in AI/ML, Cloud Computing, DevOps, and Microservices, have improved the potential for self-testing to provide a formidable solution to the grand challenges of software testing. We’ll be posting some self-testing and autonomic computing foundation papers on AISTA’s website for those interested in learning to how self-testing systems can be designed and developed.

Other AIST Presentations

In addition to the Rise of the Machines keynote, StarWest this year featured two other talks on AIST. Jason Arbon gave an updated version of his talk “AI and Machine Learning for Testers”, originally presented earlier this year at StarEast. Paul Merrill from Beaufort Fairmont presented a concurrent session on “Machine Learning and How it Affects Testers”.

What’s Next?

As much as you can read books and watch videos about AIST, there is nothing more powerful than getting hands to keyboard. In a future post, we’ll cover some tools that you can play with to get a feel for AI and ML.

Tariq King — Co-Chair (AISTA)

Getting Started with AI for Testing was originally published in AI for Software Testing on Medium, where people are continuing the conversation by highlighting and responding to this story.