DARK HORSES OF GAME DEV. Why is QA more important than you think?
In the fog of game dev
Despite the fact that video games have long become commonplace, from a historical perspective, this is a very young industry. And if major breakthroughs seem to have already occurred in the field of game genres and mechanics, the approach to game production seems completely unfledged.
If you have worked in different companies, then you have probably seen for yourself how differently you can solve the same tasks, and at the same time, how unique the production of a particular game eventually becomes, even if it’s at least a third generation copy of what was released before. Why does this happen?
Think of the human body. Hundreds of thousands of years of evolution — all of its processes are fine-tuned. The circulatory system “knows” that oxygen can be obtained in the lungs, the stomach “understands” that waste must be sent to the intestines, and so on. Anatomy is unified, and at the system level, all people are the same.
It’s not the same with games. And although for the player, individual projects may seem the same: Diablo clones, Dark Souls clones, Ubisoft open worlds — they are all like twins. But under the hood, these games will almost always be different. As a result, the interaction within the team almost always falls into place in a completely unique way.
Back to the metaphor of anatomy: imagine a system in which functions are distributed among organs in a unique way, and the usual heart, brain or skeleton is nowhere to be found–everything is completely different, as if it were an alien. But all the functions are in place: the alien moves, thinks, expends energy and consumes.
In fact, every game development studio is such an alien. We roughly understand what functions ensure its viability, but when viewed from the outside, we don’t know which organ is responsible for what and what the principles behind its operation are.
If we return to reality without metaphors, the reason for the eternal ambiguity is that professions in game development are quite flexible and do not have clear boundaries. This occurred historically. Look at, say, Shigeru Miyamoto, the father of the Mario series, The Legend of Zelda, and more. He started his work as an artist, then became a game designer, then a development manager. One person, i.e. “organ”, performed completely different functions at different times.
Often, tasks in the team are distributed according to abilities and go beyond the usual: game designers can work on game texts, an SMM manager handles support, and so on. Moreover, one and the same task can be done by several team members holding various positions. This is especially true for indie studios or small teams. And that’s cool, because it makes the team flexible.
The flip side of this flexibility is chaos. Let’s consider an abstract example: there’s an ancient icon in the game that someone once made, and now it’s time to replace it, because the function that it served has become irrelevant. The most difficult part is that the name of the person who is responsible for it is not written on the icon. And then you start asking around. You ask one specialist, another one, a third one — anyone may be glad to help you, but none has an answer to your question. Along the way you find out that the person who made this icon is no longer with the company, and you need help from the very top of the tech team to restore lost connections and uproot old graphics. If the anatomy of the team were unified, you would simply go to the “organ” that is responsible for the icons and resolve the issue in a matter of minutes. But the reality is that there is no such “organ”, and each team is unique.
We are not going to delve into why a cookie-cutter structure cannot be used to organize a game studio. On the contrary, let’s try to understand who can come to the rescue in a system where there are so many unknowns. The answer will seem simple — establish proper communication. But it also has to be entrusted to someone.
So, let’s imagine that we have a completely unique team, where everyone is talented, but individual participants in the process are not always aware of who is responsible for what specifically on this project. Fortunately, we have a good understanding of both the global task (creating a game) and the general principles of its implementation.
We have an idea generator, so to speak, most often it’s the game designers (hereinafter the GD), who impose their vision for the game. There are implementers who are responsible for all the mechanisms of the game, visuals, analytics, and promotion. Finally, there is the result of our labor — the game itself — and a happy player, for whose sake it was all conceived.
However, all of this is dandy only in an ideal world. As we know, in practice everything is closely interconnected and one intersects with the other. Thus, our structure needs to be refined. Everything is layered, tangled and intersected. It turns out like this:
Here it’s easy to understand that issues with communication occur precisely where there are overlaps. You have a question and you don’t understand who you ask: the GD, the developers, or maybe the support? After all, all of them, to one degree or another, can possess the coveted knowledge.
The next obstacle to successful communication is language differences. Artists, UXUI professionals, game designers, and developers communicate differently and on the premise of different values. Some work on the aesthetics, others on the accessibility of experience, yet others work on engagement, and some just want at least something to work properly without crashing.
Looking at the ongoing chaos, I want to find a new stakeholder, a mediator who will reconcile differences and find harmony in chaos. A person who is unbiased, neutral and distanced (in a good sense) and can explain to the “organs” of the team how they interact with each other, while looking at the process from the outside, Who could it be?
This is where the department, which we have so far overlooked and not included in our structure, comes to the rescue. It’s the QA. Let’s figure out what they can do and what place they occupy in the workflow.
Where does quality come from?
Simply put, the QA are testers who take the finished game, check the performance of everything that is included in the game and, thus, become the ultimate authority before the product finds its way to the player. And to make sure of the quality of the game, the QA need to know everything about it. To understand what the product team wanted and how it all came to life through the efforts of the tech team. The QA know all the details. For example, how animation should work, according to the artists’ idea, how it is arranged at the mechanics level, and how all this is interconnected. But the best part is that the QA are not spoiled by the position of “author and creator”, which means they are insured against the fog of emotions, which sometimes results in conflicts, when, for example, it is not possible to implement what was conceived due to limited resources. In this case, the QA, having an equal understanding of all aspects, are able to really act as intermediaries who will put out the fire.
There will be more order in our structure, if we imagine that the QA are at the center of everything, and all the overlaps and confusion in terms of responsibility pass through them.
A specific case. On our Donut Punks project, we have a writer, who uploads text into the game himself. He has a question, for example, about the readability of the Asian localizations of the game, where there are thousands of unique hieroglyphs, and it’s not clear how they will work in the game. In addition to the writer, UI designers, and to a certain degree the GD (if we’re talking about a functionally significant text, and not an artistic description), as well as several members of the tech team normally work on the text. And such a simple question creates chaos when trying to find an answer to it, because everything is overlapping. In search of an answer, you can cause a stir among half the team, distract them from other tasks, find a solution, but without any guarantees that everything will work when the game is assembled.
But it is worth adding the QA to this chain, and order will be immediately restored. The writer will contact the tester, who will forward the issue to the right developer without disturbing others. The developer will find that the issue fits into the usual pipelines, and will pass it on to the UI department. They, in turn, will come back to the QA with a solution.
It may seem that the QA become a redundant link here, but that’s not the case. And for a very simple reason — the QA’s job is to ensure the quality of the game, which means that any issue, even a hypothetical one, should fall into the field of their attention. Back to our example: if the writer tried to resolve the issue of text readability, bypassing the QA, then this would simply increase the chance of catching a bug where no one expects it. In this sense, it is extremely important that the testers not only help resolve issues, but also monitor them to make sure that the solution works.
So, the QA looks like the perfect candidate to be the core of communication in the team. We understand that testing is not an isolated process for the later stages of development. On the contrary, the QA’s work follows all cycles, starting from the earliest, where, for example, expertise in the concepts of future features is urgently needed.
The QA are responsible for quality, but quality is not only the result of polishing what has already been done. This is basically the result of work at any stage.
If we integrate the QA processes into development to the maximum, then a stable communication system arises between all participants, and the testers themselves turn into carriers of the most accurate and extensive understanding of the product. Their tasks in this case are redefined: it is no longer just a routine bug catching, but something more creative.
So, the QA is now the ideal candidate to become the core of communication within the team and help build relationships. Obviously, this view of the profession takes them beyond the idea of usual “checkers” who play games all day long.
Along with technical competence, the most important requirement for the QA is a set of soft skills, spearheaded by openness and willingness to help.
How to survive in the QA world
If the idealistic view of the QA seemed excessive to you, we can come down to earth for a while and see how everything described above is implemented in the real experience of the Donut Lab studio using the development of the Donut Punks mobile shooter as an example.
Our QA department, having taken their lumps and faced informal issues over and over again, developed their own philosophical system that helps to maintain a healthy mind and prevent burnout.
The cornerstone of this system is the principle that we call “action–reaction”, the literal manifestation of the saying “as you sow, so shall you reap”. In testing, it’s very important to properly present a bug to the team. If you come to the developers screaming that everything is broken, nothing works, the release is in jeopardy, disaster is imminent and higher powers are not on our side, you shouldn’t expect any productive reaction. If we replace drama with concern, if we stop looking for someone to blame and express a desire to help, then the developers will respond to the tester in the same way.
Another important principle is widely known in the QA circles and is laconically called “priority and severity”. These are the two criteria for assessing bugs that help you sort out all the issues and decide in what order to remedy them.
Most importantly, priority and severity are determined solely through the values of your project, which makes this approach very flexible. A simple example: in one of our builds, we found a bug with the playback of a shooting animation in the onboarding section of the game. It did not affect the gameplay in any way, shooting still worked, and such a bug didn’t even exist outside of the tutorial. So the severity of this issue for us was minimal. But we set the priority as high as possible, and here’s why. For Donut Punks, aesthetic quality is one of the key values, so we can’t incorporate a buggy animation into the game. Especially when it comes to onboarding, the stage at which the player gets their first impression of the game. This is a textbook example is of how the “priority and severity” principle helps to introduce order in the QA tasks and spend energy on correcting really significant things.
Segue to the question of how the tester should estimate their strength and inform the management of adequate deadlines for completing tasks. This is where iterative testing comes in. The product verification process here is divided into many small stages (iterations), at each of which synchronization both with the manager and with the developers takes place, the deadlines are adjusted, and the workflow is transparent to everyone. Without iterative testing, the QA would have to literally invent deadlines, because without an initial assessment, it is simply impossible to predict the number of bugs and the work that they will require. It’s easier with iterations — each stage brings clarity bit by bit. The testers are not isolated in their cozy little world and keep in touch with different departments at all times.
This is how another important principle in the work of the QA — openness — is born. Sounds abstract, but it’s very concrete: we support openness by creating chat channels that connect all the stakeholders. For example, Support (QA-DEV-Support), where the support publish complex requests from players, and the QA addresses them to specific developers. At the same time, the process is in full view, the tech team is aware of who is doing what, and anyone can join the discussion on how best to resolve a particular issue.
Moreover, at a certain stage, we noticed that the developers, without waiting for the QA, take note of players’ requests, and all the testers have to do is monitor the process. This spares us the need to have junk communication, finding out who is responsible for what, and as a result, it gives a great boost to the development of independence within the team. At the same time, openness helps to keep all processes under control. So, for example, you can avoid a collision when two developers unknowingly take on the same task.
It is worth noting that a competent QA generally tends to strive not only for selective debugging, but for building a system where all minor issues are resolved of their own accord, thanks to the said open chats, intelligible documentation, FAQ, and software tools. The testers, having spent time tweaking such systems, can forget about languishing over routine issues, and devote newly available resources to truly interesting tasks that require resourcefulness and ingenuity.
Testing is such a vast area that your team can include people with completely different backgrounds, experience, talents, and passions. So that everyone can find their place, we decided to send new-comer Donut Lab testers on an “adventure”. In the first couple of months, the specialist receives completely different tasks that require them to get to know all the departments and, as a result, the features of the project. At the same time, tasks are feasible, reasonably challenging and not overly difficult. This forms a minimum set of universal skills that help the QA team members to replace each other and swap tasks. On the other hand, our little adventure helps to see the true talents of the specialist in order to ultimately entrust them with a certain domain — for example, quality control of the Android version or, say, product testing. But even in this case, we don’t completely abandon the idea of versatility, offering each tester something like the following ratio:
- 70% are tasks where the talent of a particular specialist flourishes the most,
- 30% are challenging tasks, where you need to do something unfamiliar or unusual.
This approach strengthens relationships within the team. No matter how different the testers are, they are connected by a common goal — product quality. As a result, juggling tasks among specialists greatly stimulates the mutual exchange of experience. If you don’t know how to deal with a technical bug, ask a colleague who is more familiar with the topic for advice. IT teams often employ loners who are narrowly specialized in one thing, which means they can’t expect help from anywhere — you need to deal with everything yourself. The testers are insured against such a fate, and mutual assistance here can become one of the pillars of working relationships.
Finally, yet another basis for the philosophy of our QA was the principle “the tester will help in any way they can”. This ultimate responsiveness means that any member of the team can come to the tester with their issue and try to figure it out.
At the end of the day, the QA ethic naturally extends to the entire team.
Testers and Co.
The key is not to lose sight of the goal — the QA keep all communication between the individual parts of the team up to the mark. But there’s another important participant in this process — the players themselves. Our feedback pipeline looks like this: the community team collects feedback and communicates with gamers, while we openly publish the players’ general impressions and make them available to all production participants, but precise questions like issues, bugs or feature requests are passed through the QA, who distribute tasks among departments in a targeted way.
Naturally, the “task” in this case is not an imperative and at the first stage it is simply investigated, and only then a verdict is made on the expediency of implementing a particular request.
A feature of communication here is an additional need for translation. Let’s say, when investigating a question from a player, the developers say something along the lines of: “there are no such defs in RC_27.” The QA translate this into a more accessible language for the support, explaining what exactly will work and what will not, and it becomes: “Freezie will not be nerfed in the August patch”. And then the support turns it into an informal, simple and accessible message for the player: “In the next update, Freezie’s balance will remain the same”.
This was achieved for three reasons:
a) The support know how to exclude the player’s emotions, highlight the essence and understand how to present information to the QA
b) The QA are aware of the current development status and rely on accurate data, while avoiding vague promises
c) The QA understand the language of both the developers and the support, and therefore can quickly isolate the gist of the message and convey it in the right way
As you can see, the support in this case become the right hand of the QA and ensure that the development team will not be closed in their processes and will always feel and know that there are players out there, that they see everything, know everything and are interested in having a good time.
In the gaming industry, such a response is extremely important, because it can take several months between the development and implementation of a feature, the response from players is not instantaneous, and developers can even forget that they are performing tasks not for Jira, Trello or their project manager, but for players.
Fortunately, in game dev, developers themselves are mostly gamers. When you’re into game development, it’s a challenge to retain the player’s perspective, not that of the creator. It’s easy to fall into the trap of professional distortions and start playing only to see how a particular project implements a mechanic, interface, plot or art. In this case, of course, the main goal — the pleasure of gaming — is completely lost.
Our QA department, oddly enough, partly solves this problem as well. Every week we organize gaming sessions (internal playtests), a very special event in terms of its spirit and atmosphere. The tech team put together special builds with experimental mechanics, and then whose who wish gather in the comfortable lounge area (or connect via a video conference) to play together, share opinions and emotions, and, as a result, understand how exciting the game is for ourselves.
Organizing these gaming sessions is the QA’s responsibility, they monitor the process of putting together test builds, upload them to devices and trumpet “Assemble!” when everything is ready.
One might get the impression that in the end, our QA are heavily overwhelmed and have to babysit everyone. In fact, this is not entirely true: if the QA take over the functions of communication, this does not mean that, roughly speaking, the developers cannot independently agree on everything with the GD, or the artists are not capable of turning to the interface designers with a question.
Everything is exactly the opposite: the QA’s work to establish communication becomes almost invisible — the testers in most cases only help those who have lost their way to go in the right direction. And as soon as you get on the right path, everything suddenly starts working by itself. Ultimately, this is how healthy communication is formed in a team, where no one is embarrassed to speak out, no question is left unanswered, and all complex issues are resolved through intermediaries represented by the QA, who help techies and artists find a common language.
