Does Your Game Idea Suck?
Stop a catastrophe before it starts.
Historically games have come into existence through a good pitch, with the right team, to the right publisher. That was my job for decades, working up game concepts with my team that focused on what our publisher, Sony PlayStation, was looking for. The primary goal was not to pitch a game that consumers would love but one which our publisher would love. Hopefully, there’s a healthy overlap between those two but looking back it certainly restrained the breadth of games being made. After all, we were gamers pitching to gamers.
Mobile Gaming is Different
That was 2005. Since then the incredible success of mobile gaming has proved that anyone will play games if you present the right kind of game on a platform that makes it easy to play. That success has also dramatically changed the developer-publisher relationship. My refined pitching skills had no value in the business-to-consumer environment afforded by the App Stores — there was no one to pitch to.
Figuring out what games to make in this new landscape took some unlearning on my part. When first starting out in the mobile free-to-play world I remember asking another studio head how he’d been finding it.
“Ah… not so good. The only thing we’ve been able to make any success of is our word search games”
He considered it a failure that the more ambitious games, which were closer to the console video games we’d made in the past, didn’t get out of the starting blocks. It didn’t occur to either of us that the reason the word search games were successful was that they actually were what the market wanted to play. For too long we both tried to persuade a disinterested audience that they should be playing more sophisticated games.
Becoming part of MAG Interactive taught me many things but one of the most valuable lessons was the importance of finding out what our audience actually wants to play. This is a process that’s been heavily refined over time, and is now something that I wouldn’t even consider starting a game without knowing. It’s such an integral part to a game’s success.
Concept Testing Basics
Concept Testing is a method of finding out whether an audience wants to play your game before you make it. In short, how appealing is the idea? I’d guess that 75% of mobile game companies do this, so if you’re already one of those then you might want to skip the remainder of my article and go straight to my colleague Isak’s article which describes a smart and free way to do it.
To start testing our concept, we mimic what’s done when we release a game, which is to buy traffic. Buying traffic is an ugly term to describe the process of advertising on Facebook (or other platforms) to draw players to our game. Since this is the method we’ll use when we have a finished game, it’s the perfect method to measure interest before we have the game. The very short version is: we run ads for games that don’t exist yet and see how much interest there is.
Often we’ll run the ads before we write a line of code for the game meaning that:
- We’re not spending our resources building games if we can’t prove there’s a market
- We avoid the sunk cost fallacy where we continue with a game because “We’ve come this far, we might as well continue” which is a human response built on flawed logic that is hard to avoid
The ad we use for our test will be run in the same way that we would for a finished game. It’s often a video ad that describes what we intend the game to be, in the graphical style that we intend to present it. Here’s an ad that we ran for our game Wordzee, nine months before it was actually ready.
Collecting Your Data
Once the ad is live you should start looking at two metrics:
- Click Through Rate: the percentage of people that click on an ad when it’s presented to them.
- Store Conversion: the percentage of people that click on the install button when they get to the App Store.
Click Through Rate (CTR) combined with Store Conversion tells us what proportion of people would install the game if presented with an ad, which allows us to estimate Installs Per Thousand Impressions (IPM).
IPM is a great indicator of what the Cost Per Install (CPI) will be. Facebook wants to show as few ads as possible but get the most revenue, so an ad that creates a lot of installs is an ad that will achieve a lower CPI. Benchmarks for these vary from category to category, but for us (word, puzzle) an IPM of 8 is good.
For more information on IPM and why it’s important, check out this article from our friend Dan Greenberg at IronSource.
Measuring Your Data
You might be wondering how you can measure Store Conversion where there’s no game to put on the store. The solution to this is to present a webpage that looks similar to an App Store, but is in fact a web page. There are smart things you can do here, such as using Google’s split metrics product and including a Facebook pixel to simulate an install, which feeds back to Facebook’s User Acquisition algorithm.
When I describe this process to people, the most common question I’m asked is “what happens when a player clicks on install?”. The short answer here is that it doesn’t really matter. It could be something as simple as creating a “Coming Soon” page, but you’ll likely only reach 0.01% of your potential audience so it’s not worth focusing on trying to do something clever. Ethically, you could argue that you’re wasting people’s time by advertising a game that doesn’t yet exist, but the process is likely 10 seconds so it doesn’t seem too onerous.
Conclusion
Game Concept Testing by running ads is, in my mind, the most powerful tool a game developer has in maximizing the chance of success. It doesn’t tell you whether players will retain or whether they’ll spend, but it does tell you how appealing your game is and that’s directly linked to CPI.
In the past we’ve used products such as StoreMaven for this purpose but more recently our ASO god Isak figured out a good method of doing this with free off-the-shelf tools. Continue your understanding of these concepts by reading about his methods here.
