The Silver Bullet for Game Validation

The 4 classic methods of ensuring success

I lied. There’s no silver bullet.

I keep yapping about game validation quite a lot — you might’ve seen me speak about it in events and gatherings. Game validation is about playing a perfect game with what you have. Very similar to how the best poker players can be successful in the game some people still think is a game of luck. Game development — just like poker — is a game of luck to only those who rely on luck. (Anyone can win a poker tournament. Anyone can score an overnight hit in game development.) The best players obsessively try to minimise the luck factor. This is exactly the core essence of continuous game validation. I’m obsessed about it since it’s one of the most important things in game development without a clear solution. Well, there are “solutions”, but these are usually fake and/or misused. I’d like to take a moment to clarify a few things about the classic “solutions” below:

1. Hire an expert to make sure our games are top quality.
2. If it’s fun for the team, it’s a great game!
3. Just try many times to beat the odds.
4. We’ll do a soft launch to make the game better!

Let’s look at them:

1. Hire an expert. 
   If an expert knows how to make a successful game, you’d think they’d put up a game dev studio and made billions already, right? I’m not saying an expert can’t be useful — of course they can. They might’ve worked with a successful product before and can have valuable insight for — if nothing else — raising important questions. But I would keep it crystal clear that an expert is never the solution. An expert is an analyst of the past, not a predictor of the future. “Expert” derives from “experience” (I would guess) and has little to do with visionary skills of foreseeing what will work tomorrow.
   I would go even further and say “Don’t ever have an expert make decisions, have them only raise questions”. …maybe in some cases they can carefully suggest a general area of ideas where a solution may or may not be found.
   In poker: Make a decision based on what worked in someone else’s game a week ago.
2. Find the fun.
   Yes, please do. It’s super important in terms of motivating the team and having everybody believe in the game. If nobody in the studio enjoys playing the game, it’s pretty hard to put passion into developing it. And if the team consists of people who can find the fun in any of the successful games in the market, their feedback might even give a good indication of your game’s market potential.
   In the worst case, though, it’s a really niche opinion and you all end up being *that* group of people who whines about how stupid the players are. “We found the fun, why can’t they?” (Hint: they are not you.)
   Just keep in mind: if you are making the game for the market, make the game for the market. Totally fine to make a game for yourself; validation is obviously easy in such case. But sometimes that has nothing to do with any sort of business potential outside your garage.
   In poker: You like your hand.
3. Try, try, try again.
   Getting it right on the first try is insanely difficult, so you should definitely prepare for multiple tries. However, that doesn’t mean you can just throw random stuff at the wall and see what sticks. If the odds are, say, 1%, you’d have to throw random stuff at the wall over 50 times to have even a 40% chance of something sticking. (Please realise that 1% chance of succeeding is pret-ty high.) 
   People talk about focus, vision, direction, etc. and these are important things when it comes to multiple tries. Think about it: if you increase your odds by 1 point each time (i.e. 1%, 2%, 3% odds), you’ll need about 9 or 10 throws to get that same 40% chance of sticking. That 1 point increase can be done in multiple ways: reusable tech that has been tested, metrics that are comparable, partners that have been found beneficial, processes that work, and other types of consistencies that often are nice side products of having focus. And this is the actual validation part of “trying again” — not the multiple attempts. Re-use what makes sense, fix what doesn’t, try again. 9 times or 50 times?
   In poker: Play every hand — who knows, you might get lucky.
4. The almighty soft launch.
   This actually is about validation. It’s a powerful tool to get important answers before committing more resources into the game. Describing how to make a soft launch would take a whole article, but a good rule of thumb is that soft launch should be used to answer questions that start with “Does it make sense…” 
   You can twist that rule to get your questions past it, but at least it sets the right mindset. Here is an example:
   “How can we get the day 7 retention from 15% to 20%?”
   vs.
   “Does it make sense to try to improve the day 7 retention?”
   Even if you end up thinking about the first question, at least you went through the latter one. There might be ideas on how to improve the retention numbers, but does it make sense? Is there another project that would benefit from the same resources even more? How long are we willing to try? If the second attempt doesn’t work, shall we try a third time? If not, what’s the difference between attempts 2 and 3? It’s not a choice between giving up and continuing, it’s a choice between spending time on this game or the other.
   This kind of approach also forces you to think about the soft launch as the validation tool it is, not as a development tool. You shouldn’t go to soft launch and think “Let’s see what the numbers are and fix them if needed”. It takes a lot of time (i.e. it costs money) to reach a good soft launch, so you should rather think “We estimated that the numbers will be such and such, which are good enough. Now let’s do a soft launch to make sure we were right and it makes sense to go forward with the launch”.
   In poker: Re-evaluate your hand after each time you get new information.

https://twitter.com/anttisten