First Customers

A brief overview of our start-up’s successes and a longer look at our many mistakes

I was at the gym, trying to get rid of some of the frustration and extra padding caused by long days hunched over my laptop. When I got home, I poured a post-gym glass of wine (Yes, I know you’re not meant to, thanks for the health advice), there were 6 missed calls on my mobile from the other two founders, Paul and Dan.

We were bankrupt. Half the team had resigned. I had posted something massively inappropriate on the company’s facebook page … What the hell was wrong???

I rang Paul. For the first time in three years of working together he actually answered his phone.

“So guess what?”

“What?”

“Look on flowdock.”

He hung up. Phones aren’t really Paul’s thing.

I checked. There it was. A payment screen from Facebook. $5 translated to about £3.75, minus Facebook’s commission took us to £2.63.

Our first paying customer.

We’d done it. Someone was actually prepared to pay us for playing our game — Blackjack Attack.

I rang Dan.

He and Chris, our designer were already in the pub and several pints into the celebration. Dan had carefully worked out what % of craft beer in a London pub was justified by his share of £2.63. He explained the maths to me happily over his next pint, which, he pointed out, should count as further investment in Gamevy.

A customer.

I rang Paul back to do some jigging around the kitchen. The more gleeful I was, the more depressed-sounding Paul insisted on becoming.

“She might have done it by mistake,” he said. “Who knows if she really grasped our badly designed payment screen. We’ll feature in a black arts of UX blog warning of customer deception.”

“It’ll probably be 2 weeks before we get our next paying customer,” he added. “Still goes to show, eh?”

Translated, that meant Paul was as thrilled as a pint-drinking Dan and a kitchen-jigging me.

But as it turned out Paul was wrong. It didn’t take 2 weeks to find our next paying customer. It took a lot longer — and an advertising cost that would make Dan’s celebratory bar-bill look like a mere drop in the ocean.

In retrospect, you see, that first customer gave us a glow of false optimism. A few days into opening up the game and there was money. MONEY! But over the next few weeks, as we began to experiment with adverts, pulling more players into the game, improving the experience little by little and adding features … we would discover just how many problems there were with the game for the average player.

The Stats

Our adverts did respectably. By experimenting with text and imagery, we managed to find a combination that delivered a click-through rate of 0.5%. This translated to an install rate of 0.3%. Our cost per install was £1.40.

Now if only we could generate that £2.63 for every £ it was costing us to acquire a customer — then soon we would all be rich!

But of course, it doesn’t work like that. Installs do not translate to players. Players do not translate to paying customers. The gulf between those groups was about to become agonisingly clear to us.

Of those who installed the game, only 50% went on to use it at all. That meant our cost per user spiralled to £2.80.

And usage simply meant spending 4 minutes there. Four minutes was no use to us. We could tell that they were spending most of that time in our interactive tutorial. Were they bored? Were they confused? What was going on?

Testing

I sent out a survey to everyone who had played. The first day we had massively encouraging results from this — a really high open rate and respondents who answered that they liked the game and would recommend it. For the next batch of players though, the response rates fell drastically. And anyway — the people we really wanted to talk to — the ones who installed but then didn’t play; or who played for a couple of minutes and then left — weren’t going to open or answer our survey.

So the following day, we took advantage of a big conference of gaming professionals at Excel and headed out to do some user testing. What this meant in reality was prowling around the café area, pouncing on anyone who looked mildly bored as they ate their lunch and begging them to test the game.

Despite a fair few rejections and blank looks, plenty of people good-naturedly agreed to help me. Some took the tutorial, others skipped it and went straight to the game. I knew that those who didn’t take the tutorial would be a bit confused. I didn’t know that by my 10th tester I would be secretly hoping they wouldn’t take the tutorial!

The tutorial — our fun, interactive series of mini-games — was almost unbelievably slow and boring. Unlike on our designer’s massive monitor, on the screen of my laptop, the tiny writing made people peer anxiously at it. The buttons for ‘next’ and ‘play’ felt so small it was as if we’d been trying to hide them.

But it was only once the game began that I found out our carefully thought-out tutorials had been for nothing. Not one of them had taught the player what they needed to know.

The Goal

After a round or two, these charming people announced: “Oh you’re playing other people, not the dealer! I get it now! I guess that info was probably in all the adverts!”

Ummm.

Of course we knew that what made our game special and different from blackjack was that Blackjack Attack was a multi-player version with no house or dealer. I’d written about it on a load of articles we’d done for a mini-site. But I had totally failed to include it in the advert. Instead I’d been obsessed with what would make people click — with fun-sounding tag-lines and intriguing images.

Our tutorial explained all the key mechanics of our game — the idea of attacking, for example, and some good strategies about when to stand. It completely failed to mention the crucial goal of beating all the other players. It was implied everywhere, but never explicitly stated. For people who’d never played blackjack this wasn’t too big a barrier — but for our core target audience — blackjack players open to a game with a twist, this failure was almost incredibly off-putting.

Flaws in the UX

We knew there were problems with our UX. We’d tried to find quick fixes for some of them — including the tutorials. We’d hoped to teach: where the player was sitting; the aim of the game; how and when to attack or stand; how cards were delivered and how betting worked.

As soon as I watched these people play, the problems became even more glaringly obvious than they had been — as did the inadequacy of our solutions.

“Where am I sitting?” the customers asked me.

In my best non-leading customer research manner, I asked where they thought they were sitting. Some got it right. Some wrong. Eventually, all of them noticed the sign saying “Welcome to your seat”. Many of them were embarrassed. They apologised to me for their stupidity in not seeing this. Glumly, I realised that our beautiful font was unreadable and far too small. I also noted that 4 mini games designed to show a seating position had failed.

“What am I doing? Am I playing?”

Again after my careful non-prompting prompting, they read the sign that said “Waiting for the round to end”. They apologised again. The font seemed to be shrinking in front of my very eyes.

“Wow is this live action?” they asked me.

I made another note of an unbelievably important and obvious point that someone who had spent years working in marketing should have included in the adverts.

Finally the game began.

The action starts with players placing their bets. There’s a time limit for this betting phase — a long, slow, boring time-limit, we’d thought back in the office. So slow we’d worried the players might be leaving out of boredom.

Boredom! Ha! Not a single one of the people I tested the game with managed to make a bet on their first attempt. All of them ran out of time. That was in spite of a ‘tutorial’ about how to bet, which they had all taken part in successfully.

“How do I get a card? Where are the cards coming from?”

I crossed off the final element of the tutorial.

Just like our actual online customers, these testers had taken the ‘correct’ path in the tutorials, seeming to show they were learning our crucial mechanics. But once in the game, it was clear none of it had made any actual sense to them.

Problems piled on problems. Betting and pots was a mess. I can’t even bear to describe how far people were from getting it. No-one could see why a player won or lost. The payout animation made a tester cheer — but without any understanding of why.

I wrote page after page of the same comments. A dozen users is not a statistically significant group and a group of gaming experts in the industry aged from 21 up and all in the AB group may not be an accurate sample of your target market… But when all twelve, helpful, educated people have exactly the same problems, at the same points … well, you can probably accept that those are your pain points.

Silver linings and research fallacies

Because I was sitting next to them, these players kept looking. Where a user at home would have exited the game (as our stats clearly showed they did) after a few seconds, these people kept trying. Every single one of them got the hang of it in the end. Every single one of them said they enjoyed the game.

I left the sessions knowing that there was a golden idea in here somewhere. The problems were in our execution of the first few minutes, not in the central concept itself. Even accounting for the bias that occurs because people want to please the researcher and struggle to be rude after someone has sat and talked with them for 20 minutes, I was still convinced of this.

Are we just dumb?

I almost hate admitting this, but the three of us are supposed to know quite a lot about lean development — we’ve even written books on the subject. So the idea that we had managed to get to product launch with so many problems for customers was kind of embarrassing.

But it wasn’t as if we had never tested the game before! We hadn’t gleefully designed without customer input! We had been talking to dozens of users and players and watching them. We knew what many of the problems were… and we’d tried to fix them.

In our urge to be lean and get an MVP out into the market-place, we had tried to rush through quick and dirty fixes. Those fixes, from tutorials to on-screen messaging, had simply not worked. Our learning curve was still too steep, our UX still too complex. Now we would need to find new solutions to the problems …

It doesn’t get easier

It wasn’t long before we had brainstormed a range of possible solutions. We were keen not to make the same mistake and put weeks of work into building something that might not solve the problem. But how would we test our next batch of solutions without investing the effort in building the code?

Four weeks after our first paying customer, we felt further off our second than ever.

If you enjoyed reading this, then please recommend, follow us @gamevy, or — hey — even see the problems for yourself by playing https://apps.facebook.com/blackjack-attack/ Any comments are always welcome!