Pokémon Go will be the largest mobile app retention effort ever

7.5 million users on V1. What could possibly go wrong?

Three days. That’s all it took for Pokémon Go to take over our lives and become what already feels like the biggest game to hit the app store in … maybe ever.

According to early estimates, it has been downloaded over 7.5 million times. And you can only get it in the U.S., Australia, and New Zealand. A Canadian coworker told me her friends back home are pestering her for an American iTunes account so they can get their hands on Pokémon Go.

And at its current growth rate, in the next few days Pokémon Go will overtake Twitter for Daily Active Users. Think about that. Twitter launched ten years ago.

You read that correctly: in one weekend, Pokémon Go acquired as many users as Twitter did in a decade.

And that’s amazing. But it’s also absolutely frightening, because what the team behind Pokémon Go is about to go through is a retention effort on the scale that we’ve never seen before.

By the time most apps have as many as users as Pokémon Go, they’ve been available for a while, going through iteration after iteration. The teams behind hugely successful apps have had months or years to learn from early users. They’ve made improvements, fixed bugs, and scaled the backend to meet increased demand.

In the aftermath of Pokémon Go’s wild success, it doesn’t have the luxury to hone the mechanics of the game on early adopters. It’s learning the problems of V1 on a userbase of millions.

With this rate of installs, if it can retain users (and this is a big if), Pokémon Go could become one of the most successful mobile games ever. If not, it could be the next Draw Something—a cultural phenomena, no doubt, but one that came, and then, as soon as the next big thing hit the app store, quietly faded away. It only took a month for Draw Something to go from the top game in the app store to losing more than a third of their daily active users.

And after playing Pokémon Go pretty much nonstop for the past few days, I really don’t have any idea how it will turn out.

I could make a case for wild success or for a quiet fade out. There’s clearly something there. It got me to walk 10 kilometers this past weekend. It got me to use the metric system in that last sentence. A friend told me he talked to more strangers playing it this weekend than he has in two years in the city. Whatever it is, it’s fun.

And it’s addicting. I went out on a walk to catch a few Pokémon during lunch and ran into three different groups of coworkers doing the same.

At Mixpanel, we have a Slack channel devoted to the efforts of those of us trying to catch them all. It’s a hotbed for tips like how best to use a Lucky Egg or where to find a Squirtle (Fort Mason, of course). In there a coworker who will go unnamed shared her rules for not letting it take over her life:

So yeah, it’ll pull you in.

But it’s also frustrating, and not just from the servers that are consistently down from the onslaught of users. The game itself has its flaws. And I can’t help but imagine that these are the types of silly mistakes that they would have solved before growing if it was any other app.

Broken log-ins, system preferences not saving, glitches in battling. Those are the type of typical V1 problems that anyone might expect for a new game, and fix, but they are also the annoyance that will lead to player churn.

But even forgiving the constant bugs and freezes, which could be chalked up to connection issues, and the battery drain, the game has a serious competition problem.

Walking around and catching Pokémon is a good time, but I already feel like somehow I’m levels behind everyone else.

(Probably because they’ve actually bought in-app purchases to upgrade their account, or at least that’s what I tell myself.)

That means I can’t go to a gym to battle a Pokémon without getting my clock cleaned by some kid’s souped-up Arcanine. And short of taking a sabbatical and making an investment in PokéCoins, I don’t know that my little Bulbasaur is going to catch-up.

And so I’m left avoiding gyms and just quietly collecting Pokémon on my own, which without any way to trade or battle with friends, basically eliminates the social aspect of the game.

But that leaves a very one-dimensional game, which, as Draw Something learned, doesn’t have lasting power to retain users.

For another set of expert opinions, I called my teenage nephews. They’re both too young to have gone through the Pokémon craze in the late 90’s, and they don’t live in a large walkable city. And while they told me they downloaded the game after hearing the buzz, they didn’t really see the point of it.

Despite all of the hype, and despite the fact that it took over my life and the lives of my friends over the weekend, I can’t help but imagine a good percentage of those who have installed agree with my nephews. It’s interesting enough, but what’s going to keep them coming back tomorrow and the day after?

And while they’re almost certainly working on those features, the clock is ticking. Novelty and nostalgia will only hold up for so long, and they’re going to have an absolutely monstrous userbase that they’ll be trying to retain.

Hopefully they’ll be able to do it, and build out the game — which, for all its flaws is still a fun application of augmented reality that gets people out and interacting with the real world in an interesting way.

But if not, and if in a few weeks everyone flocks to the next hot app, at least then maybe I’ll be able to win a battle at my local Pokémon gym.