3 major bugs in Pokémon GO — and what we can learn from it…
No question: Pokémon GO is a really compelling game that proves to reach many generations of users. I involved myself early-on with the game because I had three reasons:
- Yes: Like many other I just was curious because of that big hype swapping from the US over to Europe two weeks ago.
- My both kids (girl/8 and boy/15) and lots of their friends were “infected” by the game. It gave us a new “platform” to have some fun together chasing little monsters, breeding their eggs — and (my hidden plan) talking about the technology behind all that hype…
- My company is deep into creating advanced mobile apps for well-known customers, now also moving into development of AR apps and mixed reality apps with the Hololens. Meaning: I was professionally interested.
So yes: Pokémon GO is a great game — no question. Yet it has some really basic bugs I’d like to point out.
Bug No 1: Energy consumption
The app really comsumes a lot of power — even if you are not actively chasing little monsters. My iPhone 6s plus lost 1% of its power every 3 minutes when I just put my phone on a table with the app running in the foreground and no interaction. That’s far too much power drain. Go out, catch some pokémons and you’ll easily drain your whole battery in 2–3 hours on an average phone. That’s not only inacceptable for the users — that also can’t be the intent of the app creator Niantic. My kids really get frustrated by this.
From an engineering perspective this is a bad sign regarding test practices at Niantic: Field tests — even random ones done by the devs of the app themselves — should have shown this basic issue immediately.
Bug No 2: Hang ups in energy saving mode
Pokémon GO version 0.29.3 has an energy saving mode, that can be activated in the settings of the app. The app checks the position and tilt of your phone and overlays the screen with a nice dark image if it thinks you don’t look at the screen.
That feature really helps with the energy problem — yet creates a bigger one: Doing this several times you won’t be able to interact with the app anymore. The app seems to run normally — you can see pokémons and pokéstops — but the app doesn’t respond taps and swipes anymore. All screen input is dead.
Another bug that should have been found easily — again not by automated tests, but surely in a field test.
Bug No 3: Hang ups and connection problems when entering/leaving WLAN zones
Bug No. 3 happens if you start the app in a WLAN zone e.g. at home and then go out in the wild. Sometimes you’ll notice that you won’t see any pokémons despite the app seems to behave totally normal. Kill the app and restart it and the picture will change: you’ll see little monsters all around you.
It seems as if the app registers for server notifications of certain events that won’t get triggered anymore if you change the underlying network connection. Technical reconnects seem to break the logical connections. That should never happen or at least be prevented by time-outs or heart-beat checks and corrective actions on application level.
This bug is even more annoying if you hunt for a pokémon in 3G/4G mode and enter your home WLAN zone during the catch. The app seems to check back to the server, if you really should catch the pokémon successfully or whether the little monster should break free: this conversation between app and backend service seems to break when switching between different networks and the app freezes totally. Only way out: Kill the app, lose your catch. Bad!
Again: This last bug is hard to reproduce with automated tests in a test lab, but very easy, if you closely watch test persons playing the app — or if you have the right telemetry applied to track app crashes in the field.
What do we learn?
Pokémon GO isn’t a great app from a technical point of view. It has several, annoying bugs that badly influence the user experience. But: Even these bugs don’t seem to influence the massive hype around the app. The fun playing that game outweighs its technical limitations.
This should be a very interesting finding for many software engineers: it is very hard to find or even predict the right balance between putting effort into creating value(fun) for the user and putting effort into the right level of product quality. I think two or three more annoying bugs of this category could have totally changed the success of this huge project!
I’m sure Niantic will fix these issues in the coming weeks. This will give us all even more fun with Pikachu and Eevee! ;-)
