A Peek into the Pokémon Go Hacking Scene

I’ve lately been closely following the hacking scene for Niantic’s Pokémon Go, and it really has been a fascinating experience. The hacking endeavors have been very public, with the hacker community using open source GitHub projects and using a public chat server on Discord. So the cat and mouse game between Niantic and the hacking community has therefore been mostly very public for anyone to observe. There is a LOT we can learn from this exchange, no matter which camp you belong in. I’m going to share some aspects of the journey, but out of respect for Niantic, I’ve severely limited the amount of links or images to support my words. And for those in the scene, I apologize for the various simplifications.

When Pokémon Go launched, the game was an instant sensation, breaking app launch records. There are various reasons for its success, including nostalgia, Pokémon cartoons, augmented reality, virtual rewards for physical activity, etc.. The end result is an app that makes people have to get up and move around to catch Pokémons, and then move around some more to hatch Pokémon eggs (there are 2km, 5km and 10km eggs, yielding increasingly better Pokémons). And different regions of the world may have different types of Pokémons, causing some people to make it their full time job to scour the for all the Pokémons in the game.

But this also created a strong demand for cheats. Is there a way we can hatch the eggs by not having to walk around? Can we spoof the GPS to make it think we’re walking in remote places of the world? Can we somehow scan all the Pokéstops in the world so that we know which route would maximize the number we encounter on the way to work? Are there some rare Pokémons nearby that I can just do a quick bike ride to catch?

These questions lead some hackers to try to do a simple MITM attack (MITM = Man in the Middle). And it turned out that the game was very open to that attack. The way it works is that you install a special software (that I am choosing not to name) on your computer that can sniff all network traffic. It even exposes an ip address and port that your phone can connect to. Once your phone is connected, all traffic in and out of your phone is captured very neatly in that software.

When you try this with Pokémon Go, everything is encrypted via SSL. So, if you’re not the NSA, how can you decrypt it? Well, all you have to do is install a special root certificate on your phone that is associated with the software, and voila, all the traffic is now visible in clear text!

“Wait…”, you might ask, “what is the point of SSL if its this easy for any hacker to see all the traffic?”. Well, it turns out, that while SSL itself is pretty secure, its all about what you’re using SSL for. The most typical use-case for SSL is to protect user data from external sniffers. That use-case is not designed to protect the user from themselves! In this case, you are the user of your own device, and you are sniffing traffic that an app on your device is sending outbound. And in the case of the Niantic app, they were only using SSL in a way to prevent external sniffers, not the case where the user herself is doing the sniffing.

Now that the traffic is open, the next phase is to try and understand the traffic. It turns out that Niantic uses Google’s Protobuf, an open source library for compressing/decompressing data into or out of a binary form. Protobuf is well documented, and as such, the traffic sniffing software has a nice mode where you can view the traffic in protobuf form, making it really easy to structurally see what is being sent and received.

The next step for the hackers, was to create their own ‘game’, aka, a ‘client’. The ‘client’ basically tries to mimic all the http calls that the hacker sees in the sniffing software from the actual game. Basically, mimic it 100%; every http GET and every POST. The goal is to make the server believe that this custom client IS the game itself. And it worked! Now that they had their custom client, the next step is to try tweaking things, seeing the results of tweaks, putting in custom values etc.. Eventually, they end up with a very strong understanding of the protobufs, so much so that they documented their entire results as a simple-to-use open source library called the pgoapi (Pokémon Go Application Programming Interface). One of the most popular clients was one that basically showed Pokémons in a large radius around a certain location.

But some client programmers realized they could do MUCH more than read game data. They discovered they could even write game data. i.e., the Server really thinks that this client is a game, and so the client can do everything the game can do, including, looking at the items in your bag, clicking on a Pokémon, throwing a Pokéball at it, even applying spin on it, make judgements on what Pokémons to capture and what not to capture, spin a Pokéstop, etc.; basically the client could do EVERYTHING the game can do. And in some cases, it could even do more than the game allows. Additionally, they could start their ‘bot’ anywhere in the world, and as long as they behave similar enough to a human, Niantic may not be able to tell. The result, is that bot users started registering hundreds of bot accounts, each one walking the world on their own and collecting Pokémons, reaching really high levels in extremely short amounts of time. And some of these rogue bot users would even go on to sell their high level bot accounts on eBay for humans to use!

Meanwhile, all this activity was costing Niantic a lot. The server load went up like crazy, and their game, which was never really that stable to begin with, was becoming unusable and their ambitions, such as expanding to Brazil, had to be put on hold because they simply could not sustain the bandwidth they were seeing. Apart from server load, the bots themselves were causing the game experience to deteriorate for the regular humans, as their powerful Pokémons would dominate the Pokégyms, and it was becoming too hard for humans to defeat the bot accounts. So, something had to be done. It really became a matter of survival of the game.

And so Niantic responds. The result was soooo effective, that it completely stopped ALL bots and hacks of any kind. Here is a graph of the impact on their load from Niantics’ blog:

Boom! That looks to me like about 75% of their queries were coming from from the hacks. Most of that was probably the bots rather than trackers. So, what did they do? Well, this is what the hackers needed to find out to get their hacks working again.

So they again start with their MITM attack on the client that Niantic released on July 30th. But this time, something was very different…. The game would not even start whenever they were using their traffic sniffer! What Niantic did, is that they implemented something called ‘Certificate Pinning’. Basically, the game will now check that the SSL is being issued from a certain specific root certificate. It will not accept any other certificate, and just stop the game from launching. Now, the hackers do not posses that certificate, so they have no way of bypassing this.

But hackers like to hack. And hack they did! In a blogpost a hacker explained in a very visual way exactly how he bypassed the certificate pinning. Basically, the certificate pinning is client-side code which checks for the right certificate, and if it is not correct, it blocks the game from starting. So, the hacker explains how he found the IF statement, and then what assembly-language instruction you need to change to skip the IF. The blog guides the reader to use any hex editor to make a minor modification of the Niantic code. The reader can then package it all up again and install that customized version of the game on their device and now MITM works perfectly!

What the MITM showed was that Niantic had changed their protocol (protobuf). So the hackers set about trying to figure out what changed and begin updating their open source projects to document the changes, and then trying it out. But after incorporating those changes, the Niantic’s servers were rejecting the api requests! And try as they may, no one could figure out what’s going on. A group of hackers decided to really up the ante, by moving to a real-time chat platform called Discord and opening up the discussion to the world. Hackers from all over the world started coming together to discuss the issue, and they found some fascinating things:

1. Niantic was generating some complex checksum and sending it as part of the protobuf request. The checksum takes in some inputs and has some funky algorithm that combined all the inputs together, adds some salt, and turns it into some irreversible sequence of bits. The server has the same algorithm and knows the same inputs and produces an output, and it compares its output with the checksum that the client sent, and it MUST match. The original game will have the right checksum. This checksum was stored in a protobuf the hackers had originally named ‘Unknown6’ or ‘uk6’.
2. They spent a ton of time trying to diff the latest client with the previous client to find the code that changed that may contain the algorithm that generates the checksum. But to their utter surprise, this checksum algorithm has been there from the very beginning! For some weird reason Niantic hadn’t been using the checksum earlier. All Niantic did was just turn on the check from the server side; there was absolutely no change to the client on this aspect. (More thoughts on why come later in this article).

So now, if the hack was to start working again, they had to figure out how Unknown6 was being generated. And so they started a massive hunt. It involved basically getting a lot of hackers to work together to function-by-function convert portions of the assembly code into (slightly) more understandable C code. They needed to parallelize many of these tasks and they found a ton of eager souls willing to help. Most of these people, however, were not hacker material, and it was a significant challenge for the ‘admin’ of the Discord server to organize the chat room into something productive. He broke the channels up into a private hidden one for certain well-known hackers, a channel for the vetted hackers that only they can chat in but the world can read, a restricted channel for non-hacker developers, and a channel for the general public to chat in. There was also a reddit page that continuously showed the latest status on the development of the hack. The admin set up a promotion system that would allow individuals to rise these ranks, but the goal was always to get the hack out as soon as possible with minimal disruptions and distractions, so promotions were really based on the ability for an individual to provide tangible benefit to the hacking effort.

The first major breakthrough was uncovering the entire encryption algorithm. This was a pretty impressive achievement that was primarily done by the vetted hackers. A parallel effort by the non-hacker developers was to try to figure out the inputs to the checksum function. This was very non-trivial, as it involved playing with various other fields in the protobuf, such as the tilt of the accelorometer, to see if it impacted the checksum or not and in what way.

And within a matter of 4 days, they did it! Obviously this was a big deal and there was a lot of celebration and the new protocols were published and the hackers have begun work on updating their client. They then created new channels for various programming languages (as well as human languages), and the distributed work has begun for various world-wide hackers to work on building clients for many programming languages simultaneously.

Yesterday Niantic released another version of the game, and hackers have already identified certain protobuf changes. It is not yet clear whether there will be some server-side update that again blocks the hacks based on something new in the latest client, but we can be sure that if that happens, the hacking group will be on it. And most likely, this time the work the hackers would have to do will be much easier because now the approach of doing a diff between the latest and former clients will likely yield more meaningful results than last time.

We can expect this cat and mouse game to continue, but one thing that really confused the hacking community is why Niantic didn’t enable the server-side checksum validation from day 1, especially given the fact that the checksum was already being produced and sent by the client from day one? If they had done so, it is very possible that the hacking scene would have been quite different since they wouldn’t have had a working frame of reference to compare with. My personal opinion is that this was developer error and lack of foresight, but a far more popular (conspiracy) theory is that Niantic did it on purpose: they know 100% which clients were fake and which were real by checking if the checksum is valid or not. So, given that, they probably accumulated a large amount of data on exactly how bots behave and exactly how humans behave. Now that they have all this data, they could put a Machine Learning expert on the job to figure out how it can determine who’s who by doing behavioral analysis. Support for this theory is on their jobs website:

If true, that’s cool, if not, they might as well make it true.

Meanwhile, Niantic has begun attacking the problem from a legal perspective. They have shut down bot sites via a legal proceeding called a “DMCA Takedown”. They have also shut down many non-bot map hacks as well. And they have also also issued Cease & Desist orders to key members of the hacking community. All this hasn’t seemed to dissuade the hacking community though. The hackers seem to continue doing it, and are trying to mitigate things by not exposing any copyrighted material along with making explicit disclaimers that they do not condone any illegal activity. Whether Niantic chooses to target the Unknown6 crew with legal proceedings remains to be seen.

One key learning here, is that while it may be inevitable that a system will get compromised, it is important to make it take as much effort and time as possible. It is critical to be secure from day 1, so that hackers don’t have any frame of reference to work with. Things like doing a checksum validation from day 1 would have likely significantly delayed the state of being perpetually hacked. And clearly SSL encryption is not enough! Apps really should be using certificate pinning to mitigate MITM attacks, and they should assume that the MITM will be successful and so they need to be thinking about what hackers will see. Using Protobuf seems like a disadvantage because the sniffing tools know it so well that they are able to create beautiful visualizations of it. So it would be a good idea to perhaps have another layer of encryption there so that the protobuf is not directly visible to such tools, and/or consider other not-so-popular serialization techniques.

One fascinating insight for me, is that the real ‘hackers’, the ones that actually did the hack, are generally people that consider themselves do-gooders. The admin of the whole Unknown6 effort, keyphact, says this: “We do not condone or support hacking/botting services. Many people seem to forget this. If you ever visit us on our Discord server (which you should), you will notice that our moderators remove any bot discussions quite swiftly.” And I have witnessed this to be true. Their underlying motivations may vary, including learning, teaching, exploring the unknown, getting that ‘ahaa!’ moment, solving a puzzle, helping themselves catch more Pokémons, being ‘philanthropic’ by helping other people find Pokémons, and yes, perhaps some may even harbor hidden intentions of making bots, but they certainly had very strict “do no evil” set of rules that they enforced swiftly and efficiently. In many cases I noticed that this community harbored direct ill-will towards bot developers that they blame for not only causing Niantic’s ire on the whole hacking community, but also for stealing credit for their work, which I found fascinating. For example, the actual Unknown6 team that did the hack make it very clear that they do NOT accept any donations; to quote one of the hackers, MaxLeiter: “Regarding donations — while we appreciate the thought, they’d only lead to problems. Please consider donating to a local charity instead!” Compare this to a bot website (not sharing link) that encourages donations so that you have “the good feeling that you’re helping the community”.

Another extremely positive thing I noticed was that there were so many young consumers of the hacks that were extremely inspired by the hackers, and for many this public discourse exposed them to a fascinating new world leading many to ask how they can learn programming and become hackers. For example, as I write this there are about 40 developers online, but 4000 observers to the chat! So this Pokémon Go phenomenon extends in yet another dimension no one probably ever expected to. This is generally a good thing, as the skills they may try to learn to become hackers, will certainly improve their chances of getting into good universities and/or getting better jobs.

Regardless, Niantic does seem to understand this dimension to an extent, but clearly cannot condone it. This is what they have to say to the whole hacking community (fyi a ‘Trainer’ is a Pokémon Go player):

“It’s worth noting that some of the tools used to access servers to scrape data have also served as platforms for bots and cheating which negatively impact all Trainers. There is a range of motives here from blatant commercial ventures to enthusiastic fans but the negative impact on game resources is the same.”

Hopefully the hackers get this point, but knowing what I know of them, they will ignore the ‘negative impact on game resources’ argument. So, what can Niantic do? Here are a couple of my suggestions for them:

1. Continue playing the cat and mouse game. Its important to try and continuously make the hackers have to react to every update. Any periods in which the hacks are broken is good for Niantic, as it will make them flaky for end users to consume, which may eventually lead to a reduction in demand for the hacks.
2. Improve the game mechanics to reduce the desire for a hack. One of the biggest pains has been the lack of a good way to explore and find Pokémons in the game. If Niantic can come up with a solution that makes the exploration aspect really fun, then it would reduce the desire for a pre-explored map hack.
3. They need to add a more skill in the game. For example, a Clash of Clans bot cannot really do a good job of actually doing an attack; the game is too complex. It would require something like Google’s Deepmind for a bot to make consistently good attacks on enemy bases. Whereas with Pokémon Go, the skillset required is negligible. Sure it makes the game more accessible to 6 year olds, but it also makes it much more bottable.
4. Invest in Machine Learning to help differentiate bots from humans, and either ban them or apply some severe penalty to make it far less worth it.
5. Realize that at this point, they cannot permanently stop the hacks, and so they really should prioritize shutting down any efforts towards bot development, i.e., be a little bit more discerning on who they take legal action against.
6. Give the hacking community some alternative recourse, by opening up certain sanctioned APIs that they are encouraged to develop apps on. Create a clear policy on what kind of fan content is allowed and give some allowances for monetization. Supercell does something similar with a fan content policy as well as a developer api. This will go a long way in mitigating the desire to reverse engineer the apis, as well as helping foster a educational / transformational role for the game .

I really hope to see this game find a solution, as finally we have a game that will make our kids move away from their monitors and TVs, and hacks that circumvent that really cause consumers to miss out on some of the greatest and most rewarding aspects of the game.