How to solve a problem and get fired at the same time
A cautionary tale of the ego and game product management
My greatest professional achievement as a Product Manager was also a commercial failure. I successfully identified the root causes of a terrible drop in revenue, but in doing so I was labelled unprofessional by my client because of my methods. This is the story of a battle between stubbornness and wisdom. It was July 2018; I was working on a Sunday with my partner at Digi Learnnials when our first big client called. He said, “revenue dropped 30 points and we think it is because of your update”. We were being accused of uploading a feature deficient enough to break apart the business model that we were supposed to be improving.
There are three things you need to know before continuing reading: First, the client did not want to share ad revenue metrics with us and it was stated in a clause in the contract. We only had access to Firebase Analytics custom events to analyze game design features. Second, Google launched a new feature called “Broad Core Algorithm Update” to their Play Store search algorithm. Third, around the same time, the client kept introducing changes without us knowing. It was also the end of summer holidays in the Northern hemisphere.
After receiving the shocking news about the revenue drop, I felt upset, like I wasn’t up to the task. But, before throwing in the towel, my ego stepped into the game and said “Nope. I am not a failure. Our work isn’t to blame, I just need to prove it.”.
When you let your ego take control only bad things happen, trust me. That was the one and only mistake I made and after that everything went sideways.
I went straight to Firebase Analytics and got some insights:
- the Indian audience increased by 6 times.
- Retention D1 was 50% better after the update.
- DAU increased by 40%.
- The United States audience dropped by 35%.
In order to understand the issue I asked the client to provide me with the eCPM metrics, data that I was not supposed to get hold of. They didn’t realize that with that input and a few custom events I could estimate ad revenue. This last piece of information was the key to the solution: the United States eCPM was more than 30 times that of India. My conclusion was that at the end of summer holidays and as a result of the change in the Google Play algorithm, the game’s audience shifted from the United States to India. So, despite the increasing number of users, they just did not fill the eCPM gap.
But the story doesn’t end there. The game’s development team launched a fix for General Data Protection Regulation (GDPR) that turned Firebase Analytics off before the user accepts Terms and Conditions. Hence, retention increased by 15 points but only because of the new fix.
The fix modified the amount of players measured during the onboarding process, but the amount of players on D1 on the organic acquisition flow did not change. The “first_open” event was triggered at a lower rate even with the same amount of new users. In short — the increase in retention after the roll out was not a real improvement.
So, the game had more Indian users and less users from the United States, in addition to a really low eCPM in India. At the same time, retention metrics were shaky because of the GDPR fix. I called the client, I presented the charts, I explained the four isolated events that ended up creating a rare situation that resulted in a revenue loss of 30%. In conclusion, I proved that the problem was not our update. A total victory! Or so I thought…
I was free of the charge of breaking the business model apart but now the client disapproved of our unprofessional behaviour. I asked for data that I wasn’t supposed to have and after that call, they lost confidence in us and decided to end the contract. My responsibility as a professional is linked to both finding the solution and respecting the contract, but I was so blinded by my ego that I forgot about that clause.
Looking back, I would advise my younger self to avoid making that call because it wasn’t my job to analyze that aspect of the business. But back then I let my ego take over. I needed to prove to everyone, including myself, that I was not to blame. If something similar were to happen again, I would still try to get to the bottom of the problem, but I would clearly lay out the preconditions to do so to ensure the client is on the same page as me.
Roy Magariños is co-founder of Digi Learnnials, a games consultancy based out of Buenos Aires, Argentina. You can connect with him on Linkedin here.
