Curating Our Appstore: What Makes an App Great?
Whether you develop for PC, mobile or consoles, you’ve had your fair share of dealing with appstores. Endless lists of games, apps and products competing for gamers’ attention, time and money in a struggle to stand out from the crowd.
How do appstores choose which app to feature or promote, and which should be dropped off of the store? In this article we’ll shed some light on our own curation process and explain the main factors driving our decision to allow or disallow an app in our store. Let’s dive in!
Appstores try to strike a balance between inclusiveness which increases the variety of content in the store on one hand, and quality as measured by a set of basic requirements and quality standards on the other hand.
When choosing a content curation policy for an appstore, some take a more hands-on approach in ensuring that the content published on the store is of high quality — great examples for that are Apple’s Appstore and Facebook’s Oculus games store. Other stores take a more lenient approach, setting a minimal standard for apps to be published and counting on the community to lift the good apps up and ignore the less valuable ones. Google’s Play Store and Steam are great examples of lenient appstores.
The Overwolf Appstore was introduced around 2015 as the first of its kind for PC game apps. Over the years, over 450 apps were submitted for publication — some amazing at launch, others still in need of work. We had to choose — apply a hands-on review methodology, or adopt a lenient “community filters the good from the bad” approach to curation.
Back in 2015 many 3rd party apps around games were not in tune with game developer ToS and perceived to be bloatware, cheats or malware. We knew that the only way to offer true value to gamers is by ensuring that the content offered on the Overwolf appstore only includes high quality apps — no scams, cheats or bloatware.
We therefore chose an active hands-on approach. However, the curation quest is never done, and in order to drive constant optimization of the store and the apps therein, we’ve had to develop a new set of tools and criteria by which we prioritize or remove apps from our store.
Value First
The most important way to filter apps trying to get into the appstore is checking whether they do what they claim they do, and whether these features matter to gamers using it. In general, we want to see diversity and usefulness, so we try to select new apps with unique value propositions for gamers which are different from existing apps in the store.
Before launching on the Overwolf store, new apps go through a review process conducted by our team. These early reviews focus on the app’s concept to get it pre-approved for development. Later on, the finished app is reviewed and tested in a live game environment to ensure the app does what it’s meant to do and nothing it shouldn’t be doing.
In some cases, if still uncertain, we will share the app’s beta version with testing groups of dedicated gamers who know that particular game best. Feedback from gamers with lots of in-game experience goes a long way in shedding light on app value as well as on issues we want to fix. It also helps to implement analytics (such as Amplitude or Mixpanel) and track whether gamers come back to the app or explore advanced features.
Staying Between the Lines
Another important aspect of protecting Overwolf users from bad apps is compliance, and specifically ensuring that an app will never hurt gamers’ account safety or standing when playing their game.
The most important step towards that end is full compliance — every game developer has a set of terms and conditions detailing what is allowed to do with their game. Any apps breaking or bending game Terms of Service will not be allowed on the appstore, or be immediately removed from the store as soon as we’re aware of a breach.
We stick to these publisher guidelines at all costs because Overwolf apps could not exist without fantastic games lovingly crafted by the developers; respecting their hard work as well as the rules they chose to set is step one.
Beyond that, it’s all about communication — when there are exceptional apps that need particular approval, we talk to that game’s publisher and either get approval or remove the problematic app or feature immediately.
Performance Transparency
One more metric which helps us curate and select apps for our appstore is software performance. Being a platform for game apps, we have to be able to run an app or several apps along with the chosen game while keeping game experience untouched.
Needless to say, creating software with 0 resource requirements is impossible, but we test incoming apps for resource efficiency and game performance. As long as the app doesn’t drop the game’s frame rates in a noticeable way, hurts network response time, or badly handles storage space (for example when managing video highlight saving), we’re fine with it.
Beyond ongoing performance, we monitor apps for crashes and technical issues. If we spot a negative performance spike, we investigate and try to find a solution with the app’s developer. Since it’s pretty common for game updates to temporarily break events and require hotfixing, this is something we do pretty often.
Our monitoring tools are constantly being upgraded, and we plan to establish more automated controls down the road. In the future, any spikes in CPU, GPU, network or RAM usage will automatically alert users, close faulty apps and prompt Overwolf to investigate.
Live Curation — The Voice of Gamers
As much as we like, we can never know everything that happens on the appstore, especially with live apps being updated constantly. Therefore, the feedback and preferences communicated by gamers are perhaps the most important tool in the curation toolkit for real time monitoring.
The Overwolf community of gamers often raise issues such as an app having bad performance, a terms of service compliance breach, or just bad UX. That feedback is then shared with the app’s developer along with our recommendations and support for fixing it. Honest user feedback shared by our community helps immensely in making our appstore better with each passing day — and we’re extremely thankful for it.
Retention: The Ultimate App Value Metric
Beyond basic performance, retention monitoring is our most important tool for checking which apps are doing well with gamers. Retention is the percentage of users which come back to use the app after trying it for the first time. For example, if 100 users installed the app on day 1, and 50 kept using it the week after, we can say the app has a 50% 2nd week retention.
In Google’s mobile Play store, for example, over 77% of an average app’s users leave within three days, and fewer than 5% stay for longer than three months with a given app. The most successful 50 Android apps out there retain around 40% of users after three months, giving us a good example of what good retention looks like.
In the PC ecosystem, we found that if more than 50% of gamers who try an app uninstall or stop using it shortly after installing, the app’s retention is not good enough and requires our attention. On the other hand, seeing 65% or more of an app’s user base sticking with it for weeks or months after installing is the strongest evidence that an app is great.
As mentioned earlier, curation is a never ending journey, but so far we’re happy to see great results on the Overwolf appstore, with many apps meeting the 65% retention threshold — going toe-to-toe and even beating the top 50 apps on Google’s Play Store.
Evolution of Ecosystems
To summarize, curation of content on our appstore involves a mixture of analytics, direct feedback, and a lot of hard work with the great partners who publish their apps on the Overwolf platform.
The gap between objective performance and actual user experience is bridged by softer analytics like usage time and frequency, as well as by gamers sharing experiences in their own words sent to us as feedback.
Evolution is driven by constant change, and for us the best possible outcome of curation is not just removing a faulty app from the store, but working with developers to improve their apps to the joy of gamers on our platform. Our team has seen hundreds of great (as well as not-so-great) apps, and we’re ready to pitch in and help developers with user experience, technical performance or look’n’feel as needed.
Are you thinking of building an app around your favorite PC games? We’d love to hear your ideas and help you turn them into a live app that provides great value to millions of gamers worldwide!
