What Happened to the App Store Algorithm on August 27?
On August 27, MobileDevHQ noticed a big change on the App Store algorithm. Lots of devs saw their number of downloads dropping. What happened?
This article was originally published on the WordData Blog.
The last days were not easy for the faint-of-heart developer. Nothing ruins a dev’s day more than watching your apps suddenly losing rank positions and a significant number of downloads.
iOS 8 is coming, and thus we’re living in a time of change. Apple says this OS update is developer focused, and includes big changes on app discovery.
What wasn’t previously disclosed is if Apple was planning any big changes in the App Store algorithm. With more than 60% of users using search as the primary way to find new apps, any algorithm change deeply affects the number of downloads and ranking positions of any app in the store.
As Ned Stark was so fond of saying — before they lopped off his head — Winter is Coming.
How It Began
MobileDevHQ spotted a big change of app ranking positions with its Sonar tool on August 27. Although not an algorithm metric per se, it’s very strong proof that the algorithm had some kind of update.
A couple of days after the spike, user “raymng” posted a new thread on the iPhoneDevSDK forums speculating that this change was related to the weight keywords found in the app’s name in the algorithm, suggesting that the bonus Apple gave to those keywords no longer applied — or worse, that they were given even less weight than keywords from the keywords field.
It made sense: In the last months, Apple was getting less and less flexible in app names. Long app names were almost certainly rejected, and the minor suspicion the reviewer had that spammed keywords in this field could cause the same result.
Other users quickly followed “raymng”, sharing their own app search results ranking experiences. Keyword search results rankings changed in an almost random manner; no obvious pattern was found, since there appeared to be a balance between keywords in which their apps search results rankings had increased, decreased or not changed at all. Some old (even abandoned) apps had a resurgence, too; jumping several positions without any kind of paid advertising or update.
Some users started arguing that the algorithm could now be penalizing longer app names, but there were some weird cases like that of “hogie1418” that could debunk this hypothesis: One of his apps dropped 100 positions in the search results for the only two words contained in the app name, while positions on keywords from the keywords field remained unchanged.
There wasn’t a consensus on how the updated algorithm was working other than being app name based. The last hypothesis considered that keyword relevance might be a variable, somehow, with the algorithm giving more weight for core functionality keywords. But, if there’s one thing that the devs could agree on, it’s that the number of downloads mostly decreased. Which pissed them off, as you might imagine.
Our Quick Experiments
My apps were clearly affected by the update because like most developers, I had negative results when it came to download numbers. So, to better understand what’s happening, I got my team to run some tests and discover any possible pattern in these changes.
We decided to test current hypotheses first. First, is this whole change app name related?
We’ve picked one-half of our apps and started comparing app name keyword search results ranking changes between them on App Annie, looking for any kind of pattern. Unfortunately, nothing relevant was found.
For 42% of the app name keywords, our apps ranked lower after the update, while they ranked higher on 51% of keywords. If we cut any changes below 10%, focusing on changes that were certainly caused by the new algorithm, the number went to 27% for lower search results rankings and 21% for higher. These numbers can’t confirm the hypothesis that the only change was the decreasing of weight app name keywords have on the algorithm.
What about other apps? Maybe we’ve just got lucky.
We’ve chosen 16 more apps from very different overall ranking positions and proceeded with the same test we’ve done on our apps. The result was quite similar. These apps ranked higher for 32%, lower for 36%, and kept positions for 32% of their app name keywords.
We looked back on our apps and began analyzing the relevance of every keyword. Maybe Apple was indeed using relevance as a new variable and giving extra weight for more relevant keywords. But, as with other tests, there wasn’t a pattern to be noticed, with core words increasing and decreasing search results rankings as if they were randomly chosen by the algorithm.
So, we took a step back, looked at the problem and started wondering: What if the sudden drop of app downloads wasn’t related to app search results rankings at all?
A few weeks before this whole mess happened, we noticed something very strange regarding keyword localization: Not all English localization keywords returned our apps in the search results outside the country.
Usually, an app will rank for keywords found in the English metadata on searches from every country’s App Store. While searching for blacklisted keywords after one of our apps got approved for an update by Apple, my team noticed that the app wasn’t ranking for English keywords in New Zealand. We’ve found it weird, since it never happened before, but thought it was some kind of bug and left it as it was.
So, after the last test returned no plausible conclusions, we decided to go back to this blacklisted keywords test and scale it up. We picked some of our games and started searching them with keywords from their English localisation. All of them were found, except for one app, which was ranking on Brazilian and Greek App Stores for only five of the English keywords. We compared localised vs. non-localised languages and yet, the results were the same. Once more, although strange, this information wouldn’t enable us to reach a conclusion.
So, What’s Happening With The App Store?
It’s not looking as simple as “keywords on app name don’t work anymore”. This was probably a more complex fine-tuning, making it very difficult for a single app developer to find patterns. Also, it’s difficult to determine what really happened because Apple could have changed several algorithm variables at the same time. Maybe this update is related to the Chomp search engine acquisition two years ago, who knows.
Gabriel Machuret, an App Store specialist, commented on the August 27 update. He believes that app ratings are driving the ranking changes, noting that apps with a large number of user ratings, usually 500 or more, have actually increased their search results positions, while smaller apps suffered huge drops.
Another interesting fact is that, on September 2, Apple launched a new App Review guidelines page and, prior to that, a new page explaining the most common reasons apps get rejected on the App Store. Adding this to the algorithm change, makes it clear that Apple is indeed paying special attention to the App Store for the upcoming iOS 8 launch, shaking things up a bit.
Meanwhile, we keep waiting, hoping that the changes will, at least, lead to a better app discovery.
Learn more about App Store Optimization (ASO) on the WordData blog.
Enjoyed the post? Hit the ‘Recommend’ button below and spread the knowledge!
