Benchmarking Social AR Success
How Pretty Big Monster Evaluates the Effectiveness of Facebook & Instagram AR Effects and Snapchat Lenses
On a recent Social AR campaign, it appeared our client had a verifiable viral hit on their hands. Hundreds of thousands of organic impressions had been delivered over 48 hours. They were excited. We were excited. Everyone was patting each other on the back.
Then the senior stakeholder asked, “but how does it compare against benchmarks?”
We had no idea.
No one had ever asked us that before.
Google yielded nothing helpful. Reaching out to the platforms themselves didn’t get us much farther. The response we received was:
Each case is unique depending on the audience it’s reaching, the goal of the campaign, the market it’s in, etc. The best way forward is to track benchmarks per client and create a business-specific baseline for them.
~ Snapchat Platform Representative
While there’s truth to that, the same could be said for any marketing benchmark, yet they are still published. A benchmark that spans clients is still worthwhile. So we took matters into our own hands and established the first Social AR benchmarks (at least that we’re aware of).
Our goal: a benchmark for measuring AR lens effectiveness, independent of platform, volume of impressions and whether paid media was used.
Cross Platform Metrics
When we started Pretty Big Monster, we surveyed the marketplace and noted that many agencies created lenses that worked either on Snapchat or Facebook and Instagram but not both. Due to the differences in policy and scripting languages, often what’s possible on one platform is not on another. We made the decision early on to differentiate ourselves in Social AR by always proposing lenses that worked across all three of the major Social AR platforms; Snapchat, Facebook and Instagram.
This strategy has its limits, but the ability to reach the broad range of audiences of each of the social networks with a single asset is valuable to our clients. However, it’s not just the platforms’ content policies which are very different, the reporting varies as well.
Snapchat reporting includes:
- Views
- Plays
- Shares
While Facebook and Instagram include:
- Impressions (analogous to Snapchat’s views)
- Opens (analogous to Snapchat’s opens)
- Captures
- Saves
- Shares (same as Snapchat’s shares)
Impressions and views, while important for the overall campaign and necessary for benchmarking, were less useful in evaluating individual lens performance. Impressions can be boosted with paid media or supercharged by influencer usage. We wanted our benchmarks to be useful regardless of the volume of impressions or views.
Because Snapchat does not provide reporting on Captures and Saves, Plays/Opens and Shares were left as the common metrics available to measure across platforms.
To ensure that we could compare lens effectiveness regardless of the number of people who engaged with the lens, we looked at the open and share rates rather than the absolute numeric volume.
The equations we landed on are:
Open Rate = # of shares / # of impressions
Share Rate = # of shares / # of impressions
With each expressed as a percentage, we arrived at common benchmarks with which we could judge a single lens on a single platform, or a single lens across platforms, or roll up all of the lenses across all platforms and have a common metric for evaluation.
Arriving at the Benchmarks
Now that we knew what we were looking for, it was time to pull the data together. We looked at metrics from across all the campaigns we’ve created lenses for. The final tally was data from more than 100 million delivered impressions.
100 million+ impressions evaluated
Some lenses had been promoted with paid media, some ran only a short amount of time. Some had run only on one platform, but most ran on all three. Because we were only looking at Open and Share rates, these variables didn’t matter. We had everything in place to create our benchmarks.
The Benchmarks and Platform Insights
Once tallied and graphed, we not only had our benchmarks, we had an insight into the performance of the major Social AR platforms.
Snapchat had the highest Share and Open Rates. While Instagram beat Facebook on the same counts. It should also be noted, that in apples to apples comparison, Facebook routinely had the lowest impressions delivered.
Snapchat
Open Rate: 58.0%
Share Rate: 2.4%
Open Rate: 30.9%
Share Rate: 2.1%
Open Rate: 46.8%
Share Rate: 3.0%
Conclusion
We now had an answer for our client. Not only was the campaign a viral success, we could confidently communicate it beat the benchmarks as well.
Now, whenever Pretty Big Monster completes a Social AR campaign, the data is added to our benchmarks to keep it evolving and reflecting current consumer behavior. Next, we hope to issue more precise benchmarks by vertical as well as how to optimize lenses to ensure they perform well against the benchmarks.
We are interested in comparing notes and discussing methodology with other Social AR creators and marketers. Please feel free to get in touch with any questions or to discuss creating a campaign for your brand or product.
