Published in


6 Steps for Successful A/B Testing of Your Mobile Ads

(This article was originally published on Udonis’ blog.)

A/B testing of your ads doesn’t have anything to do with the alphabet. It has more to do with testing two different creatives to figure out what works the best.

Do you know that moment when you have an idea in your head and it sounds great? Well, often, after we’ve executed the idea we realize it is not that great anymore. It happens to the best of us, and there is no certain way of telling if a certain ad will work or not.

The chances are, the one you’ve thought will work the best, often becomes the worst performing ad, and the best performing ad is the one you’ve never thought it will work.

To stay on top of your performance and achieve the best results, you need to constantly measure, analyze, test, and optimize.

Yes, the game itself matters a lot. But it is a bit unrealistic to expect that everything will work from the first try.

That’s why it is crucial to properly track everything that happens inside the marketing campaigns. When something doesn’t go as planned, you’ll be able to retrace your steps and identify why it happened.

One of the best methods to test your mobile ads is A/B testing.

How Is A/B Testing Performed?

You change only one thing in your ad and observe the results. Are the results getting better or are they not going as planned?

This method is also called split testing, since it is used to discovering which element, visual and copy works the best for your target audience.

Knowing who your target audience is is an important step of creating as well as A/B testing of the ads. The same way as you shouldn’t rely on your feelings when deciding which ad is better, the same way you should approach researching the target audience.

What You’re Trying to Learn With A/B Testing of Your Ads?

To conduct a successful A/B test in your game, you’ll first need to have a clear idea of what you’re trying to achieve with the A/B testing.

If the players of your game are abandoning the game just after they’ve seen the ad, maybe the ad is interrupting their experience. There are many reasons why they could be behaving that way.

1. Hypothesis

After you’ve decided what you’re trying to accomplish with your A/B test, try to assume what will you achieve with the new creatives, new placements, new copies… When coming up with the hypothesis, make it as specific as possible and make sure you have the ways of measuring the results.

Don’t be afraid to come up with all sorts of different hypotheses. They need to be tested, and they don’t necessarily need to be proven correct.

2. Segment Your Users According to the Hypothesis

Not knowing and understanding your users is like taking a shot in the dark. It is like making a guess or estimate with very little or no assurance as to its accuracy.

If you have already set up mobile app analytics, user segmentation will be much easier. For user segmentation within your app, you’ll use audiences. Audiences are users that you group together on any combination of attributes that is important to your business. That way you can see the behaviors of different segments of your users. Mobile app analytics include segments — in which users are grouped by the activity, the ARPU, the app version they use, age, gender, country/region, their interests.

You can A/B test almost anything. If you wanted to test which color of the in-app purchase button works better, you could segment the users into two groups and change the color for one segment of the users. In that case, the A group would have the blue in-app purchase button, unchanged, and the B group would have the yellow button you’re wishing to test.

You should always have a control group. A control group is an integral part of A/B testing. The control group is the one for you’ve won’t be changing anything. B group is the one that will use the button with the new color, for example, and the results of the B group will be tested against the performance of the control group.

You can do the same with even more segments, just remember to keep one segment as a control group. If you’re having doubts about how big your groups should be, use a sample size calculator. The more accurate you want to be, the larger the sample size needs to be.

3. Continue A/B Testing

After launching the A/B tests, sometimes things will feel off. The preliminary results could be terrible and your first reaction would be to stop the tests. But don’t be discouraged by the first results, mobile ad networks and platforms need to take time to learn what works the best. You should approach this as a scientific experiment and let the results level through a certain period you’ve designated for the testing.

Different ad networks come with different A/B testing options. Let’s say you have a video ad for your mobile game. On Google, you don’t even have to make multiple videos to test them, you can just try different aspect ratios. The one that works best will then be shown more frequently.

Other ad networks offer interesting A/B experiments as well. If you want to test different creatives on Unity Ads, you will need to create separate campaigns. However, they all share the same budget.

4. Ad Optimization

After A/B testing brings you a winning ad, you can test even further. Create ad variations based on the same ad concept to utilize its potential to the fullest. This also helps with preventing a well-known issue of almost every campaign: ad fatigue.

Creating ad variations means changing small elements of the same ad. Possible variations include:

  • ad dimensions
  • caption
  • CTA button
  • colors
  • backgrounds
  • main characters
  • buttons and boxes

These are just some elements you can test. However, never change multiple elements at once. Instead, stick to one change per variation. This way, you will always know exactly what triggered your ads to perform better or worse.

5. Implementing The Changes

After you’ve successfully finished your A/B tests, you’ll have a clear idea do you want to implement the change or not. It is pretty self-explanatory. Changes that resulted in improvements to your results you’ll probably choose to implement, and changes that didn’t improve anything you’ll probably just want to forget.

After finishing the test, you can start it all over again.

6. Using A/B Testing for App Monetization

In 2018, ironSource started a revolution in the mobile gaming market with the new function of “user-level ad revenue”. Up until that point, it was impossible to calculate the true ARPU and build accurate LTV models. With this function, it is able to determine ad revenue on the user level, from any ad unit, across every ad network on the ironSource mediation platform.

7. Try Dynamic Ad Testing

A/B testing is amazing, but as of recently, it became quite limited.

At least not if you’re targeting Apple users.

The thing is, Apple’s depreciation of IDFA (identifier for advertisers) had a tremendous impact on all app advertisers. One of its biggest consequences is limitations to A/B testing.

For this reason,, almost all major ad networks developed new ways to test ads. Dynamic creatives are the new type of personalized advertising. They allow advertisers to serve dynamic ads to users based on targeting data they provide.

This includes Facebook, Google, and TikTok.

Previously, app advertisers could test hundreds of ads and variations. Now, the number of campaigns and ad groups is limited. Plus, they are getting limited demographic, install, and revenue data.

This made a lot of advertisers try dynamic creative testing. All they need to do to start with it is to upload different assets (video, image, CTA, text, etc.). Dynamic creative testing does the rest of the job and tries to find the best performing ad elements.

Of course, this is no perfect approach.

One of its main flaws is that it doesn’t show exactly how each element contributed to ad effectiveness. With the good old A/B testing, you have detailed insight into this.

To sum up:

A/B testing is a never-ending process. There is always something to be improved and optimized. After you’ve finished one A/B test, there is always another behind the corner. Changes in the world of mobile games are constant and happen often, to keep up with the trends don’t forget to A/B test everything.

I think you’ll also find these articles interesting! 👇 💁

Mobile user acquisition — 7 ways to attract new users
What is user acquisition (UA)?
New Facebook Ads Library — all Facebook Ads in one place?

About Udonis

Since 2018 until today, Udonis Inc. has acquired over 150 million users for mobile apps & games. We’re recognized as a leading mobile marketing agency by 5 major marketing review firms. We helped over 20 mobile apps & games reach the top charts. Want to know how we make it look so effortless? Meet us to find out!




Udonis Inc. is an award-winning marketing agency specialized in mobile apps & games.

Recommended from Medium

Day 1 — Niche It

CallRail: Phone Software for the Modern Marketer

All You Need to Know About Google Analytics

Effective Sales Call Planning Guide: 7 Questions to Ask Yourself

Fast-Casual Digital Strategy

The purrrfect product design

Kitna deti hai?

3 ways to boost Content Conversions by making your customers rich

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


More from Medium

Conversion Optimization — 10th Week Review

Introducing Similarity Card feature on LinkedIn

Hey, Product Managers!!

Conversational UI: 8 Insights Into Smarter Chatbot UX