Why you don’t need to do half of the tests you’re doing on Facebook ads.

In old times we were taught to test every element of the ad: image, text, headline. It was a very true for times when we mostly used static images and didn’t have a smart optimization algorithms. So the only thing we were trying to do was to increase CTR of the ad (without misleading).

But even those days we had a significant problem with this approach. As we all know — high CTR doesn’t mean positive ROI. And when we do a test we want to use CPA as a KPI, not a CTR. But problem here is that it’s often quite hard to get enough conversions to make a statistical significant conclusion. If your CPA is not super low and a difference between A and B is not 100+% then it takes a lot of money to see a significance. A lot of marketers would get 20 sales and if A get 12 sales and B get 8 sales they would assume that A is a winner. Which is not true. They maybe equal or even B can be better. But to learn that they would need more data and to gain that data they need more time and money. A very few people are ready to do this.

The smartest thing to do here is going up to the funnel and determine a different conversion as a KPI — if this conversion has a strong correlation (you can exam it mathematically) with an ultimate goal then it may work.