5 top takeaways from the largest CRO conference in the Baltics — or the first steps to your website optimization

In June 2017, I attended the Digital Elite Camp conference in Tallinn.

The most exciting reason to attend this conference was its strong focus on website traffic and conversion rate optimization (CRO) — not only does it gather hundreds of data geeks from all around the world, it also features CRO masters such as Peep Laja, the founder of ConversionXL, Erin Weigel, the Principal Designer at Booking.com, and many more.

Anyway, I came back with lots of useful ideas and didn’t want to keep them under my hat. So, here are my five top key takeaways from the Elite Camp.

But I’m warning you — I got a lot of insight out of this conference, so this post won’t be a short one. ¯\_(ツ)_/¯

Here we go:

1) Purchases and registrations are not (always) the right metrics to focus on

Instead, what you should really focus on if you want to grow your business, is your North Star Metric (NSM).

Your business’s north star is the real value your customer gets from your product. The north star metric measures the key benefit you deliver to your customer with your product.

Let me explain:

Let’s say your product is a messaging app. Then your north star metric is how many messages a day your customers are sending through your app.

The fact that your customer user has downloaded your app and now has it on the device, is not the value the user gets from your product. The ability to send messages, on the other hand, is.

If you focus on downloads only, in the long term, you focus on the wrong metric. What you should really pay attention to, is how to get people to send more messages through your app. Because this metric is the one that indicates product fit for your customer.

The higher your north star metric is, the better your product fit and your customers will keep coming back. And the more often they use your product, the more they will suggest it to their friends, family and other people they know.

Here are some more examples of north star metrics:

If you write a blog, your north star metric the total time people spend reading it — that shows the content is useful and relevant to your blog visitors. On Google Analytics, look for the ‘’Avg. Time on Page’’ metric.

If you run an ecommerce store, it can be repeat purchases. The fact that buyers are coming back and buy again means you’re selling the right products and people are happy with your service. On Google Analytics, keep an eye on the number of transactions and the total conversion value from your ‘’Returning Visitors’’.

If you offer a booking tool or service, then your north star metric is the total number of bookings made through your website. On Google Analytics, look for the ‘’Goal Completions’’ metric, which in this case will be bookings made.

So, again:

To find your north star metric, first find the key benefit your product delivers to your customer. Then find a metric that qualifies. Then, keep an eye on this metric and try to increase it month by month.

2) You should totally invest in your customer research

Many business owners assume they know their customers. Just like that. No research, no data, no detailed buyer personas. No clue.


‘’If you don’t know your customer well, 99% of your assumptions are wrong.’’ (Well said, Daria Nepriakhina!)

If you think you know your customers, you probably have no idea. And that makes it almost impossible — or in the best case, very difficult — to improve your business results.

To know what products to offer, what new features to add, what design to create and what product descriptions to write (or optimize your existing ones) — to do all that, you first need to know your customer to the bones.

If you don't know where to start, here are my favorite customer research, slash, spying methods:

One: Go outside and meet your perfect customers

Attend events and go to places your target audience hangs out, then observe and talk to these people. Note things like:

  • Who are they? Mostly men or women? What age?
  • How do they look? What do they wear?
  • How do they act? Are they quiet and reserved or loud and expressive?
  • What do they talk about? What are their interests?
  • Why are they coming to such places and events? What do they like or dislike about the place or event you are at?

Two: Find your customers on social media

Today almost everyone has a social media profile — the virtual self shows you almost as much as meeting that person in life. So, take advantage of this, find your customer on social media and analyze their profile. Write down things like:

  • Where your customers live?
  • Where do they work? What are their positions?
  • What do they care about? What are they interested in? Are they fans of something particular? Check their interests and the pages / brands they follow!
  • Any significant life events they have in common and have happened shortly before they bought your product or service? Just moved to a new city? Engaged? Became a parent?
  • What topics are their latest status updates and tweets about?

Three: Analyze your customer feedback

Check your product reviews, your customer survey responses, and the feedback and questions you receive in your support inbox. Talk to your sales and customer support teams — they communicate with your users every day, and know them best. Ask your team:

  • What words do customers use to talk about your product or service? What words do they use to describe their problem with your product or service? Use the same language in your website copy.
  • What products or features are users requesting over and over again? What problems do these features solve for them?
  • What do your customers use your service or product for?
  • Why did they choose your product or service over other similar ones?

The more of these methods you combine together, the more complete picture of your customer personas you’ll get.

Note: that doesn’t always have to be a customer persona: you can also categorize your users by their pain points or goals they want to accomplish by visiting at your site.

So see what data you get, analyze it and then it’s up to you to decide how to group the findings in a way you can work with them in the future.

3) You can’t optimize all landing pages at once — here’s where you should start

When you know what metric you want to increase and have defined your customer personas, you can start looking for things on your website to improve.

Your goal here is to find out what works and what doesn’t, and which copy / design version works better than the other one. And you want answers based on data, not simply based on your opinion, gut, or whatever.

‘’Ok, got it! Now, where do I start?’’

Start with optimizing your best and worst performing pages. Find your high traffic landing pages that perform better or worse than the average, and start improving them first.

Head to your Google Analytics account (if you don’t have the account, I’m surprised you’re still reading), and find your top landing pages with:

  • High traffic and high conversion rates
  • High traffic and low bounce rates
  • High traffic and high bounce rates
  • High traffic and low conversion rates

Also, go to Behavior → Behavior Flow → then, look for pages with high drop-off rates. These are the leaks on your site, where you lose the most of your customers.

Now that you’ve picked out landing pages you want to improve, start the optimization process with things that really need no testing:

Analyze these pages to see if there are no bugs. Run page speed analysis (here) and do cross-browser and cross-device checks. To perform well, your landing pages simply must be functional, load fast (people really, really, really hate slow webpages!), and work on all of the top browsers and devices.

When you’re done with fixing technical errors, you can start testing design elements and website copy.

And while there are many elements you can test, again, before you start — do your user research. There are cheap or free tools that helps you discover how customers are using your site, and use these insights to find things to improve:

  • See where your users click with the SumoMe Heatmap feature (Free)
  • Record your user sessions, and see what exactly each user does on your site with Inspectlet (Free)
  • See how low your customers scroll with Mousestats (Free)

Read more about other types visual analysis, and how each of these can help you discover new website elements to optimize.

4) Throwing spaghetti at the wall is bad — do your optimization right from the beginning

So you’re getting your hands dirty and ready to start improving your landing pages through A/B testing. Here’s one last tip before you start:

Do it right from the beginning. Which means:

  1. Define which metric you want to increase before you start testing. Is it your north star metric? Conversions? Which conversions — registrations, purchases, downloads? Button clicks? Time spent on site? Bounce rate? Other metric?
  2. Make sure your tracking tools are set up properly. That is, Google Analytics tracks your website data, and the data is correct.
  3. Always develop a proper hypothesis before you start the test. Without a hypothesis you won’t know what data you are looking for.
  4. Run your tests until you’ve reached statistical significance and for full weeks. (I’ll talk about this in a second.)
  5. Do a proper analysis of your results, only then implement the changes. This is a no-brainer.

Okay, but now, let's talk more about statistical significance, sample sizes and how long you need to run your tests. Because too many tests are cut short, which leads to inaccurate results and messed up webpages.

What is statistical significance

Statistical significance answers the question: ‘’How likely is it that my test results will say I have a winner, when I actually don’t?’’

For example, you want to test two headlines — H1 versus H2, so you run an A/B test on your landing page. Your aim is to reach the widely used 95% statistical significance, which means you want to reach a 95% probability that in the future, one headline really performs better than the other.

Put simply:

If the H2 headline wins, you can be 95% sure the test has determined the correct winner. Meanwhile, there is a 5% probability that the H2 headline is actually performing worse than the H1.

If you’re okay with the 5% risk, you can use the 95% statistical significance. If you test more controversial changes, you can increase your statistical significance to 99%. However, be ready that such tests will need bigger sample sizes and you’ll need to run them for a longer time to reach the result.

How to calculate your sample size and how long your test should run

Once you know the statistical significance you want to reach, you can calculate the sample size. Use this handy sample size calculator from Optimizely to do it, where:

  • ‘’Baseline Conversion Rate’’ is your landing page’s average conversion rate
  • ‘’Minimum Detectable Effect’’ is your estimation (← read again) of how much the change you’ve implemented will increase the existing average conversion rate. The bigger changes you test, the smaller sample size you’ll need. And the other way around.

And when you know the needed sample size, you can calculate how long you’ll need to run your test to reach the statistical significance you’ve set.

If 100% of your traffic will be allocated to the test, use this formula to calculate how long you’ll need to run the test:

And don’t forget that even if you have reached your 95% or even 99% confidence level in less than a week, you should still keep the test running for a full week. (Read why!)

5) You can test even if you think you can’t: how to test low-traffic websites

Picture this:

Let’s say you run a small e-commerce shop with around 700 weekly visitors and a 6% website conversion rate. And you want to run an A/B test and test the color of your ‘’Add to cart’’ button.

Ok, cool. Let’s calculate the sample size you need for this test:

To reach the statistical significance of 95%, you’ll need a sample size of 10'000 visitors per each variable. That is, 20'000 visitors in total, which means you’ll need to run the test for…

20'000 / 700 = 29 weeks = 7.25 months

… over 7 months to reach a significant test result.

Your store can bankrupt before the test ends!

But this is a very common problem for small websites with low traffic. Here are some ideas you can try to decrease the sample size you need for your tests:

Test more radical changes

Testing the color of the call-to-action button is a fun way to waste time, but in most cases this change won’t radically increase your website’s performance results. Therefore, your ‘’Minimum Detectable Effect’’ will be low and you’ll need to compensate that with a big sample size.

But if you change the button entirely — make it twice as big, change the color and change its copy, all at the same time, that lets you increase your estimation of the ‘’Minimum Detectable Effect’’, which will decrease your sample size:

Now the total sample size you need is 4'600 (2300 x 2), which lets you cut your testing time to 6.57 (or 7 full) weeks.

7 months vs 7 weeks. Great difference, no?

Focus on micro-conversions

The average conversion rate of our imaginary online store is 6%, which in this case means that 6% of the website’s visitors make a purchase.

For specific landing pages this percentage will be higher or smaller than the average 6%. The cart page by default will have a higher conversion rate percentage than, let’s say, a product page. That’s because there are more steps where the customer can drop off between the product page and your thank you page, when compared to the cart page and the thank you page.

The further the page you’re testing is from you thank you page, the lower its conversion rate, and the bigger sample size is needed to reach the statistical significance.


Focus on micro-conversions rather than website purchases when optimizing these pages. If you’re optimizing a product page on your online store, it can be clicks on the call-to-action button.

Less than 6% may be finishing a purchase after visiting the product page. But 12% of all product page visitors click on the call-to-action button. For our online store it means:

… we’ll need a sample size of 1860 (930 x 2) people for our A/B test, and we’ll be done in 2.65 (or 3 full) weeks.

Instead of button clicks, you can also focus on other micro-conversions, like:

  • Next page path
  • Bounce rates
  • Traffic sources
  • Pages per visit
  • … and other.

Consider lower statistical significance

Finally, you can also lower your statistical significance. This may not be recommended if you’re testing controversial changes, but can be used with improvements you’re quite sure will increase your results (read: your ‘’Minimum Detectable Effect’’ is high).

For example, we know that big call-to-action buttons in contrasting colors in general works better than the small and colorless ones that are difficult to notice.

In this case, you can afford the risk of lowering your statistical significance to 90% to be able to finish your A/B test faster:

In this case, you’ll be done with your A/B test in 1.34 (or 2 full) weeks. How cool is that?!

So here you go —all the most important lessons learned from the conference. And some lessons I learned myself through writing this post for you. If you have comments or something to add — feel free to comment below!