How we made a TV ad for our seed stage startup. Part 2.
In which we run the ad and figure out if it works. Spoiler — it does, but not as well as other channels.
This is the second piece in our series on how we made a TV ad for our startup. You can read the first piece here, which outlines how you actually make the ad and get it approved for broadcast in the UK. Here’s the finished advert:
In this piece we’re going to explain what we did to prepare our site for the expected spikes in traffic, what tools we used to try and measure the impact and what we learned about the effectiveness of TV advertising for our startup.
TV is hard for internet startups
Before we start, it’s worth pointing out that TV advertising doesn’t really play nicely with digital businesses.
First, when you advertise on TV using a ‘direct response’ advert you get big spikes in visitor numbers. These are hard to manage technically, unlike the smoother growth that comes with native digital advertising such as paid search and social.
Second, its also very hard to track the impact of your TV advertising. Digital marketers are used to getting very detailed reports on the effectiveness of every individual advert, whereas on TV there are no direct reporting mechanisms.
“TV advertising doesn’t really play nicely with digital businesses”
On top of the reporting challenge, its also very hard to actually act on the data to optimise and improve your tactics because the cycles of ad production and buying are so long.
Having said that, it is possible to overcome some of these hurdles and below we’ve explained some of the things we did to try and overcome these challenges at Lost My Name.
Preparing for the TV traffic
As mentioned in the previous piece, we already had some experience of the impact of TV on website traffic when our co-founders Asi and David had appeared on Dragon’s Den, a BBC TV show where entrepreneurs pitch to famous investors.
Back then we made a video that cut the appearance on the show with shots of our real time Google Analytics dashboard to show what happened.
Alas, in their infinite wisdom, the BBC have decided to remove this from YouTube so you can non longer see for yourself the amazing impact their business show has on businesses. ☹
Regardless, the short version of the impact of our appearance on Dragons Den on traffic was that at one point we hit 13,000 visitors on the site, and it didn’t break. Result!
This was mainly down to two pieces of preparation that Patrick and the engineering team put in place.
First, we put a global Content Delivery Network (CDN) called Fastly in front of the website.
A CDN is a service that ‘caches’ (saves) your content on many different servers around the world and then sends it to visitors directly from these servers when they want to access your site. This prevents your own servers from slowing down when lots of people want to view your content.
In our case, the main piece of content we wanted to make available via the CDN was our homepage.
In addition to adding the CDN severs to spread the load on our homepage, we also upgraded our own servers substantially on Heroku, increasing the size and performance of our database and the available ‘dynos’ (computers) that do the work on our site of rendering the book and managing the check out etc.
So, based on this successful experience we decided to copy this strategy and add a few more optimisations. Here’s what we did to prepare our site for the TV ads.
1. We implemented another global, cached CDN
This worked before, so we did it again!
In between our appearance on Dragons Den we’d completely rebuilt our site on a new e-commerce platform called Spree, so we had to do this work all over again.
We ran into one additional snag during this process. Along with rebuilding the site in the intervening period, we’d also launched Lost My Name in multiple languages. This added an extra layer of complexity as we had to cache multiple versions of our site with Fastly and serve the right language to the visitor, so this took a lot longer than anticipated.
2. We optimised our mobile experience
At Lost My Name we get about 50% of our visitors on mobile devices (tablets and smartphones). On smartphones our conversion rate from visitor to customers is about half that of desktop and tablet.
It’s something we’ve been trying to improve for some time, but the TV ads gave us a shot in the arm to try harder as we assumed that a lot of people watching the TV ad would visit the site on a phone while sat on their couch.
We tried three things to improve the experience for our mobile visitors.
First, we simply checked over the pages and tried to optimise page load speed. We did this primarily using the Google Developers page speed insights tool.
Second, we did some tweaks to the user experience of the customer journey on small screens to bring the various ‘call to actions’ (buttons) together into more consistent locations.
Finally, and most ambitiously, we tried to tackle the stage of the customer journey with the biggest drop off on mobile phones — the jump from the preview page on our site to the cart page.
This was a major piece of work in its own right, and probably deserves a whole blog post.
The short version is that we hypothesised a big contributor to the lower conversion rate was the poor rendering of the book preview on mobile phones, so we re-wrote our book preview ‘widget’ from top to bottom and introduced a new ‘swipe’ based user interface for customers on small screens.
3. We increased the performance of our database and dynos on Heroku
Finally, as before with Dragons Den we also increased the number of dynos and the performance of our database dramatically on Heroku, with the intention to allow up to 400 simultaneous connections to our database.
So what happened to our site?
Despite all this, we still experienced a big fail when our website went down on the second day of TV advertising. D’oh!
Here’s what happened during one of our biggest spikes, during an ad that went out during the Twilight Saga.
This chart shows a huge slowdown in response time for visitors — most visitors in this time would have seen a ‘500' error page because the web application was going so slowly.
But, but, but, why!? We implemented Fastly didn’t we? Well, if you dive into this chart in more detail you can see that if you hide the request queuing data, the fail is actually a very slow database response time.
After some diagnosis we found the culprit — a hidden inefficient query on the homepage.
On our homepage we show the value of a customers cart in the cart menu item on the top right of the screen.
The database query that generates this value on the homepage is actually loaded in using Javascript via a query to our application database, and so wasn’t cached by Fastly.
This wouldn’t necessarily matter, but it also turned out that the actual query that generates the cart value was being run very inefficiently due to a missing index in our database. This missing index meant that each query was running all the way though every row in the respective table to find the necessary data, rather than using an index of the table to look things up.
The outcome for customers was that this generated lots of super slow database queries — compounded by a poor user experience on our site whereby to add multiple books customers have to return to the homepage.
Once we’d established this issue the fix was simple, we just turned off the cart value on the homepage.
In hindsight, we should have load tested the homepage more effectively, but this would have been very hard to do.
We did load test simple visits to the page, but to find this specific issue with a load test we would have had to create scripts that acted like customers that added books to their carts, and then returned to the homepage to add more books. Not impossible, but hard to do and something we didn’t think of.
Reporting on the impact
So, that’s what we did to prepare the site. What about the reporting? As mentioned at the start of the piece, figuring out the impact of your TV ads on sales is quite tricky — much tricker than figuring out the impact of native digital advertising which generally has reporting built in to the same interfaces that you use to buy the advertising.
To help us overcome this we used a nifty service called TVsquared. In a nutshell TVsquared compares your historic traffic and sales with the traffic and sales during your TV ads and therefore allows you to attribute a given portion of those sales to your ads.
This is a big timesaver. In the past, Ian, our CFO, has done this manually using minute by minute data pulled from web analytics services and transactional data from the database. This actually takes hours and hours to do, and results in a chart like this:
TVsquared does the same for you automatically, producing charts like these:
TVsquared then does another useful thing and matches the completed order data to the cost of that TV ad slot, giving you a Cost Per Acquisition (CPA) per show.
In the image above, you can see that the cost per completed order (the same as the CPA) varies a lot by show. Hairy Bikers is giving us a healthy CPA of £12.47 whereas Ice Age 2 is giving us a very poor £78.25.
Our most profitable show was Impractical Jokers — we spent £255 on ads and sold £900 worth of books, yielding £645 profit.
And we got the best absolute ROI from Little House on the Prairie — We spent £25 on ads and got £300 worth of sales, and amazing 1000% ROI!
There was also significant variation in the performance of the different TV stations:
In total, after we’d done a week of ads we got the CPA down to around £38. Not bad, but not good enough for us — this doesn’t give us a positive return on our investment.
In terms of the actual amounts, we spent ~£67,000 on buying the ads, and made ~£59,000 in sales in return.
We spent £67,000 on the ads, and made £59,000
This isn’t a ‘fail’ per se. With time, and now armed with more data about the types of shows and times that generate sales we could probably get our TV ads to profitable levels.
However, three things stopped us from doing this.
- It would have just been plain expensive. We’d probably have to spend another £150k or so to get the ROI to positive.
- The opportunity cost of this spend would have been too high — because we could spend that same money on Facebook ads or paid search and get a much higher ROI.
- By this time we were hitting physical printing capacity issues anyway, so even if we could justify the spend, we wouldn’t have been able to fulfil the orders.
By the end of the second week of TV ads it was pretty clear that we were on track to smash even our most ambitious sales targets for Q4 using primarily Facebook and paid search advertising alone so we just couldn’t justify continuing.
Annoyingly, around this time we discovered a new service from Sky called Sky Adsmart that takes a more ‘digital’ approach to buying ads and identifying profitable audiences that looks very promising. But we were too late to try it out. Next time!
What did we learn?
Overall, we were pretty happy with the results of our TV ad experiments. The team pulled together well and worked hard to create and ship a TV ad in record time with an insanely small budget.
The CPA was ok, and we probably could have reduced it if we’d continued. We had a ‘good problem’ in that our Facebook and paid search CPA’s are so good that its very hard to justify spend elsewhere, but we now know much more about the kinds of TV audiences that respond well to our product.
From a tech perspective we could have done better — our site went down — but aside from one rogue query we did pretty well at coping with the lumpy traffic amid a wider story of 20% growth week on week. But, the development team responded quickly to the error which is more important than missing the error in the first place.
Overall, we are now much better placed to make TV work in the future when we have bigger budgets and more time to prepare a deeper multi-channel strategy.
One bonus outcome is that we’re now going to try some YouTube ads using our TV advert — YouTube seems like a more natural place for us to investigate video content. We’ll write a post on how that goes later on.
If you enjoyed this we’d love it if you could hit the recommend button below. You can also follow us on Twitter for more updates on making Lost My Name — and if you’d like to come and join us, we’re hiring!
