Christina Scott, CTO of News UK and Jonas Templestein (née Huckestein) , co-founder of Monzo

Changing media: 31 things we learned about experiments, optimisation and inclusivity

Change was a product and data forum organised by Fluxx and News UK in London this week. We saw speakers from Netflix, Lego, Monzo, the BBC, The Times and a bunch of hot startups. This is what they told us…

1. Netflix found that movie ratings are not a good predictor of movie enjoyment; people will give five stars to Hotel Rwanda, but they’d much rather watch Paul Bart Mall Cop, and might go even go on to watch Paul Blart: Mall Cop 2. [Gibson Biddle, Former VP of Product, Netflix, who’s slides are here, and who talked to Rory about decision making here.]
2. In 2016, Monzo had to change their name (from Mondo) for legal reasons, and asked their customers for ideas. They had 10,000 suggestions, including “ 350 variations on ‘Banky McBankface’.” [Jonas Templestein, co-founder of Monzo]
3. The Times judges stories according to a Dwell Time Index that acts as a proxy for reader engagement and enjoyment. 100 is average, 200 is amazing. Every journalist can now see the DTI index for their pieces using a browser plug-in, and the number is often discussed at morning news conference. [Dan Gilbert, Director of Data, News UK]
4. In 1964, the average lifespan of a company was 33 years. In 2016 it was 24 years, and by 2027 it’s expected to be just 12 years. [Scott Anthony, via Jennifer Torry, Creative Strategist, Fluxx]
5. 59% of people will share a link before they’ve read it. [Dhruv Ghulati, co-founder, Factmata]
6. In 2017, during Hurricane Irma, CNN launched a ‘lite’ site with no pictures and no ads. It uses very little data and loads incredibly fast — the pages are only slowed down by the analytics tags CNN use to see if anyone is looking at them. The site is perfect for people with a weak phone connection, but also handy for people who don’t like ads, or waiting. [Lifehacker, via David Kennedy, Design Director, WordPress VIP]
7. 5.9% of people have a smart speaker in their bathroom [VoiceBot, via Dan Whaley, Senior Architect, BBC Product (Voice & AI)]
8. Life & Lipstick is a beauty podcast on the Entale platform. In one episode, an extraordinary 30% of listeners clicked on an advertisement. [Hannah Blake, Head of Partnerships Entale]
9. At Finimize, every team is expected to perform five experiments every month, and to kill new features that don’t work. For example, they found that by adding audio versions to the content on their app, the completion rate — the number of people who finished the pack after starting — increased by 30%. [Max Rothery, VP of Community, Finimize]
10. In 2009, NPR added transcripts of radio programmes to their website. The feature was designed to help deaf people, but also sent engagement rates up 7% and unique visits up over 4%. It turns out that what works for deaf people also helped Google as a form of SEO. [David Kennedy]
11. Monzo was one of the first UK banks to use Amazon’s AWS cloud storage, rather than building an in-house data centre. They realised it was possible because the Financial Conduct Authority were using AWS themselves. [Amazon via Jonas Templestein]
12. 37% of British workers think their jobs are meaningless. [YouGov, via James “JP “ Poulter, former Global Head of Emerging Platforms, LEGO Group]
13. Three ways to answer the question “Is using voice a good idea?”:
    1) Can I say it?
    2) Am I comfortable saying it?
    3) Will it make this interaction quicker?
    If the answer to any of these questions is no, maybe a traditional interface will work better. [Dan Whaley]
14. 1 in 5 people worldwide have a disability. [David Kennedy]
15. “At Netflix there wasn’t a person in the building who didn’t believe in better living through math.” [Gibson Biddle]
16. Monzo believe that the idea they’re special, or ‘a bunch of geniuses in a room’ is pure hype. “The only thing we did was to work out what works, and keep doing it.” [Jonas Templestein]
17. Voice interfaces are tricky: Amazon are keen to make things easier for users, so you can say
    “Alexa, play Radio 6 Music”
    instead of
    “Alexa, ask the BBC to play Radio 6Music”
    It’s clearly better for users, but makes the BBC nervous; it weakens the listener’s connection with the corporation that provides the audio, and might be part of disaggregating the bundle that a radio station represents. [Dan Whaley]
18. The Times has developed a Smart Subscriptions system to optimise subscription sales. Every time someone visits the website, they are given a score that predicts how likely they are to subscribe, based on the very limited data available (Country, Time of Day, how many times they’ve visited before). Visitors with a high propensity score are shown a signup page to buy a subscription, while low propensity visitors see a free registration page. In six months, the system has improved conversions by 190%. [Claire Draycott, Deputy Head of UX at News UK]
19. When developing apps for Amazon Alexa, the BBC spend a lot of time thinking about the ‘Implicit Invocation’ — the exact words that can be used to summon the app. It’s just like SEO, but for speech. [Pat Higbie via Dan Whaley]
20. Instead of a stream of news, The Times is an edition that is editorially curated and updated four times a day. [Aeneas McDonnell, Group Head of Product, Times Newspapers]
21. Between 2015 and 2016, the amount of time office workers spend “doing the primary duties of my job” decreased from 46% to 39%. Instead, they spent 16% on email, 10% on ‘wasteful meetings’ and 8% on ‘interruptions’. [Workfront via James “JP “ Poulter]
22. 71% of the world is still using 2G or 3G connectivity.[GSMA via David Kennedy]
23. The Times newsroom has a statistics Slackbot called TopBot which responds to journalist questions like “@topbot, what were the top 20 articles today?” [Dan Gilbert]
24. Liz Jackson is The Girl With the Purple Cane, who talks about designing with disability, rather than for disability — here’s a great talk she gave last year. [David Kennedy]
25. Back when Netflix was still a DVD rental business, they tested a perfect user experience — where customers always got brand-new films the day they were released. The trial customers loved it, but it only improved churn rates from 4.5% to 4.45%. It would have cost $5m to implement, so they didn’t do it. [Gibson Biddle]
26. More recently, Netflix tested a notification to warn new customers that their free trial was about to end. While it felt like ‘the right thing to do’, it reduced conversion rates (from 90% to 85%) and would cost $50m in revenue. They introduced it anyway, as it was a relatively small cost (out of an annual revenue of $16bn), good for the brand and easy to reverse. [Gibson Biddle]
27. Working with startups is all about relationships: “You need to like each other and you need to trust each other. Trust your gut. Remember that the idea and the product can change, but the people won’t. Make sure you’re happy to be in their company” [Jennifer Torry, who wrote about culture in startup incubators here]
28. Netflix use data to predict how successful a new show will be, then uses that to set budgets. The algorithm predicted that Stranger Things might find 100m viewers, so the first season got a $50m budget, while Bojack Horseman might find 2m viewers. It was still worth making, but on a much smaller budget.” [Gibson Biddle]
29. “Nobody really likes the taste of avocados. They like the taste of garlic, salt, chilli and pepper.” [James “JP “ Poulter]
30. “In the early ’90s I used to put a photo of the product manager I was competing with on my desk.” [Gibson Biddle]
31. In the early years of Monzo, they spent 3 months building ‘Monzo Me’, a payments system to try to make the bank more viral. It was a “total flop, useful to 1% of our customers”. Then, in one week just before Christmas, they created ‘Golden Tickets’ to let customers let friends skip the waiting list. This was a spectacular success, leading to 5%/week growth for nine months. “We almost didn’t build it because we thought it was so stupid.” [Jonas Templestein]

You might also enjoy: 31 lessons from startups doing it better than you, or How Fluxx uses jugaad innovation Every Day.

Tom Whitwell is a Senior Consultant at Fluxx, a company that uses experiments to understand customers, helping clients to build better products. We work with organisations such as Condé Nast International, Energy Systems Catapult, National Grid, BEIS and Severn Trent Water.