Lessons learned from deploying Courier into beta
I am really proud of how the team, here at Codeq, handled themselves during the initial rush of user on-boarding. Doing what we do at scale is half the battle. My biggest takeaways were just how important your on-boarding process is, how much authenticity in branding matters, and finding out how long our team can physically go without sleep. Email is one of the hardest kinds of applications to build and operate. Adding a state of the art super AI on top pushes the difficulty level past 11.
Don’t make content heavy landing pages
Learn from our mistakes. Don’t do what we did. Your landing page needs to be lighting fast and the signup process should be as painless as possible. Don’t use video backgrounds in your landing pages that bog down load times. If you’re using an email signup flow, look for low barrier to entry options like Ship by Product Hunt. Simplifying your on-boarding experience will result in more sign ups.
That being said, I am happy with the roughly 460'sh beta subs we have at the moment. I am positively thrilled there are people out there willing to take risks and try out a new email app that hasn’t proven itself when the selling point seems like black magic to the average consumer. I’m constantly amazed anyone was willing to put up with the 4–5 step process needed to get the app on their phone! 💖 Every last one of you.
Don’t make promo videos
We spent about a thousand dollars and broke into a friends startup just to film our promo video (Thanks Norm Liang). The whole thing was over produced. I am a terrible actor. We filmed 40 minutes of footage and ended up with about 1 minute of useable material. Just buy one from a vendor and call it a day.
I’d also recommend not getting overly cute with URLs. We picked courier.email because it looked cool, but as you can see^, nothing recognizes it as a URL. We will likely use something more practical for our iTunes launch promotions.
Where you post first matters
Product Hunt and BetaList were our primary sources of almost all traffic. Combined, they account for about 80% of our total beta tester signups. I’d also recommend Killerstartups and Betabound. I had positive experiences with all 4 platforms as ways to find early adopters. At peak adoption, we were getting a new signup every 2 minutes sustaining for about 2–3 days.
Launching during a solar eclipse = mistake
90% of the total population of the United States was outside getting partially blinded by the sun when we launched on August 21st. Future space product person: If you’re beaming your mixed reality app directly into peoples brains in 2023 via FTL blockchain metaverse, avoid launching October 14th. You’ve been warned.
People really don’t like double opting in to email lists
…Especially ones that don’t have a great mobile interface. MailChimp is great platform for collecting validated emails. The problem we encountered was its out of the box mobile UI isn’t amazing for collecting emails. It was especially bad at handling someone attempting to sign up multiple times. We had a couple users get stuck in a flow of reentering their email + visual captcha over and over. When I invent a time travel device, the third thing I do will be to go back and convince myself to use a different provider.
Don’t write super long Medium manifestos
I spent days writing my first two medium posts and the results dictate the Sharpe Ratio for both was about 50% above return. From now on I will focus on keeping anything I have to say publicly below a 10 minute read; beyond the 10 minute mark you will start losing people.
Notification summarization is spooky 👻 good!
If nothing else, you should try Courier just for the killer notifications. I’ve spent months comparing our’s side by side with our competition’s and I am confident in saying, “We have the best email notification system on the planet.” I have a dropbox folder with hundreds of examples to prove it.
Some choice side by side comparisons from the collection:
If you want to see more examples, ping me. I can come to your house with a slide projector and we can spend all night looking at them over a bowl of popcorn like it was my last trip to Hawaii. Do people still do that? People still do that.
Feedback is paramount
It’s really hard to get user feedback without feedback interface elements directly inside your application. We opted to use Apple’s native Testflight feedback tools and rely on our own email infrastructure to get feedback. It wasn’t successful.
We are currently in the process of rolling out our own feedback tools so users can easily tell us when our summarizer technology is and isn’t working properly.
The numbers are 👌🏻
50% of users invited to the beta joined the program. 95% of installs became retained users. 1/3 of those became daily users of Courier.
Session numbers turned out great. An average DAU has 90 sessions a day at about a median 85 seconds. People who actively use Courier are super engaged for short periods of time (which was our intended goal).
Top 10 actions taken in Courier:
Current stats on summarization are pretty amazing; currently, we’re consuming about 200–300k emails a week. 86% of those we were able to categorize and summarize. The remaining 14% being spam, chat, or content not deemed suitable to process. If you can find someone out there beating these numbers, by all means send them my way.
There’s also some really interesting stats on the various % of email types we’re seeing:
What surprised me the most was the % of personal email. Industry standard is around 10%. Also interesting is 10% of all personal email is someone requesting a task of some sort. When people are communicating over email, there’s a significant chance someone is going to ask you to do something.
Payment email segment breakdown:
Diving deeper into segmentation tells a fascinating story about how people use email for interacting with institutions. I always assumed shipment confirmations were one of the largest providers of email volume. Apparently, confirming various types of purchases is over 50% of volume. This tells me a majority of email people get is confirmation, from websites, that their payment was received (bills, movie tickets, delivery subscriptions, etc).
Our machine learning subsystems are getting 🔥 fast thanks to Paulo’s wizardry:
iOS 11 hit us like a brick
One of our biggest speed bumps came from supporting iOS 11 and the new HIG. Courier was designed assuming an iPhone was always going to be a little black box. The introduction of curved edges threw us a curveball. Email, in principle, has always been a list of objects with actions at the top and bottom of your screen. Adapting to the new form factor will be an ongoing challenge as the iPhone X rolls out as an information consumption tool.
What’s next?
We’re looking at rethinking some of our design choices based on usage patterns and user feedback. We aren’t happy with compose. Attachments need some love. iOS curved displays are coming. Pretty animations need to be pretty. There’s a story somewhere around filtering notifications based on their categorization. Servers got to keep scale’n. Skynet doesn’t build itself here, people. It’s a long dusty road to iTunes.
This is one of a series of posts about my life, Codeq, and Courier — our new email app! Please join our beta list and help us save the world one email at a time. On average, we think our email app can possibly save you about 15 hours a week by summarizing your email for you. Best of all? It is free.
