#MilValChal: Ghosts of Experiments Past
Introduction
My name is Alan Foster, CEO of Lore Media Inc. Our company is looking for ways to use technology to improve the experience of reading literature rather than just replacing it. We’re doing that by formatting 2D text as 3D books in your web browser and VR/AR. It was inspired by the books found in Skyrim, a videogame from the Elder Scrolls series.
Last year I accepted the #MilValChal, conducted 100 interviews, and I posted 2 experiments on Trello. One of those experiments couldn’t be completed, and the other failed miserably. Since then I’ve spent over 6 months working on my startup structure while trying to figure out what went wrong. In this post I’ll share some advice on how to properly create an experiment, as well as how to avoid the mistakes I made myself.
Experiment 1: UESP iFrame Embedding
The first experiment involved embedding our early technology on a third party website (UESP.net, an Elder Scrolls fansite) and testing readers’ preferences. The experiment was designed perfectly, but when it came time to embed our <iframe> code, the website owner changed his mind with concerns about how users would respond to the change. Our primary testing ground had been pulled out from under us like a rug, which meant we had to rethink our strategy.
The biggest lesson from this experiment was to make sure you always hold the pieces to your own puzzle. Experiments that rely on other people to follow through can lead to a potentially failed experiment. We realized that we would need to provide website administrators more evidence that we were creating something that readers would enjoy, and decided it was time to build a following.
Experiment 2: FaceBook Likes
The goal of our second experiment was to build that following by connecting with fans on social media, specifically FaceBook. This would not only increase the number of testers for our software, but also open free lines of marketing we could use for future announcements. Rather than using traditional targeted marketing using ads, we chose to see how many community pages would be willing to share it with their members. Then we’d be able to track conversion and see what trends emerged to help solidify our target audience.
Our goal was to have 5 out of 12 communities share a post about our technology, and a total of 0 communities did. It was a big red flag that something was wrong; the testers who visited our website were extremely impressed with our 3D reading app, or at least most of them.
A Pivot Like No Other
I went back to the surveyed data we collected from about 30 early testers and began combing through it, looking for trends I had missed. After several hours of comparing data sets I discovered that there was no direct correlation between peoples’ familiarity with Skyrim and how interested they were in our product. I had wrongly assumed that people who had played the videogame would “get it” because they were familiar with the formatting. The true correlation was found in how often they read literature, and how much they hated e-books.
Our original business model had been a website for publishing short stories inspired by previous works of art such as videogames or other literature (these short stories are known as Fanfiction). It was around this time that Pokemon Go had recently come out, and we had just begun to realize that our technology was an early form of augmented reality. We realized that our value proposition would be much stronger in VR and AR, specifically the mainstream adoption of augmented reality in the form of “smartglasses”. Over several months we discovered that when you calibrated an augmented reality device to your eyes specifically, you no longer needed reading glasses to read a holographic book.
We also began speaking with librarians about our technology, specifically those at Boston Public Library. One of the librarians made a comment about potential applications in Rare and Historical Documents. Apparently, BPL preserves more than 1.7 million rare documents, many of which are now inaccessible to the public due to their fragile condition. AR may give us the opportunity to experience what it’s like to be in the presence of those pieces of history without the training or facilities required to handle them.
Conclusion
After 2 failed experiments and several months of deep digging, I’m excited to show everybody a video of where our technology currently stands. We’ve been accepted as finalists to pitch and attend the MIT PlayLabs Accelerator, and we’ll learn this week if we’ve been accepted to receive $20k. We’ll be spending the next several weeks running lean experiments to validate several more early adoption markets. I look forward to sharing our results!
