Evan Solomon Can Fake Anything

Data Scientist at Medium, not a fan of Big Data

Evan Solomon, a data scientist at Medium, learned to fake it young. “In fifth grade,” he recalls, “we had this assignment where we had to interview a family member about their life history. It didn’t sound like fun, so I didn’t do it.”

What did he do instead?

The day the assignment was due, I walked up to the teacher’s desk with everyone else and then just turned around and walked back. A week later, my teacher said, ‘Evan, you didn’t turn in the assignment!’ And I replied, ‘Oh no, I did. It must have gotten mixed up with all of the others.’ When she gave back the scores, I got a 100%. You know she just felt terrible—I can’t believe I lost this kid’s project, it’s so unfair. But that’s when I realized, hey, you can totally fake anything!

Solomon is quick to add that he does not feel even remotely guilty about this stunt: “I actually feel really good about it, especially now. I think a lot of life works that way. You have to fake stuff and act like it’s OK, and that ends up being true.”

Faking it became a way to make it through college—almost. Solomon partially funded his education as a finance student at NYU by working as a professional poker player. “I liked the academic parts of finance,” he says, “but I didn’t like the kinds of jobs you get in finance. I dropped out with a couple of semesters left. Mostly I played poker in college.”

The online poker scene was just getting started when Solomon was skipping class to play. “The early days of online poker were like scientific discovery,” he remembers. “No one knew how much of the old ways we should retain.” As it turned out, the slow, conservative style of play favored by pros didn’t apply online, where the game rewarded more aggressive, haphazard play.

Figuring out how to play an old game in a new way, Solomon contends, was similar to trying to understand a software product: “You have a vaguely defined problem, lots and lots of inputs, and this general idea that more money is good, more traffic is good, more engagement is good. But there’s this big gap between the things you can easily measure directly—how many hands do I play? how much do I bet?—and the outcome. There’s all this noise in the data.”

Learning to play an old game in a new way is also key for thinking about online publishing. As Solomon points out, “We don’t know a lot about how people read on the Internet yet. It’s a deceptively hard problem. What I like about Medium, which for all its complexity behind the scenes is a simple product—you write something on a page, publish it, and there’s a link to it—is that it’s still unclear what works about it and what doesn’t, what we need to improve and how.”

Photo by Misty Xicum

Indeed, he works hard to avoid making assumptions about growth and adoption patterns. Medium’s data science team focuses primarily on measuring reading time, a growth metric devised to be as simple and uncontestable as possible. As Solomon puts it, “The fewer actions that go into your metric, the better. And everyone has to agree when to cheer.”

Far from a data hoarder, Solomon believes that quantitative data is most useful when it tests hypotheses developed from qualitative evidence. “Page views and unique visitors are very surface level,” he observes. “And there are many more people who can store petabytes of data than can accurately critique a user flow or get value out of talking to actual people using your product.” He is clearly warming up for one of his favorite rants:

“People get so caught up in data mining, A/B testing, Big Data—I hate that phrase, it’s terrible. All the buzzing, blinking dials can blind you to basic logic. Starting with quantitative data is like trying to take the million monkeys and reproduce Shakespeare. Some people will win that game, but the chances are worse than playing the lottery.”

Solomon, who dropped out of NYU only to land in tech-related PR before spending several years at Automattic as a growth engineer, admits that one thing he can’t fake is a sustained attention span. “I feel like George Costanza on Seinfeld when he says that he can’t read anymore because he’s been spoiled by books on tape.”

Medium, Solomon notes, “was a revelation to me—it has all this great content that’s also the perfect length. The first thing that I do when I open a link someone sends me is look for a reason to close that tab. With Medium links, I’d open them and just start reading.”

For the data scientist who can fake almost anything, the platform “was such a consistently good experience that, as a user, I started to trust it. Now that I’m here, the most important thing our team can do is keep the rest of the company as informed as possible, make sure they have access to the data they need to make the best decisions for the platform. It’s about never closing the tab.”