Hulu, we have a problem

Whenever one of your daily products makes a change, it can be a jarring experience. Even slight changes to Google and Facebook have been national news. So on April 15th when I signed into Hulu and realized that their Queue feature had been replaced with Watchlist, I was taken aback.

Instinctively, I knew something was off, but I’ve been through this before on other products and held off judgement. Perhaps with some use I’d get over the shock of change and appreciate this new feature for what it was. I’m still waiting for this to occur three weeks later.

For the uninitiated, Hulu’s Queue featured asked each user to “Favorite” shows they enjoyed watching. Whenever a new episode aired from one of those shows, it would be added to their queue. For example, I’d wake up Friday morning and have one location to access all the new Shondaland episodes.

Watchlist moves from a user-maintained list to a predictive algorithm to determine which shows a user would want to watch. In theory, this should also float up all my Shondaland shows on Friday morning… but in theory communism works.

Predictive Algorithms are like Communism

Predictive algorithms are a notorious pitfall for user experience. They sound so enticing, “We know all this information about you, so we can present content tailored perfectly for you.” Online advertisers have used this method for years to great success. Search for a new car in Google and for a week every website’s right-hand column will present you a plethora of choice.

This works for online advertising because if it’s wrong, the user doesn’t care. It’s low-risk, high-reward. If I’m reading a post about that awesome new twist on Scandal and I see an advertisement for a hybrid SUV I was googling last week… great! If I see an advertisement for a gas-guzzler instead… c’est la vie! If my roommate used my computer and my advertising tries to sell me headphones and purses for a week… I don’t care.

Facebook uses what I call a “medium-risk, medium-reward” model. Their newsfeed, probably the most sophisticated predictive content model in existence, uses what they know about you to sort the content you see every time you visit them. Most of that content is something you’ve consciously opted into, you friended or liked those pages. Most of the time Facebook gets it right, and if it doesn’t you probably wouldn’t know because you weren’t expecting that content in the first place.

Hulu is now using a “high-risk, low-reward” model. As a user, you know exactly what you want to watch, and you know when it should be available to you. You’re not exploring Hulu and discovering new Shondaland content on Fridays, you’re going to Hulu specifically for your Olivia Pope-fix. If for some reason their algorithm is off and the content you want is hiding, that’s a recipe for frustration.

Every day since Hulu has launched Watchlist, I’ve had a frustrating experience.

What Not to Watchlist

Predictive algorithms success or failure is completely dependent on the system’s ability to account for real human behavior. I imagine in the future the design of these systems will be a UX skill-set akin to interaction, industrial, or visual design. Let’s break down all the ways my real human behavior broke their system.

I fall asleep.

I fall asleep watching TV, always have. Most of the time I remember to turn on the sleep timer on my TV before I do, but sometimes I don’t. Because of this, I’ve woken up in the morning six episodes into a show I’ve never heard of. When Watchlist launched, it assumed I had wanted to watch all of these shows. The first day, I spent twenty minutes figuring out how to delete shows from my watchlist, as over 50% were shows I had no interest in ever watching again. Then I learned how to turn off auto-play.

I watch TV.

I don’t have cable at home, but I do travel quite a bit. When I do, I watch live TV in the hotel room. Then, I pull up Hulu on my laptop later and my watchlist is filled with shows I’d watched already. In the past, I could just mark these episodes as “watched”, but not anymore. If there’s a setting to do this it’s hiding (I’ve looked, extensively). So the watchlist keeps reminding me to watch the show, and then because I don’t, it assumes I don’t want to watch that show anymore and stops telling me about new episodes.

I have a roommate.

I have a roommate, and it’s her Hulu account that’s set up with the living room television. Much like with the live TV, this has caused chaos in the algorithm for both of us. Sometimes we watch shows together, sometimes I watch shows without her, sometimes she watches shows without me. The Watchlist on her account is a nightmare. I can’t imagine what it must be like for families.

I watch stuff and don’t like it.

I tried watching a new show. I watched less than five minutes of it. It was boring. It was on my Watchlist for a week until I finally went in and manually deleted it. That’s what’s called a negative option. A user is forced to say “don’t” rather than given the opportunity to say “do”. It’s bad UX 101.

I sleep, I watch TV, I have a roommate, and I don’t like some shows. These are not bizarre circumstances. These are natural human behaviors, and they are wreaking havoc on my viewing habits.

Fixing Queues

How did Hulu get it so wrong? I can only guess why they made this change in the first place. Reverse-engineering design decisions is always a fun game, and in this case I hypothesize two issues:

1. Users weren’t favoriting shows, and so the queues weren’t working.
2. The queue system wasn’t expanding user’s viewing habits into new content.

These are easy fixes.

1. Add a “Add to Favorites” option after watching a show.
2. Modify the existing “Watch Something Else” option that appears after a show to have two options instead of one. Have one option be content the users likes, and one be new suggested content.

Done.