Tapping the election

Or, “My harrowing experience of learning a new framework during Build The News — and then putting it into production four days later”

My team’s project for Build The News was “TapDat”, a webapp for providing a second screen experience during the all-party television debate. The idea was to create a “worm”-style sentiment tracker that would chart the collective feeling of the audience during a debate, while doing away with the physical dials used traditionally in order to allow a wider range of people to participate.

Yes, we actually photoshopped creepy-as-hell smiles onto the party leaders as a way of depicting current state. Suffice to say, this particular feature did not make it into production.

I’d already built something similar for the Hackney Citizen (which I frequently use as a lab to try out various journalism technology ideas) for the Hackney mayoral hustings that took place last year. The outcome of that initial experiment was fairly mixed — I’d used an HTML dial component, where turning to the left depicted a negative sentiment and turning to the right was positive. As there were five candidates debating and each would speak in turn, this made following along rather difficult: given the range of opinions presented, it required way too much user interaction, and many found their batteries being quickly drained by the process (my reliance on websockets probably didn’t help that either). Not only that, there wasn’t much incentive to engage with the live-polling process, and the data I got from it was sparse at best.

To this end, we rebuilt the idea using several key principles:

• Sentiment “votes” should be sent only on user interaction, with “neutral” as the default state.
• The sentiment tracking shouldn’t be the sole reason for using the webapp; it should provide additional information or social connectivity that accentuates the debate.
• The data created by the project should be available to viewers in realtime during the debate.
• It should focus on iPad, as the vast majority of our audience consumes our content via tablet, but bonus points if it also works on mobile.
• It should be somewhat fun.

With that in mind, we started work on a prototype that displayed all the debate participants in a grid, with users either tapping or pinching each face in order to send a positive or negative sentiment (respectively). We even discussed creating a gesture based on the accelerometer, such that shaking the device up and down would give all the candidates negative sentiment.

Alas, things don’t always go according to plan — I’d decided to try Meteor, a framework for building isomorphic JavaScript applications, wherein the same codebase running on the device also runs on the server (I very much do not recommend trying to wrap your head around this concept while under pressure at a <strike>hack day</strike> code jam). Meteor’s super cool, but kind of a challenge to figure out. This was definitely compounded by the fact I couldn’t get any of the data visualisation libraries I’m familiar with to work in Meteor; I couldn’t even get the Meteor Hammer.js package to work. Whether that’s the quality of the Meteor package repository or my utterly n00b-ish Meteor code, I’m not entirely sure, but after frantically trying everything all day on Sunday, I eventually pulled the plug and we presented my colleague’s jQuery-based front-end mockup instead, which was received well enough. You can see that at tapdat.timesdev.tools.

I was determined, however, to get this into production before the debate that Thursday. Working through my lieu days on Monday and Tuesday and then cracking away all day and night on it Wednesday, I trimmed back a lot of the extra functionality we depicted in the prototype and eventually got it (mostly) working in time. I replaced the gestures with clearly-labelled buttons, cut out the onboarding and Twitter functionality and renamed it to Times Election Dashboard.

Although I didn’t manage to figure out Meteor’s reactive programming paradigm in time to make the graph live-update, I did get a good number of users and ultimately derived ~800 different sentiment points during the debate. You can see the graph I produced afterwards here.

Click the above link for the interactive version.

Honestly, hiccups aside, it worked out much better than I thought it would.

Lessons learned from this:

1. Learning a new framework or programming paradigm over two days (one of which was spent woefully hungover) is not a good plan. I did the same thing at the first Build The News when learning Angular and was similarly overwhelmed. The fact I tried to do this a second time proves I’m either insane, or an idiot, or both.
2. Don’t assume anything on AtmosphereJS.com actually works, especially if running against a deadline.
3. Using Meteor.com as a production platform is fraught with peril, though to its credit, it did stay up the entire time people were using the webapp.
4. “Ændrew’s First Law of Google Analytics” will bite you in the ass every single time if you’re not careful. I’m still trying to figure out how many people actually used the thing because I forgot to add analytics (shocking!) and am needing to figure that out via server logs instead...
5. Having a decent front-end dev at a code jam can and will save your project (Thanks, Josh!).
6. Follow your gut on data visualisation — simpler is generally always better. We initially were going to go with a streamgraph for the output, because darn it if they aren’t pretty. But after several days of trying to figure out how to actually read the damn thing, I replaced it with a much simpler line graph. If you can’t understand your own data visualisation in under ten seconds, chances are your audience won’t understand it either — and they won’t go to the same effort as you to figure it out.
7. I really need to get better at MongoDB. I spent a lot of time writing code to get the server to effectively do a map-reduce, but I probably could’ve done this way more easily in Mongo if I was more familiar with its aggregation functions.

I’ll probably open source the code once the election’s over, it’s an idea that has some really interesting uses (I’ve considered resurfacing it for the next season of X Factor or something.). Until then, keep an eye on github.com/times, where we keep all of our open source code.