Hardly an original title given where I work
Apparently Mark Twain said that. Although his real name was Samuel Clemence and he suggested he heard it from Disraeli?
Who maybe heard it from Lord Balfour or maybe Baron Courtney?
So what I am saying is even the bloody saying can’t be trusted let alone the numbers!
Anyway hopefully you’ve learned something from the other speakers today. They’ve shared their hard earned insights and knowledge for the benefit of everyone here.
I’m not going to do that. I’m going to tell you a story. A story that will hopefully leave you feeling more positive about your darkest professional day. A story filled with tragedy, comedy and a whole host of errors. In fact you might even call it Shakespearean.
In fact 20 years ago at this very university (though not this shiny building) I was studying Shakespeare at the same time I discovered the World Wide Web.
If only I’d stuck to literary pursuits.
Instead what I did was carve out a web career being one of *those* guys. You know the ones. No real discernible talent. Can’t code. Can’t design. Negligible project management skills. Knows a little bit about a lot of stuff and gives good meeting.
I embraced social media to such an extent that I almost slipped in to full Web 2.0 douchebag mode. I was probably only days away from calling myself a Guru or a Ninja. Thankfully I saw the light. For the most part.
Anyway these days I have the rather grand job title of ‘Head of Digital Transformation’. I know what you are thinking and you are correct — it means nothing — but it kind of sounds like I could be part of the Optimus Prime gang so I make do. In certain corners of the civil service job titles are given in lieu of well money I suppose so there has been considerable inflation in recent years.
I work for the Office for National Statistics here in sunny Newport. I’ve been there 13 months and I’ll just take a moment to share with you a few things I’ve learned in that time as if you are anything like I was before the fateful day I took the job you don’t really have a Scooby about the joys of official statistics.
So here are some stats about the ONS.
http://www.ons.gov.uk/ons/about-ons/who-ons-are/index.html
Oh and 50% of the bloody staff are related.
Anyway as I said I am here to tell you a story.
So.
Once upon a time (April 2013)
In a land far, far away. (Well about 15 minutes away on the number 30 bus.)
I joined the ONS with a rather ambitious, if nebulous, set of objectives that read something like;
“..inspiring and leading development of innovative rich content outputs for the ONS website and other channels, which anticipate and meet user needs and expectations, including those of the Citizen User.”
I think you’ll agree that sounds just vague enough to be fun.
Now I’m going to be honest. I didn’t really do my due diligence on the ONS before I started. I knew enough to get through the interview but I had my hands full turning 40 in the weeks before taking the job so I wasn’t really doing much research beyond looking up bars in Brooklyn.
One thing I did do was purchase a book called ‘Facts are Sacred’ — written by a journalist called Simon Rogers who at the time was the editor of the Guardian Datablog and would soon by moving to Twitter to lead their data visualisation team.
It was while reading this book on the days running up to my first day that I got an inkling that I might be in trouble.
Because in this very successful book from which I learned much was the following line;
“..it [the ONS website] is also the world’s worst website.”
OK then. Just one persons opinion.
How wrong I was.
A couple of weeks later I woke up on a Saturday morning and shaking off my standard hangover I reached for my iPad and checked Twitter.
Yes. yes. I realise how sad that is.
Bleary eyed I was surprised to find an unusual amount of mentions waiting for me. I immediately assumed I had stepped in to some Twitter storm the previous evening after a few too many Doom Bars but on closer inspection it was clearly more than that. It was the sort of thing that makes you long for those days stacking shelves at your local Tesco.
Tim Harford is an economist, journalist and broadcaster. He is the author of “The Undercover Economist Strikes Back” and the million-selling “The Undercover Economist”, a senior columnist at the Financial Times, and the presenter of Radio 4′s “More or Less”.
Tim Harford is also not a fan of the ONS website.
On Saturday the 8th June he used his Financial Times column (print circulation around 250,000 worldwide not to mention online views) to basically give the ONS website a solid kicking.
The line that was most striking — and tweeted — was one that will stay with me forever;
“the ONS website is a national embarrassment.”
Now I’m betting bad as some of your projects have been you have never had a best selling author use his national newspaper column just to give you a bad review?
Can’t get worse than this I thought to myself. I mean seriously the site is actually two years old at this point people must be tired of complaining.
You’d think I would get tired of being wrong.
There was even a whole Parliamentary Select Committee (PASC) hearing in to the failures of the website which said;
“The Office for National Statistics website makes figures hard to find and statistics are often presented in a confusing way..”
That line always makes me think of this meme
A recommendation from the report was simply;
“The ONS website must be improved.”
Peter Wilby in the Guardian pointed out;
“..the Office for National Statistics website is virtually unnavigable.”
Chris Giles, who is the senior Economy Journalist at the FT, has been more succinct on a couple of occasions;
“the ONS website — aaaargh”
I don’t mind a challenge. But come on!
Now if any of you have encountered the publicity machine that is the Government Digital Service you are probably wondering why those self styled white knights of public sector websites were not riding to the rescue at this point.
Unfortunately in this case, but sensibly in general, the ONS is independent from the Government of the day to ensure that statistics are not subject to political pressure at source. It is a difficult balance but for the purposes of this story the outcome is that there was no cavalry riding in to save the day. We were on our own.
At this point I had started to think of my role as being responsible for creating a kind of online sleight of hand. A digital art of redirection.
We were having some decent success on social media and with some of our editorial work. We were shamelessly cribbing from people like Buzzfeed and Ampp3d and while it wasn’t always popular with a vocal percentage of our audience it was reaching further than our usual stuff.
However I will say two things.
(1) If you are going to do a semi light hearted piece on divorce stats make sure nobody senior in your organisation is going to take offence.
(2) Always plan to be as pedantic as your audience.
“..factoid is a questionable or spurious (unverified, false, or fabricated) statement presented as a fact, but without supporting evidence.”
Not just a feature on the Steve Wright Show on Radio 2 https://twitter.com/bigshowfactoids
It was clear however that there was only so much we could do to distract users from the failings of the website so we needed to step up and make some serious changes.
So we started work on the two most blatant failures of the website — the navigation and the site search.
So nothing important then.
We started with the IA stuff and we really wanted to prove that we were ready to work in a user focused way.
We concentrated on our statistical taxonomy — this is where most of our problems were identified.
The first step was to review our existing taxonomy to see what was there, what was missing and, most importantly, how our users were interacting with it.
We did this by looking for patterns within our web analytics, reviewing every single item in the taxonomy (over 1400 and 5 levels deep) and by reviewing hours and hours of video taken from our previous usability studies.
Armed with this information we then asked a range of users (around 200 of them covering a wide range of demographics; economists, media, academics) to perform a card sort test.
We then analysed the results, highlighting topics where user opinion was split as to where they should be grouped and formed a new taxonomy.
We then performed a series of tree tests.
Again, a number of users were interviewed whilst completing these tasks to further understand their reasoning and motivation.
At the end of each day the taxonomy would be revised based on feedback and presented to new users the following day. In line with our user-driven approach we also asked a number of users to perform the same tasks using our current taxonomy to give us comparability.
In total over 130 users took part in the tree tests and the results clearly showed our new, user-driven taxonomy out performed our existing one.
It took users half the time to complete the tasks and our task success rates were almost 30% better than that of the existing taxonomy.
The new taxonomy reduced the number of themes from 11 to 4 making it more relevant to the statistics we produce.
Feedback from participants suggested the new taxonomy to be intuitive, making it quicker to find the content, although the user experience needs to be improved once they find what they are looking for.
I’m not going to lie we were feeling pretty chuffed with ourselves. The improvements to the navigation were testing well. The parallel work to move to move to a better internal search was coming along nicely and a significant amount of user feedback regarding search was being captured around the edges of the taxonomy work so we were confident we were going in the right direction and the prototype was providing the kind of improved results we had hoped for.
For a brief shining moment we thought we had cracked it.
Again how can one man get so many things wrong.
On Saturday 25th January while I was sat in a room in London’s City Hall leading an unconference session on the future of statistics on the web I received a text message telling me that the changes to the taxonomy had been successfully deployed.
I allowed myself a little smile and continued with my session.
By Thursday 30th though an unidentified bug created during the deployment had caused the performance of the website to degrade to such an extent that we ended up losing the site for 11 hours! from 11am until 10 pm. In the middle of the week.
This was straight up one of the worst days of my career.
Though I did manage to have a little fun with it.
I decided to take over the main corporate Twitter account for the extent of the outage — there were some unhappy users around and if anyone was going to get the grief it seemed fairer for it to be me rather than someone in my team.
I was like a teenager getting the keys to his old mans car [I assume — I never did learn to drive.]
The aforementioned Tim Harford did us a major favour by adding an element of humour to the situation in an early tweet. People were unhappy but there was a very British dark comedy element to many of their complaints.
I made a bit of a snap decision that I would work with that tone rather than play it completely straight. The opportunity to put my social media principles in to practice was too good an opportunity to miss and so..
There was really no going back at that point. As an organisation we aren’t really known for our self-depreciating humour. I tried to make sure I kept the actual information updates going.
Sometimes I had a little more fun.
But I tried to add a little value and share some links to sites that people didn’t seem to be as aware of as they should/could have been.
A few people seemed to think it was a decent approach.
I managed to send my final tweet of the evening at 21.55 from a train just before heading in to the Severn Tunnel.
The reality is that it was my colleagues who were pulling out all the stops to get the website back up and running who were doing the real work and mainly what I was doing was trying to offer a bit of a morale boost and distraction.
Bad as it was that day I don’t think we really realised the predicament we were in. There had been some concerns about the fragility of the site before but when our IT guys really got in to the nuts and bolts of things those concerns only grew.
I’m not going to go in to the nitty gritty of went wrong on the technical side. Some of it is beyond my understanding. Some of it is not my story to tell. Suffice to say it was bad. Really bad.
Sunsetting. It is a term you here from the likes of Google and Yahoo regularly. Usually a few months after they have spent 10s of millions on a startup they then give up on.
It isn’t really something you can do with a major Government website that supplies the data that runs the economy.
If only.
Now despite some helpful advice I have received on Twitter. No we can’t just build the site on Wordpress. There are all sorts of complexities in the site that make it awkward. Not as awkward as we seem to have made it up until now for sure but complex none the less!
That said one of the favourite things I have read in recent months was from Mike Dickerson, the Google engineer who (essentially) volunteered his time to lead the team that fixed the Healthcare.gov site in the US after its catastrophic launch;
“It’s just a website. We’re not going to the moon.”
While I had a little fun at the expense of GDS earlier on the fact is the Service Manual they published has been invaluable in planning for this. Not to mention the fact they have shared so much of their code on Github.
That gives us a place to start. A place to start that isn’t spending six months wrangling with vendors over CMS choices. Especially as those CMSs will inevitably fail to deliver.
One of the things I did to start thing was to accept I needed smarter people than me working on this if it was going to be a success. I am lucky enough to work with some good people on the user research and content side of things but I needed more. So I submerged myself in the swamp of despair that is Government procurement and started trying to get some help.
The team at the Open Data Institute answered the call.
For those of you who don’t know the ODI were founded by Tim Berners-Lee amongst others and thus have a staff with a pretty solid pedigree when it comes to working with the web and open data.
They provided us a technical vision to aim at that was both user focused and technically groundbreaking. Whether it is actually achievable in our timescales is another matter.
One of the [many] challenges we face at ONS, which the ODI team acknowledged, is the wide range of users that come to the site.
During all the user research we have done we settled on three main user persons. We call them ‘Expert Analyst’, ‘Information Forager’ and ‘Inquiring Citizen’, each with a unique set of goals, behaviours and motivators.
As an organisation we have traditionally focused on the experts — though that isn’t to say the website has served them particularly well! The idea of a ‘citizen’ user has been pretty poorly served over the years and there is a great deal of debate about whether this is an audience we should cater for or whether ONS data should allow other channels to serve these users (i.e. the media or applications built on top of ONS open data).
Anyway trying to develop a site that serves these persona types equally is a headache.
So I bet you are all expecting a happy ending? Well sorry to disappoint. On this day, on this stage I can only say we are working on it. We have reams of user research. A wall of index cards filled with user stories. A Github account full of Bootstrap wireframes testing out ideas. An incoming fresh development team and some big ideas.
Hopefully one day we’ll have a site that is less of a “national embarrassment.”
But not today.
Like I said — a Shakespearian tragedy. Or maybe a comedy of errors.
THE END.
- with thanks to Jonathan Porton who wrote most of the user testing portion of this talk.
