Never ask users what they want

(aka a Software Engineer’s take on the Brexit vote) (aka how to do good requirements capture)

Back near the start of my career I worked for a large UK university library on a project to capture and publish PhD theses electronically. This was to increase the access and availability of previously hidden research.

Prior to that, PhD theses were submitted to the library in print (two bound copies) and held in Special Collections where readers could come and view them.

My colleague and I were tasked with understanding the traditional thesis submission workflow and their subsequent usage in research, and how that would translate into a digital environment.

One of the things we did was go up to Special Collections to ask them how it currently works and what they would like to see out of the new system.

At my university at that time Special Collections was based on the top floor of the library, and if you are a libraries and archives nerd, you would have loved it. Rows of old books and manuscripts, drawers of maps, the odd physical object which didn’t fit anywhere else; a wonderful place. PhD theses also took up a big chunk of space, and their presence was divided into two areas:

  1. The stacks themselves — rows upon rows of PhD theses going back well over a hundred years
  2. A reading room where researchers could study these theses

Two copies of the thesis were delivered to Special Collections once the student had completed their qualification. One copy was placed in those stacks for use by researchers, and another copy went into deep archiving, not to be touched again if at all avoidable.

Researchers would come to the department and request a thesis. It would be delivered to them in the reading room, and when they were finished it would be returned to the librarians who would return it to the stacks. It did not leave the department.

It’s very important said our colleagues in Special Collections that users understand their copyright requirements for these theses. For each thesis that is borrowed a user MUST sign a form that says which thesis they are reading and that they have read the attached copyright notice. They cannot look at the thesis before doing this. We MUST have an equivalent feature in the online version — we want users to “sign” for a thesis before they can view it in their browser.

Now, copyright, as you may be aware, is a complex beast, and of tremendous importance to our large institutions. A lot of time in the library world is spent (rightly or wrongly) hand-wringing about copyright. So, we took their requirement seriously and put quite a lot of thought into how we were going to meet it.

But something wasn’t right.

For a start, we wanted to ensure that these theses were public. The whole point of our work was to unlock all this hidden research and make it available to the world, to readers who would otherwise never have known it existed or never have been able to physically roll up at our building to see it. Introducing the idea that someone would need to be sufficiently well identified to “sign” for something electronically was going to be hard and a barrier to the usage that we wanted to encourage. We could maybe manage that for our own on-site users who had staff numbers and usernames, but what about researchers around the world, or even members of the general public?

Secondly, the idea that you have to explicitly agree to a copyright notice is odd. Copyright simply exists, whether you have agreed to abide by it or not. There’s an argument, certainly, for making it easier for end-users to see what the copyright terms are, for their own information. But making them “sign” that they have read the agreement? We didn’t think so.

This requirement did the rounds in our team for a little while, and eventually we went back to Special Collections to discuss it again, to see if we could convince them that this wouldn’t be sensible. In the discussions that followed a genuine revelation occurred that I still think about today (hence this post) and which helped me become a much better engineer.

It turns out that readers signed a copyright agreement, because it was knowing who the reader was that was important. The fact that the piece of paper contained a copyright agreement was almost incidental — the reader must identify themselves on paper, might as well also give them the copyright terms so they are signing something official-looking. Ok, so why do you want to identify the reader? Well, we need to know who they are in case we need to find them again and ask them what happened to the one and only public copy of the thesis.

Let that sink in for a moment.

Identity is important here because and only because the resource is a singular physical object. As a result, the readers interacting with it were important in case they stole or damaged it.

As a result, this “requirement” evaporated into thin air. Copyright was not the point of the signature, and in a digital environment there is no singular physical object, so we don’t care who has their hands on it!

What mistakes had we made here, in capturing the requirements for our new system?

Most critically, we had asked the users what they wanted rather than trying to understand what they are trying to do. We thought they had given us a requirement whereas they had given us a proposed solution to [an imagined] problem. The copyright form had been in place for so long (also going back a hundred years or more) that its true purpose had been lost, or at least mislaid. When asked about it initially our colleagues had responded with a proposal to replicate the current physical workflow in an electronic form, without giving it any more thought. It took a detailed discussion on this process for the true purpose to once again come to light.

Users will do this, so watch out for it. They do not (nor should they be expected to) understand how your new system is going to change the very nature of their work. They will almost always ask you to replicate whatever they do right now, however inappropriate that would be.

Don’t ask users what they want, instead ask them what they are trying to achieve.

What’s this all got to do with the Brexit vote? Is it just shameless click-baiting to get you to read my blog? Perhaps a little. But also, there’s a serious point which I have only now been able to articulate, and which I haven’t really heard discussed elsewhere.

First, some background information, for those of you not following UK events. In June 2016 the UK voted in a referendum to leave the European Union. The ballot contained two options: “Leave the European Union” or “Remain a member of the European Union”.

You might already see where we’re going with this.

I want to be clear before we go on that I’m not interested in convincing you that you should take one side or the other in this debate. What I want to do instead is approach this like a Software Engineer should — by understanding what the developers (politicians) should be constructing for their users (the voters), based on what those users are trying to achieve in their daily lives, and what would make those lives better.

The UK government have taken the imperative from the voters to “Leave the European Union” as a requirement. But it is clear that this is really a proposed solution to [a poorly understood] problem. They have asked their voters what they want rather than asking them what they are trying to achieve.

What is concerning is that we may therefore be implementing a solution which will not solve the actual (or perceived) problems. We should go deeper into the true requirements.

And there are problems with the EU and EU membership, of course, nothing is ever perfect. To pick a very specific example, my father is convinced that doing regulation across such a large heterogeneous area can’t suitably take into account local variations in requirements. I don’t know if he’s right, but he’s an engineer, and does think that, so it should be talked about. Some people feel that the democratic structure of the EU is inadequate, others feel it is too bureaucratic, and others are concerned about immigration.

For the government to simply agree that leaving the EU will fix all these issues the way the electorate want is poor engineering based on poor requirements analysis. Leave voters should be genuinely concerned that they aren’t going to get the change that will fix their actual issues, no matter what the final deal turns out to be.

Let’s give a concrete example. Among other things, my father is concerned that he can’t buy a hot-fill washing machine, because of EU regulations. He’s wrong, of course, you can buy a hot-fill washing machine (though it is true there aren’t many), and the EU doesn’t ban them. In reality it’s a manufacturer’s decision based on what’s efficient and convenient and what works for most consumers. When we leave the EU, he will still have the same level of access to hot-fill washing machines because companies that make them will continue to make them, and companies that don’t still won’t. His problem is not fixed.

If instead the government had asked him “what do you want to achieve?” and he’d said “to wash my clothes with a hot-fill washing machine” and they had then said “why do you want to do that?” and he’d said “it’s more energy efficient and thus cheaper to run”, then they could instead have addressed that problem directly. I don’t know what the right answer is to this, of course, but the government will have levers of power over the cost of the washing powder (biological washing powder is more efficient in cold-fill washing machines), the machines themselves, and the public consumer information regarding your options. If this problem alone needs to be fixed, it seems that there are realistic, relatively straightforward options.

This is how real requirements capture and analysis is done, leading to good, clean, efficient systems design.

Obviously I understand that the real politics behind Brexit is far more nuanced and complex than this, containing good and bad actors, hidden agendas and unsaid requirements. It may be that the correct solution to the nation’s issues is to leave the EU — but that is a decision for the engineer based on the requirements, not the user based on their wants.

Just sometimes you look at government as an engineer and wonder if applying a few more engineering principles could help somewhat.

Don’t ask voters what they want, instead ask them what would make their lives better.

And what about our electronic theses project?

Because we caught our mistake, we were able to do a quality requirements analysis. We were able to go ahead and put our institution’s theses online with unencumbered access for all users wherever and whoever they are. We were one of the first universities in the world to put our theses online, and I’m pleased to say that 16 years later that repository of knowledge still exists and continues to grow. We laid much of the ground-work of the future for online theses, including developing metadata standards and open source software to support it. That project eventually also informed the British Library EThOS service, which places electronic theses in the hands of a legal-deposit organisation with a serious mission to make and keep this material available.

And we also learned that you should always play the “why game” with your users. When they tell you they want something, ask them why. And keep going until you get to something concrete that you can build your design on.

Richard is a Founder at Cottage Labs, a software development consultancy specialising in all aspects of the data lifecycle. He’s occasionally on Twitter at @richard_d_jones