Design in time of data-driven decision making
Originally published — and most updated version — at: http://franrosa.com/articles/data-driven-design-and-decision-making.html
Nowadays data-driven decision making is considered a best practice among internet companies, both because it’s easy to collect and analyse data on users’ behaviour on internet related services, and because relying on data to make decisions is considered the opposite of having preconceived ideas that can be biased, or proved wrong.
Use of data-driven decision making as peaked together with agile software development, because every small feature or change can be tested and implemented or discarded on its own results. Data is used before, during and after any feature or change is introduced, analysing data to make hypothesis on what changes or new features to work on, measuring its impact through implementation on a group of users or using A/B testing, and deciding to implement and keep development, or discard and go back to a previous state based on posterior analysis of data.
Big companies like Google or Facebook have become leaders in that way of working because given their user base the amount of data they are allowed to collect is a competitive advantage, being companies with a small user base not able to collect enough data quickly enough, resulting sometimes in decision making based on data so small is not really meaningful. But ridiculous use of data has happened also inside some of the leaders’ teams.
‘Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such minuscule design decisions.’
Douglas Bowman, ‘Goodbye, Google’, 2009
It seems that being data-driven decision making such a young discipline, sign of the times, it is normal that mistakes are made. But limit the scope of design to data analysis and interpretation is as old as modern design methodology.
Walter Gropius founded Bauhaus art school in 1919 as a way to unite fine arts and arts and crafts. It was closed 14 years later, but its style had a great influence over the 20th century. In 1953 former Bauhaus student Max Bill founded the Ulm School of Design with Inge Aicher-Scholl and Otl Aicher, and their design teaching approach is the base of any design teaching even today. If Bauhaus had united arts and technology, the Ulm School of Design introduced scientific method: design as structured problem-solving. After Max Bill left the school there was a discussion over focusing solely on analysis and removing the role of art in design, but some professors including Otl Aicher didn’t agree to that shift. Internal conflicts continued until its closing in 1968.
‘Design is like history, something that is created rather than inevitable.’
Otl Aicher, ‘archithese 15’, 1975
What’s interesting of that discussion over design methodology in 1964 is that among considerations like ergonomics or semiotics, design based solely on data analysis should include business analysis. Economics, as well as sociology, psychology, philosophy, and politics were part of the curriculum from the beginning, but the main point was to substitute art for business. And in this context art means not only the craft but also the consideration of the designer as an individual making decisions based also on personal positioning both intellectually and aesthetically.
Both data-driven decision making and agile software development are methods based on business goals, not on producing the best product or on users’ best interest. It isn’t that good design and good business are mutually exclusive or forces in opposite directions, specially in user interface design. Stuart K. Card wrote on a paper at Xerox Palo Alto Research Center about invention and innovation on user interface design.
‘Our analysis of success led us to distinguish three varieties: invention success, engineering success, and innovation success. In fact, we suggested measuring impact quite literally with a Seismic Scale of Invention. Systems that score high on this scale we called Pioneer Systems; those that score low we called Settler Systems.
Most user interfaces are Settler Systems. They seek engineering success: systems that meet their defined objectives for usability or human performance (and objectives that are reasonable). Most methods in the HCI literature are oriented to the production of engineered Settler Systems.
Innovation success, success in the arena of engagement between a system and its public, is the third kind of success we considered. Innovation success is as an outcome of a much wider set of factors. An innovation success requires four basic ingredients to come together: technical merit, embodiment merit, operational merit, and market merit.’
Stuart K. Card, ‘Pioners and Settlers: Methods Used in Successful User Interface Design’ (PDF), 1996
Leaving aside the fact that paper was written 21 years ago, the notion of innovation and invention as different animals is still valid. Innovation is still today the highest goal for internet companies, meaning not the success of invention, but the success into gaining engagement. It was plain and simple for Kevin Systrom after Instagram’s controversial copy of Snapchat’s great part of their product last year.
‘This isn’t about who invented something. This is about a format, and how you take it to a network and put your own spin on it.’
Josh Constine, ‘Instagram CEO on Stories: Snapchat deserves all the credit’, 2016
This is a pill hard to swallow for designers, not only because innovation is not that valuable when it can be shamelessly copied, but also because if core parts of how a product works and is used don’t define the product, the role of design on those companies is not important at all.
Design is used mostly to try to improve business by playing cat and mouse with users, and creating ways to modify their behaviour to fit in the company business goals, be it by spending more time using their services, consuming more content, or buying more. Create new ways to improve conversion is the ultimate goal for everything, no matter what the purpose of the company is, or if it fits the users’ goals. Jeopardising the role of design in these organizations prioritising short term results, be it new users, engagement, conversion, or revenue, is creating a toxic environment for designers to see and work on the big picture, because on this ever-changing environment where nothing lasts there is no more a big picture any longer.
I’m sure design will change, because the world is changing. It’s just that now design is not evolving or growing, it’s just adapting itself to the state-of-the-art of business models in order to survive, and it is wrong in the long term game.