Brontosaurus & other erroneous design decisions caused by human error due to insufficient data

Published in

Bootcamp

8 min readMar 9, 2023

As a member of Generation X, I grew up when there was a pretty finite number of dinosaurs, and none of them were feathered. One of them was the mighty Brontosaurus, that massive thunder lizard that had an incredibly long neck, a massive head, and dragged its immensely heavy tail on the ground. Younger people — or parents whose kids loved dinosaurs, like mine! — might recognize that as the first design iteration of what’s now known as the Apatosaurus. It had a much smaller head and actually held its tail aloft, rather than dragging it around everywhere it went, which seems incredibly un-aerodynamic. If you don’t follow fossil discoveries, you might not have heard of the Bone Wars.

What were the Bone Wars?

I’m glad you asked! This was 130 years ago, during the beginnings of paleontology in the United States. There was a bitter competition between Yale’s Professor of Paleontology and museum curator, Othaniel Charles (O.C) Marsh and adventurer and academic, Edward Drinker Cope. Cope was actually in the field writing and directing digs most of the time, while Marsh managed and organized finds. The two men didn’t like each other, so instead of cooperating to advance paleontology more quickly — which would have made a lot of sense given their individual skillsets, but was not at all the way things were done back then — they raced to get new dinosaur discoveries into the annals of history. Matt Lammana, curator at the Carnegie Museum of Natural History in Pittsburgh, relates that there were even stories of at least one of the men telling their fossil collectors to smash other skeletons that were in the ground, to prevent the other from getting to them. Cope rerouted Marsh’s train of bones to himself in Philadelphia. Marsh scattered new bones from different eras over Cope’s site. In short, Marsh did a lot to delay the progress in the field!

Why should I care about these Bone Wars?

As a UX designer, it occurred to me that putting together the puzzle of data and having to satisfice when you have data gaps, is very much like what these two men were facing when finding incomplete skeletons and bones scattered everywhere. Remember, the year was 1877, not 2023. It was around the time of Jack the Ripper, to put it in context. Charles Dickens and Mark Twain were some of the most popular writers, if you’re a literary geek like myself. Around that time, Twain wrote a book called Pudd’nhead Wilson, wherein two seemingly identical babies were identified by the difference in the whorls of their fingerprints. No one had any idea about DNA back then (if you’re a DNA/genealogy buff — again, like me — I find it so cool that Jack the Ripper was at least “caught” posthumously by an AI system called H.O.L.M.E.S. There is an awesome documentary that explains how the system analyzed his patterns and tracked him down, but it’s oddly incredibly hard to find. You may have to search the dark web. Remind me to write about this, as we true crime fans love us some Jack the Ripper!)

Making decisions with insufficient data — we all have to do it

Back to making any decisions — not just design, so stick with me if you’re not a designer; this is still relevant! — with insufficient data. Please excuse me for that trip down memory lane. We all have to do it, not just in design. Our human selves have memory, reasoning, and time limitations — we also have to factor in our emotional investment in the decision. In this agile world of “fail fast, fail often” and “just get something out there, fast”, we often don’t have time to make the absolute best decision. We have to use an AI-like mentality. Reaching back to my grad school days where I studied human factors and AI in its earlier years, I believe that’s the greedy best-first algorithm — or a possibly non-optimal, but fastest solution — although as a UX designer, we never start from zero, so it’s probably a combo of this and the optimal solution derived by A* algorithm. It’s been quite a while; I need to — excuse the pun, I can’t resist — bone up on my AI algorithms!

Getting back to the two scientists, Marsh and Cope — so, in 1877, Marsh discovered a partial skeleton of a herbivore dinosaur with a long neck and tail, which he initially called Apatasaurus. Unfortunately, its skull was missing! Talk about a data gap! So a few years later, in 1833, he added the skull of another dinosaur — which we think was a Camerasaurus — to the Brontosaurus. Then two years later, he found another skeleton — this one he called Brontosaurus. However, it was really just a more complete Apatosaurus. Interestingly, paleontologist Elmer Riggs discovered the error in 1903 — that Apatasaurus was just a juvenile Brontosaurus — , but because of Marsh’s power and reputation in the palentological world, he wasn’t going to publicize his own mistake. Thus, the erroneous design decision wasn’t fixed until 1979, well after Marsh’s death.

How does this relate to bad product design decisions?

If you’ve been in UX for any length of time, particularly if you’re in a startup or small company with low UX maturity, you’ve likely seen some bad design decisions go out there. Unfortunately, this can happen when there is a high-powered, highly-paid, non-designer (HIPPO) who is running the show. You’ll definitely run into this person, so you need to find a way around it.

One, make sure that you understand this person’s opinion and perspective. Are they highly technical, but are trying to make decisions for a Line of Business (LOB) user? Are they downplaying the importance of a critical user task? If so, why? It’s important to have them talk through their reasoning, so that you can mount the best defense for the user. Do a cognitive walkthrough — I’ve found that a storyboard telling the user’s journey from their point of view or an empathy map can be very helpful. Technical people can be less impressed and swayed by qualitative analyses like this, so if you can, bring the hard quantitative data.

How can you bring the hard data?

If it’s an existing feature and you have any sort of analytics tool (besides Google Analytics Universal, which is pretty limited), you should be able to determine current usage of that feature. As a UX generalist, when I don’t have the data, I set up a data pipeline to get it. That means I dug into the analytics tool capabilities — this was Gainsight PX, which if you want to try this tool, please contact me for a thousand-and-one reasons not to — including creating a data plan for what we wanted to measure and why, setting up URL tracking for each feature and action we wanted to track, working with dev to ensure that these URLs were no longer randomly changed, and working with them to add unique CSS selectors to each button or link that we needed to track clicks on.

Case 1: Reducing cognitive burden caused by too many choices
In this case, I didn’t have usage data to confirm my belief that we only needed the two most-frequently accessed tabs — a list of flows and the dashboard that showed the flow’s progress and performance — plus one Admin tab that would chunk all of the management information, such as audit log and setting up notifications and managing versions. Instead, the technical HIPPO was convinced that all tabs (there were many!) were equally important, despite customer anecdotes, walkthrough videos, and internal expertise which showed the contrary was true. As any good UXer will know, the fewer choices you can give to a user — the 5+-2 rule — , the less cognitive burden they’ll experience and the easier they’ll perceive your product/service to be, which of course, impacts their satisfaction and retention. Combining the qualitative with the new quantitative usage data confirmed that the two tabs that I’d initially suggested were correct, but also that the audit log should be a top-level tab as well. Unfortunately, I didn’t have this data soon enough to keep the sub-optimal experience from going out, but could build support to get it revised. This is the challenge you run into when building a data-driven company. You have to start somewhere!

Case 2: Reducing time on task and keeping users in context
Another big case was one involving a critical task — that of resolving errors that occurred while creating or running a flow. My hypothesis was that users thought of all errors as being inter-related. As I read through more error messages and built my own data integration flows, this strengthened my conviction that not only did users need a quick way to get all information about an error in order to troubleshoot it quickly, they also needed a way to page between errors, as downstream errors were often caused by earlier errors while a flow was running. It was a paradigm shift for the company, which was used to thinking that each error was an individual event.

To bring qualitative and quantitative data to the equation, I put out surveys to gather quantitative and qualitative data on perceived ease of use of the UI in general, as well as for the tasks of creating flows & resolving errors in particular, watched customers walking through the UI, and used my QA testing skills to count the number of clicks and context switches. The surveys brought customer feedback this critical task took several hours instead of minutes. I calculated that for the two personas — technical and business user — my solution could reduce the number of context switches (changing pages) for each by 100% and reduce the number of clicks for the technical and business users by 65% and 77%, respectively (this was because the technical users wanted to dig into the details, while the business users were looking to resolve errors with one click). In so doing, I reduced what could be a half-day task to mere minutes, and customers and hard-to-please stakeholders (including the HIPPO!) were ecstatic in their praise. Even better, I’d formed a new data pipeline and process of triangulating data sources!

A picture is worth 1000 words

For the second case, here was the old design, where each error message was an individual event and there was no quick way to get error details (including the whole error message) or other relevant info, nor was there a way to page between errors that were often inter-related, where downstream errors were caused by upstream errors.