There is a design without testing.. and then there is a design without right testing. Both are equally dangerous whether it’s your digital application or a machine.
Alongside UX, I am aviation enthusiasts as well. And there is no better example to explain design without testing then the Comet disaster.
The Comet (de Havilland DH 106 Comet)
Design with not enough testing!
In 1952 the first aerodynamically clean design passenger jet with four engines buried in the wing roots, a pressurised cabin, and large square windows took the skies for the first time.
The de Havilland DH 106 Comet was the world’s first commercial jet airliner. Developed and manufactured by de Havilland at its Hatfield Aerodrome in Hertfordshire, United Kingdom. First prototype Comet 1 was built in 1949, just over 2 years before the debut.
Design and features were praised globally within the airline industry and the passengers alike. However, within the first year of service, 3 Comet aircraft were lost after suffering catastrophic in-flight break-ups resulting death of everyone on board.
What went wrong?
Despite being the state of the art aircraft of the era, Comet suffered a catastrophic failure. And when you hear the reason for the crash, it gets even more puzzling. Investigators found the main cause of these deadly crashes were the design of the window and escape hatch! That’s right… windows design.
The design of window did not cater for extreme compression and decompression, fuselage go through during each flight. The engineers did test the prototype, but not rigorously enough to identify the extreme forces involved. Only after the disasters, they grounded the fleet and started testing the aircraft and discovered the design flaws.
It was a brand new design, therefore testing should have been even more in-depth and rigorous. I will leave the link here if you want to read about the disaster further. Aerospace Engineering Blog.
Comet disaster is a reason we see oval shape windows in today’s aircraft. Design was eventally fixed but at the cost of many lives…
Testing without the right users!
From the last decade or so, England is converting its motorway network to smart using technology to monitor and manage the flow of traffic. The technology is controlled from regional control centres which can activate and change signs and variable speed limits.
Part of that was to convert the hard shoulder into an active lane whenever there is a traffic load. And in an event of a breakdown, shut the lane with the broken-down vehicle using motorway communication signs.
The first smart motor was M42, a road outside Birmingham (if you live in the UK, you will know). The government used this as a testing ground to validate the idea of the smart motorway. If you have driven on the M42, you’ll see the motorway is most of the time chocked with the traffic and you hardly get a chance to drive past 40 or even 30mph.
So what went wrong here?
After removing hard shoulders, the government introduced emergency refuge areas on M42 after around 600 meters apart. In case of breakdown driver can pullover in a safe place and wait for the rescue (what hard shoulder does really). The plan and testing proved to be a success and the government started to roll it out nationally.
That’s where the problem started to emerge. M42, drivers were different from M1, or M6 or even M25. Using data from the M42 pilot, the government started converting other motorways to smart. So much so, the government increased the distance between each emergency refuge area to 2.5miles, thinking if 600 meters was fine on M42, it should be fine to increase it.
This change has resulted so far in 38 deaths in the last 5 year. When people breakdown in the active lane, they had no choice but to stay in the active lane and hope that someone in the control room will see them and shut the lane. Unfortunately, in some cases, it didn’t happen and they lost their lives.
What government failed to understand that driver on M42 is different than M6 or M1. Their driving habits and requirements are different. I use M4 regularly, but when I was using M25 for a few years, the way I drove on M25 was different the way I drive on M4. Had the government piloted each motorway and changed the plan accordingly, it would not have resulted in causalities.
What to learn from this?
In Comet case, I learnt that when designing something new, you have to test the prototype as rigorously as possible. Since it’s a new design, you will have to make sure the design does not have major flaws. Minor can be corrected later, but major flaw can cost you product as well as the user their lives!
In the case of Smart Motorways, all government has to do is to test each motorway separately. Yes, difficult but at least should have got more qualitative and quantitative data instead of the relying on single pilot.
The reason for this article is that, whether you are an engineer/scientist or a UX designer, if you don’t test your finding and design or don’t use the right user for the testing, you are not only risking the failure of your product but risking people lives as well.