People vs. Code: Who ends up in the hospital? An NHS Case Study in How Software Gets Blamed For Everything

People talk a lot about digital transformation. I talk a lot about digital transformation because I am in tech sales and those are words we tech sales people say these days. Here are a few of IT sales’ favorite words by year, perhaps you’ve heard them a million times: digital transformation (2016), big data (2015), cloud (2014), consumerization of IT & BYOD (2013), DevOps (2012). We say these words a lot because they sound clever and because it makes our managers happy. But is IT transformation always good just because it’s trendy?

Wow! Not if you were in a leadership position at Cambridge University NHS Trust’s Addenbrooke’s and Rosie Hospitals!!

AN NHS HOSPITAL DID WHAT? TO CHILDREN?? SURELY YOU’RE MAKING THAT UP!

With likely the best intentions, leaders reportedly spent £200M on an Epic paperless records system in 2014. There were … technical glitches … in the roll out. Patient records were lost. The system caused mistakes in prescriptions… for children! Staffing issues were exacerbated by the extra time it took everyone to learn the new system, wait for it to complete records requests, and having to write everything down a second time on a piece of paper in case the system lost it. Important communication broke down. Patient care deteriorated. Within a year, the hospitals were put into special measures after losing £1.2M per week! The CEO resigned in September of 2015. Thanks to the professionalism of the staff no one was hurt, although the Care Quality Commission (CQC) report claims there were several “near misses” of mistaken prescriptions in the pediatric department prevented only by the vigilance of the staff.

Total time elapsed between well intentioned software purchase and CEO resigning over nearly killing children: 11 months.

Software is hard! I am not. Making this up.

ANOTHER SIDE OF THE STORY

“An incompetent government IT failure,” so the popular story goes and so it was reported in the press. But was it really?

It is tempting to blame the people and software involved because this flatters all our preconceived stereotypes about inefficient government, incompetent IT and technology doing more to impede than improve our lives. But are they any different from us? It wasn’t a bad idea and it wasn’t necessarily bad software! Paperless records are great! They improve cooperation, decrease miscommunication, and free up staff to provide better care. Even the CQC report admits, “EPIC had improved efficiency within the department giving staff better access to patient information.” The CEO made a good decision in deciding to implement them. The software functioned well mostly. What went wrong?

It is also easy to blame the NHS. I am American. I am accustomed to going to the hospital with a minor ailment and returning with a major bankruptcy. There is a check box on US medical forms as to whether you’d like Chapter 7 filed automatically or if you prefer to do it yourself later. But lucky me, I’ve got Donald Trump working on a solution. As an impartial outsider I’ve found the NHS to be compassionate, efficient, and tech forward. In the UK, you can even safely see your GP without an attorney present! And who can argue with the price? Similarly, I have great respect for those public servants who sacrifice higher salaries in the private sector to fight the daily frustrations of government IT working to ensure we all get better care. Good people. So again what went wrong?

WHAT REALLY WENT WRONG: SOFTWARE CAN BE SCARRY, BUT USERS? USERS ARE TERRIFYING!

In reading the 157 page CQC report a few things stand out. The underlying issue was not the software. The underlying issue was the funding crisis and chronic under staffing common in most hospitals, and most businesses, these days. But the Epic software still got called out by name 130 times in the report! Marketing must have been thrilled! When staff were interviewed about the root cause of underperformance they weren’t about to blame themselves or their colleagues, they blamed that annoying bit of new software that had been grating on them.

Herein lies the problem. If your software is perfect you will be taken for granted and forgotten, because it will function seamlessly as part of your users’ every day lives. If your software under performs, they will blame everything from their lack of productivity to the poor taste of their coffee on you. You will wish you were forgotten. They will complain to their children at dinner about you. Reporters will write articles. You will not like the Google search results for your name. A successful software roll out hides many failures, but a high profile, expensive botch will render all of your other successes meaningless. Your software can be good, but if users don’t like the implementation they might blame the failure of an entire business on you. It’s not fair, but it is reality.

One can blame the unrealistic expectations of users, but is it really their fault? They have been trained to expect intuitive functional software. Blame that on Google and Facebook. So now you have to compete with Sergey Brin and Mark Zuckerberg. Mark Zuckerberg makes billions. How much do you make? Again it’s not fair, but it is reality.

“It’s easy, why can’t you be more like me?”

A SIMPLE SOLUTION: DON’T FORGET YOUR USERS!

Don’t forget your users! Think of your users first and the brilliant capabilities/fantastic ROI of your software second. A simple way to do this is to monitor user transactions. You can see problems as they arise, in context and in real time, with greater clarity than the users themselves. Not only do you know exactly what’s going wrong (i.e. 3rd party data base transaction) and how it is impacting people (slow patient records transfer) you can precisely identify the code, infrastructure or network at fault in minutes, not hours. If you are reactively using logs, stop it. Monitor your users, not your servers. You don’t have to buy AppDynamics just because we have a bedrock culture of excellence in everything we do and happen to produce the best application performance monitoring platform in the world, but you should do something! Think of your poor users! In Aldenbrooke’s and Rosie’s case simple user transaction monitoring could have identified where records were being lost, where bottle necks were slowing transactions, in short where users felt the pain and frustration they later took out on IT. The problem was not bad software, the problem was being blind to where code conflicted with people. So they threw the software under the bus. Can you blame them? The beauty of application intelligence done right is not just better software performance, but better human performance & cooperation. Better implementation of software leads to better culture. Don’t forget your users, or they won’t forget you!

First, look after your users. Hopefully they will never have to testify to the CQC, but if they do, at least they won’t blame you.

They will probably blame HR.

- MK

Please remember to thank all our NHS professionals you see today! If you have similar horror stories or lessons, I’d love to hear about them in the comments.

If you would like to be part of our culture of excellence at AppDynamics and take advantage of our fantastic platform our trial is completely free for anyone who likes this post. Download it here and be amazed!

Thank you to Sooraj Shah’s omnipresent reporting for making me aware of this story. Give him a follow or scoop @sooraj_shah. All contents of this article are entirely my own opinion/research and do not reflect the views/consent of my employer, AppDynamics. CQC report here.