Many organisations are striving to develop and implement eHealth solutions in low-resource settings, to improve immunisation service delivery, child nutrition, maternal health and many other health areas, where the lives of children and mothers are affected.
These eHealth solutions promise to generate individual-based electronic health records, and improve data reliability, availability and time-savings for health workers. If you look at each of those solutions individually, most of them are more sophisticated and far more user-friendly than what we find in Sweden for example.
But the problem is that most, if not all eHealth solutions fail and are not sustained in low-resource settings. There’s even a name to this problem — Pilotitis. You may have heard about it.
For us at Shifo, eHealth solutions become successful when two conditions occur:
1) Systemic change — the new eHealth solution was able to completely replace old and bulky paper forms and monthly reports that health workers used to fill in
2) Local ownership and sustainability — local governments have taken over the responsibility to sustain and finance the operating costs of the new eHealth solution
Everyone wants to replace existing paper forms — those big bulky registers that health workers must fill in during immunisation sessions in health centres or community outreaches. In many countries these registers have been in use since the 1970s, if not earlier. Nobody likes them, especially health workers.
But yet, it is difficult or almost impossible to replace these immunisation registers, tally sheets and other paper forms with new shiny gadgets with beautiful apps, which in many cases even resemble the paper forms, in electronic format. Why? Low-income countries have struggled to replace paper systems with electronic alternatives, owing to the significant resources required for implementation, including skilled human, financial and technical resources, and the high cost of set-up and maintenance, due to poor existing infrastructure, frequent power outages and network failure, and lack of security in health facilities.
Any person who has been dealing with eHealth for a while knows that the true challenge for eHealth solutions starts during operations. As soon as health workers start using these solutions, several challenges immediately become apparent. Electricity blackout in this clinic, data synchronisation problems in another. A health worker became ill/left the job and a new one needs to be trained on how to use the gadget. An end-user accidentally dropped the gadget and it needs to be replaced. The gadget is stolen. A health worker forgot, or was unable to charge the batteries and the gadget stopped working in the middle of the service delivery. The system cannot be logged into, or some functions in the app cannot be used due to bugs or other problems, or the app needs to be updated. End-users lose motivation and need to be incentivised. The list of problems can go on and on, almost indefinitely.
It means that during these times, patient data is not entered using the gadget, or entry is delayed or duplicated. Some of these problems may take weeks or months to solve.
Did you know that if a health worker does not use the electronic device to input patient data for just one day, this equates to less than 95% data accuracy/completeness for that month?
The Ministries of Health of most countries require that health facilities achieve at least 95% data accuracy/completeness between child health registers, tally sheets and monthly reports, given that high quality data enables better decision-making, and planning for interventions. Health facilities that fail to achieve 95% data accuracy or completeness receive a penalty.
It is easy to imagine what would happen if, in the face of the aforementioned problems related to eHealth solutions that are beyond their control, clinics are penalised because of a lack of data accuracy/completeness. It’s a big responsibility on behalf of those organisations that are bringing eHealth solutions.
In the current system, there is a proper checks and balance mechanism to check for data accuracy or reliability. People who do M&E check immunisation registers, tally sheets and monthly reports, and they can also crosscheck with Child Health Cards. In many countries this method is used on a quarterly/annual basis, but in many instances, health facilities that implement eHealth solutions either don’t know how or don’t measure data accuracy/reliability of a new eHealth system. True, it is a resource-intensive activity but lack of resources is not the only reason. In many low-income countries, the implementation of eHealth systems is done in tandem with continued use of paper forms, which can lead to health workers inadvertently incorrectly transferring paper-based data to the electronic system. Checking data accuracy during service delivery would be inefficient and disruptive.
So how should we do M&E on the new eHealth solutions to measure data completeness/accuracy/reliability? What checks and balance mechanisms should be in place, so that we can measure data completeness and reliability regularly?
Until we secure reliable operations of eHealth solutions for end-users and measure data completeness/reliability/accuracy regularly, new eHealth solutions will be more of a toy/by the way/fun to use from time to time, rather than replacing the existing paper forms, and being seriously considered as part of the country’s bedrock for generating reliable data.
Local ownership and sustainability
Local ownership ultimately is about local governments taking over responsibility for sustaining and financing the operating costs of new eHealth solutions.
Even if you managed to get everything right and the new eHealth solution is generating more than 95% reliable/complete data, this may not guarantee success of the eHealth solution.
Lack of local ownership and sustainability by local governments is the most important reason why many eHealth projects fail, period.
Most eHealth projects don’t do a cost-benefit analysis of the current system and new eHealth system. There are many reasons for not doing so, one being lack of resources. In the projects where a cost-benefit analysis is done, they try to measure every single benefit and turn them into money equivalent. There is nothing wrong with this, but it is not effective.
If we consider the printing costs of the existing paper forms, it is roughly 50 cents (0.5 USD) per child per year for immunisation service delivery. The hard reality is that the source of funding for governments for eHealth solutions is the same budget allocation used to cover the printing costs of existing paper forms. Anything above this budget requires a lot of convincing and struggles on the part of the Ministry of Health.
Based on this, the big challenge that many eHealth projects have to overcome is to figure out ways to make operating costs as affordable as possible, so that it fits within the same budget — 50 cents per child per year.
There are very few cost-analysis studies that are published within eHealth projects in low-resource settings, but one done in India and supported by the Bill and Melinda Gates Foundation, shows that it costs approximately 5 USD per child per year to operate an eHealth solution within a specific heath domain. It’s hard to know how many gadgets per health centre this equates to, but the understanding is that one smartphone per clinic is what is included in the Indian study.
Even internal studies done at Shifo show that if we want to reliably operate eHealth solutions in the health facilities, this requires between 5–10 USD per child per year worth of investments from governments.
Asking governments to invest 10 times more money to operate eHealth solutions is like a “Mission Impossible”.
Certainly, the theoretical benefits are far more with eHealth solutions compared to existing paper systems, but the inescapable reality of lack of resources to sustain and finance operating costs is the main reason why most/all eHealth projects fail.
Does this mean we should all stop working with eHealth projects? Not at all.
Equipped with new insights and experience, we can develop eHealth solutions that can be equally priced or even more affordable than 50 cents per child per year in operating costs. And we can design solutions that can have a good chance of replacing existing paper forms, and truly bringing systemic change.
Let us start measuring the impact of eHealth solutions by the systemic changes we are able to make, and by the rate of adoption and financing of operating costs by the Governments.
This is what we decided to do at Shifo, because we believe it is the right thing to do. We hope this message will resonate with other organisations and governments trying to focus on strengthening data quality in low-resource settings.
We should all unite and work together to increase the knowledge and understanding of how to make a systemic change happen and how to ensure that every new eHealth initiative is positioned for success and long-lasting improvements.
Because ultimately it will affect the lives of mothers and children in low-resource settings.