Five ways we can improve health center data in developing countries
It is exciting to see the way data has become such an integral part of international development after project effectiveness was rarely measured for decades. That importance is exemplified in Global Health Corps, where 20% of all fellowship positions this year are in monitoring and evaluation or something related. This large-scale, global transition towards better data use, however, at times overlooks some of the challenges it creates.
I am currently working at LifeNet International, a small NGO that primarily uses sustained monthly medical and management trainings to build the capacity of faith-based, non-profit health centers in Burundi and Uganda. Fortunately, LifeNet International has been built around using data to measure its outcomes and drive decision-making from the beginning. LifeNet International is also not stuck in its systems — it is open and excited about finding new ways to measure output and outcome.
By working directly with some of our partner health centers and working with leadership on developing our monitoring system further, I have had the opportunity to see a range of data and monitoring challenges for health centers and global health organizations in Kampala.
Staffing and Turnover
In Uganda, Human Resources for Health has recently become a central focus in improving healthcare. The Ministry of Health, the religious medical bureaus that coordinate faith-based health centers, and donors such as USAID are working together to fix the chronic staff shortage and staff turnover plaguing every level of Uganda’s healthcare system. In addition, only two out of the 10 health centers we currently work in in Uganda have annual budgets that includes an HR plan to add staff based on need. Low budgets and lack of financial management and planning mean that even if turnover is reduced, the health centers we work in will still likely be understaffed.
When understaffed and trying to provide life-saving care, it is understandable that record keeping and data is not prioritized. This often means that medical staff are not documenting many of their practices, without realizing how this can damage the care they are giving. Understaffing can also translate to not having a trained cashier or accountant. At 2 of our health centers in Uganda, the accountant is a nurse taking on extra duties to fill a gap. In Burundi, the number is much higher — 25 out of 60 health centers have nurses serving as cashiers. This means health centers may not have reliable financial records, which in turn reduces the ability to plan for hiring new staff, which continues the cycle of understaffing.
LifeNet International’s model focuses on fixing some of these data issues. Our medical curriculum focuses heavily on the importance of documenting every referral, every procedure, and every step medical staff take to fix issues. Our management curriculum has our trainers work with the health center staff on management practices and improved record keeping. LifeNet International uses internally developed quality scorecards to look at the improvements over time, and for most categories, health centers receive 0s if they do not document what they have done, even if they perform the work perfectly. While this does not immediately fix staffing problems, it does work to fix documentation problems. In the longer term, better documentation, record-keeping, and budgeting could lead health centers to be able to better project and afford their staffing needs. LifeNet also includes Human Resources curriculum at clinics, which serves to enforce the use of job descriptions and hiring based on sufficient qualifications.
Baseline Data and Measuring Outcomes
Our quality scorecards capture a lot of information about our health centers’ quality of service and record-keeping over time. However, we want to be able to capture other changes in addition to these. How has patient volume changed over time as our work improves the quality of services available? Do our health centers become more profitable as we put these systems in place?
Answering these questions feels a bit like a trap. If we are acknowledging that the accounting systems are not accurate, we cannot use these numbers as a baseline to improve from, and it is difficult to measure our progress over time. If we use these numbers, we are using numbers from a system we know is not working and so, when we improve the system, we sometimes see lower (but more accurate) patient volumes or profit. How can we reconcile these two items? At my last job, I was serving as a research assistant on a number of experimental and quasi-experimental studies on international develop program impacts. We had beautiful, long, limitations sections where we could add these caveats and someone out there would read it. However, most of our donors do not have the time or interest in reading such limitations. As one would expect, they are looking for answers, not more questions.
Many projects try not to rely on existing data to avoid this problem, while others use it despite similar issues. While it certainly does not solve many of the issues here, I think data accuracy needs to become an outcome we measure consistently. We are well-versed in the power of data, and quality improvement is a job position and a focus in many settings. However, we rarely see improvements in data quality celebrated, because it is something that we take for granted. Including data accuracy as an outcome is a way to reframe this as an accomplishment instead of simply making up for past mistakes.
Fortunately, at LifeNet we are able to measure at least some of our data accuracy impact through our quality scorecards. We plan to pull together several data accuracy indicators we include to create one score that we can measure and report on. This is easier for us because data accuracy is a focus of our programming, but data accuracy should be an outcome for other organizations, even if their programs do not include improving data as one of their key objectives.
Data Reporting Fatigue
Most of our health centers have reporting requirements for the Ministry of Health, external donors, and the religious medical bureau they work with. These reports are well intentioned, and help groups track trends and show how government and donor money is being utilized in their health center. The Ministry of Health monthly report is ten pages of data recording for outpatient services, and an additional 3–4 pages for inpatient services. This requires a lot of time, and is due on the seventh of every month at the latest. A seven-day turnaround gives most health centers enough time to complete their reports, but not enough time to verify numbers, or check any numbers that appear to be mistakes. When donors ask for additional reports, this puts a high level of burden on health centers.
Additional reports or additional indicators may thus lead to both higher rates of data reporting fatigue and lower levels of data validation. When LifeNet has checked records books at some health centers against Ministry of Health forms and reports to religious medical bureaus, it has found that each report often has different numbers for the same indicator. Because of this, LifeNet International has decided to collect the same data as the Ministry of Health so that it can use already existing forms instead of creating new ones and new work. This should be something that more and more organizations seek to adopt as well.
Health Center Data Documentation and Use
Most of these mandatory monthly reports health centers focus on sending information to the Ministry of Health and to donors for their aggregation to see trends across health centers and over time. Unfortunately, the reports do not aim to help health centers use their own data.
In the last few months, the Ministry of Health reports have been updated so that they break data out into four age categories instead of two. These reports help the Ministry of Health make country-wide and district-wide decisions using informative data. This unfortunately means that at a lower level it is very difficult to use the data coming from the reports. Our health centers report on the number of females over age sixty that have visited their health center in the last month, but do not report on their total patient volume. This can be a big issue. For example, one health center we visited has had a decrease in patient volume for several months straight and was unaware of the extent of the trend until we looked at their ministry of health reports and pointed it out. While in Uganda clinics are required to keep a copy of their MOH reports, the Burundian Ministry of Health collects reports and returns them, certified, one month or more later. Few clinics make photocopies of the reports, so these clinics do not even have records of the totals they compiled for the last month.
Part of LifeNet’s mission is to improve the data culture at each of our health centers so that medical and management staff understand the importance of the documentation and data they are collecting and are able to use it to inform their decision making. In full, the goal is to use trainings to help health centers feel they have agency to make changes to their clinic based on the data they have and in turn that they start to use the data they have to make those changes.
As global health practitioners, it is not enough for us to collect data on our work to see whether we are making a difference. I believe we need to do the following:
1. We need to focus on improving the data available, and be upfront about the limitations to collecting baseline data instead of using data we know to be unreliable.
2. We should shift the paradigm so that improving data accuracy becomes an outcome we seek to measure and reward instead of just a challenge in measuring other outcomes.
3. We should minimize reporting requirements so that we focus on collecting information on targeted indicators, instead of collecting data on everything that could potentially be valuable.
4. We need to encourage health centers and other beneficiaries not only to improve data documentation, but to understand why it is important.
5. We need to work with our beneficiaries to understand how they can use the data they have.
LifeNet International has designed its training curriculum around some of these processes, and intends to align its programming with these recommendations moving forward. I invite you to reach out to me if you are interested in our initiatives or in continuing the conversation about improving data collection and use in global health.
Molly Brune is a 2015–2016 Global Health Corps Fellow and is serving as the Monitoring and Evaluation Officer at LifeNet International. She is based in Kampala, Uganda.