Child looking out of a window during a school screening programme in Zimbabwe.
Credit: Rolex
What we measure really matters. However, what matters most is usually the most difficult to measure.
In health care delivery there are many complex factors that relate to someone getting the treatment or intervention they need, at the right time and right place. A growing movement is underway which recognises that if truly sustained, high quality services are to be provided to everyone who needs them, there is an imperative need to make health systems stronger. The problem? It’s complicated, involving many factors, influences and moving parts.
Often, agreeing on how to measure progress towards universal health coverage is a major challenge. Even when clear metrics are defined, the way they’re gathered is fragmented and varies between different organisations and locations.
The good news is that in eye health, I believe there is a core objective that ultimately aligns the missions of organisations, individuals and populations. That objective is to increase the proportion of those living with an unmet eye health need who are having that need met.
For example, someone who cannot see due to cataracts or uncorrected refractive error has an obvious health need. Until they are located by health services, treated and regain their vision, that meet is not met. Once they are treated as required they are no longer classified as having an “unmet need”.
Defining the boundary of met and unmet is nuanced. It needs to be considered within local context as well as defined by and with those it affects – the population, health service providers and other relevant stakeholders such as local government. However, once the boundary is agreed, population survey methods can be used to develop a really good understanding of the current situation. This forms the baseline for action, which change can be measured against.
The problem we face is that the metrics which are most straightforward to report only tell part of the story. They are primarily “inputs” – for example, how many people were screened, how many were treated. These numbers provide good proxies for activity level, which eye health initiatives are held accountable for. However, if they are the only numbers we track, there is a risk that services could be reaching only certain groups of people or that treatment isn’t taken up by the most vulnerable groups. In most cases, this inequity could remain completely invisible, even where a programme appears to be successful in terms of the number of patients screened and treated.
Focusing on increasing screening or treatment activity is often needed to ensure income and demonstrate that good work is being done. But without sufficient visibility of whether unmet needs are being met, we cannot understand the system effects, and ultimately this creates pressures and tensions. Often this ends up as pressure to do more with limited resources, which usually means providing services to those who are easier to reach, relatively speaking (closer to the services, better educated, more mobile, able to pay, in non-minority groups, gender biases, etc).
The unintended consequence of this pressure is that we pay attention to those whose need is being met, and not those who are being left behind. The reason? Simply because we don’t see them and often don’t hear them.
A positive movement and message around equity in eye health (and health and social care more broadly) is growing. It is essential that we have the tools and data to ensure no one remains invisible. Not doing so will ultimately lead to ever-increasing equity gaps, greater health service instability, and future generations grappling with problems that we could have dealt with now rather than passing on. If we want to see systemic change, we have to focus on measuring what matters, not what is easy to measure.
A recent programme Peek has worked with, led by an inspired and dedicated partner’s in-country team, illustrates this point perfectly. There was plenty of good prior data regarding the numbers of people screened and treated over the previous years. Many people with an unmet eye health need had their needs met. However, those achieving ‘met need’ status (i.e. having their eye problems resolved) were unfortunately only a small fraction of the entire unmet need population.
The team needed to know the extent of the overall problem in the region and how long it would take to solve with current resources. The answer, based on current programming, was another 30 years, by which time the initial problem would have continued to grow. What looked like a success story – thousands screened and hundreds treated – was in fact masking a very low proportion of people with an eye health problem having their needs met.
In our experience, teams that are under pressure to deliver change may find this information initially challenging; it doesn’t fit into existing reporting structures and can raise questions about a programme’s effectiveness. However, we have found that it is exactly what eye health implementers want to see so that they can improve the lives of those they serve.
In the example above, within one month of reviewing the data on unmet need, teams working on the ground were able to adapt their approach by looking at the root causes of why people didn’t have their needs met and making changes to address the issues they surfaced. The result was that they almost doubled the proportion of people have their eye health needs met in a short space of time, and understood exactly where they needed to focus next to make even bigger improvements.
Eye health providers around the world are working tirelessly to deliver change and address the global vision crisis. But by focusing on what is easiest to measure, there is a risk that the organisations on the ground are being unintentionally asked to choose between serving the needs of their patients and the need of their reporting requirements. For the first time, Peek and other digital health tools are now giving eye health organisations in low- and middle-income countries the capability to measure what matters and adapt their services accordingly. The result is that those who need our collective efforts the most will reap the benefits. Small steps soon add up to big change, provided we can all see where we are going.
About the author: Dr Andrew Bastawrous is CEO and co-Founder of Peek Vision, and Associate Professor of International Eye Health at the London School of Hygiene & Tropical Medicine. He has worked and undertaken research in over twenty countries, including two years living in Kenya where he led a major eye disease study and the development and testing of Peek.