The science of assessing the benefits of new medicines
The fate of potential new treatments hangs on the ability to test their effectiveness accurately and reliably in clinical trials. In this blog, we explore the tools that are used to measure the benefits of new therapies and how they are developed.
--
The main way that the benefits of new Parkinson’s treatments are measured is by using assessments to rate the presence and severity of key symptoms, and their impact on everyday activities and health-related quality of life.
On the face of it, these assessments — which range from questionnaires to completing tasks — may seem simple. But developing tools that truly capture change and can tell us if and how a treatment works is actually a science in itself.
Outcome assessments are crucial to developing and testing new treatments. And, because their purpose is to determine whether or not a treatment provides a benefit to patients, they can ultimately be the difference between a new treatment being made available or not.
In a complex condition like Parkinson’s, which affects people in such different and individual ways, ‘benefits’ can be difficult to understand and quantify. This makes choosing the right outcome assessment extremely important and tricky to get right. And it is getting even trickier as we attempt to measure the impact of novel therapies that aim to slow the progression of Parkinson’s — a new field of research.
The different types of outcome measures
There are four main types of clinical outcome assessments that can be used to assess the benefits of treatments in clinical trials, which are briefly summarised in the table below.
In most clinical trials, a variety of different tools will be used to look at a range of different aspects of the condition to help fully understand if and how a treatment is working.
But while many different assessments will be used, when a trial is planned, the researchers usually must choose one single assessment as their ‘primary outcome measure’. This should be the one that the researchers believe will most validly and reliably capture the expected benefits of the treatment. They must also work out a target for how much improvement needs to be achieved to provide convincing statistical evidence that the treatment ‘works’, and ‘works’ well enough to outweigh the potential risks.
This target — known as the ‘primary endpoint’ — is used to determine the success or failure of the trial.
All trials must set a primary endpoint before they start and stick to it. This is so that researchers can’t just use every measure and then at the end of the trial look through all the results to find the most impressive outcome and use that to suggest the treatment works.
While the primary endpoint is crucial, the other assessments performed during a trial (secondary endpoints) are vital too as they help to build a fuller picture of the effects of the treatment. The additional benefits or risks they reveal are important for providing information about treatments to doctors and patients — such as in the leaflet that comes with the medicine.
Which assessment is best?
Historically, clinician-reported outcomes have been widely used to assess the benefits of treatments — and are most commonly-used as primary endpoints.
But times are changing and there is an increasing focus on the importance of patient-reported outcomes which, if done right, can provide more relevant and meaningful evidence of the impact of treatments on how patients feel and function. Patient-reported outcomes might be used as primary endpoints, or they can be used as other important endpoints to provide more information than what might come from clinician-reported outcomes alone.
This move towards patients being front and centre in drug development is not just coming from patients themselves but also from the regulatory authorities — including the Food and Drug Administration in the US and the European Medicines Agency.
Regulators are the organisations that are responsible for assessing the evidence from trials and they ultimately make the decision as to whether to approve new medicines and what information can be communicated about the medicines. So, their endorsement of the importance of patient-reported outcomes is a powerful incentive.
Clearly, there are many advantages to using patient-reported outcomes to assess the effects of new treatments but just because an assessment is patient-reported doesn’t automatically mean it’s good… Many patient-reported outcomes are flawed either in that they don’t ask about the right things, ask about them in the wrong way, or are used in the wrong people. This is why it is so important to make sure we “get it right” so that the patient reported outcome assessments we use are valid, reliable, and crucially, important to those patients with the condition under study.
Developing the right tools
Over the years many different tools have been developed to assess different aspects of Parkinson’s in clinical trials — but it’s important to recognise that no tool is perfect, each one has its strengths and weaknesses and will be appropriate to use in some studies and not in others depending on the patient population being studied, the treatment being investigated, and its potential effects.
This means selecting the right tools to use is absolutely critical but too often outcome measures are chosen from the existing toolbox that are not quite right for the job at hand.
A new treatment may take years to painstakingly develop in the lab and will have to go through a huge amount of rigorous and detailed testing before it eventually is ready to be given to patients in clinical trials.
We need to take the same rigorous scientific approach that we apply to developing the treatment to developing the tools we’ll use to assess it in clinical trials.
So, how can we make the right calls when it comes to outcome assessments?
Step 1: find out what’s important to patients
The first step to using the right patient-reported outcome tool is to get a deep and detailed understanding of what is important to your specific study population.
You can do this by reviewing the literature and speaking to experts in the area, but the really crucial part is conducting in-depth interviews and/or focus groups with people affected to fully explore:
- how the condition affects their lives
- what bothers them most
- what treatment benefits they’d most like to see
Being really focused on the specific patient population is key. If the planned trial will test a treatment for dyskinesia, you need to involve people living with dyskinesia. If it will address hallucinations, you need to speak to people experiencing them.
Step 2: ask about the right things…
Once you have built a holistic view of what the patients experience and what matters most to them, you’re ready to start thinking about what aspects of the condition you need to measure.
There may be an existing assessment tool that fits the bill or that could be modified, or you may need to create a brand new assessment to fully capture the information you need.
Step 3: …in the right way
Once you have your patient-reported outcome assessment (whether it’s an existing tool or something new) you need to check that the questions will make sense to your study participants.
So, it’s really important to do some follow-up research, known as ‘cognitive debriefing’, to see how people interpret the questions — do they understand the meaning as you intended it? Is the language correct? Are the response options appropriate?
This is a really important step to fine-tune a new tool (or tweak an existing measure) to make sure it really will provide the information you need in your study.
Step 4: time to test your tool
Finally, before your tool can be used as a trusted outcome assessment in clinical trials you need to road test it. This means using your new assessment measure in another study with the same target population first — this might be a clinical trial or an observational study — so that you can see how it behaves in real research conditions.
You can analyse the data and do statistics to see if your tool is capturing the aspects of the condition you hoped. It’s also really important to compare how your new tool behaves alongside other more established assessment tools (if they exist!), do the scores make sense and behave as you expected them to? Or are there any surprises that we need to fix?
Putting the theory into practice
This may sound like a lot of work, and it is — but it’s worth it to have tools that we can trust really do tell us whether a treatment works or not. With the wrong tool we could fail to show a benefit with a product that really works, we could have “positive” results but not be able to understand what they mean, or we could inadvertently show a “benefit” when the drug doesn’t actually do anything good for the population!
We need to know that the ‘benefits’ we’re detecting really are true, relevant, and meaningful to patients. And we need to be able to translate these benefits into numbers and statistics that provide the evidence that regulators need.
Major pharmaceutical company UCB is working with people with Parkinson’s and two patient organisations — Parkinson’s UK and Parkinson’s Foundation in the US — to apply this rigorous approach to develop the right tools to assess outcomes in their upcoming trials.
Thomas Morel, Director of Global Patient-Centred Outcomes Research & Policy at UCB and leader of this ambitious collaboration explains why they decided to work in partnership with people living with Parkinson’s:
“There’s now much more recognition that patients need to be at the centre of how we measure the benefits of new treatments. Getting this right means that patients themselves have a crucial role in the process of creating and honing these tools.
“We’re really lucky to be working closely with 6 fantastic patient advisers in the development of the outcome measures in this project. They’ve been absolutely central in shaping the project from the start and are involved at every stage. We’re delighted with how things are going so far and I would strongly encourage other companies to include patient advisers in their teams that are developing clinical trials programmes.”
Special thanks to Thomas Morel from UCB and Ashley Slagle from Aspen Consulting for their expert help and guidance in developing this blog.
Thomas Morel works at UCB, a patient-centric global biopharmaceutical company committed to Parkinson’s research, where he leads the design of patient-focused outcome measurement strategies.
Ashley F. Slagle, MS, PhD, is a scientific and regulatory consultant providing advice on patient focused drug development and focuses largely on patient-centered outcome measurement. She previously served at the US Food and Drug Administration (FDA) and now shares her passion and expertise in putting patients front and center in drug development through her consulting work at Aspen Consulting, LLC.
