Benchmarking Cardiovascular Risk Models
Patients can now use three publicly available models online to assess their 10-year risk of incurring an atherosclerotic cardiovascular disease event (stroke, heart attack, etc.).
The three models are MESA, ASCVD Risk Estimator Plus (ASCVD Plus), and Framingham. The table below describes their respective input or variables they use to estimate your risk.
The last row in the table above describes their overall sensitivity level to the inputs. As described the MESA model is by far the least sensitive to the variables' inputs. We will uncover that shortly.
We are going to benchmark these models with each other based on the following factors:
- Age from 40 to 75 years old
- Total cholesterol from 180 to 260
- Systolic blood pressure from 120 to 160
We will sensitize each of the factors within the range specified above. And, we will test each factor independently. For instance, starting with age, we will only change the age factor, and keep all other inputs unchanged. Thus, we need default inputs that are described below
Age
Disclosure:
For the MESA model, I estimated the risk for 45-year-olds and 75-year-olds. And, I used an exponential function to fit the other age categories. This was to render the estimation across all ages reasonably efficient. I am confident the estimation is close to the values I would have obtained if I had rebuilt each scenario manually.
For the Framingham model, I estimated the risk for 75-year-olds even though this model was specified for up to 74-year-olds. Given that the estimate at 75 seemed reasonable, I used it.
The tables above use colors to differentiate the relative risk levels disclosed by the models. Green denotes relatively low risk. Red high risk. This color coding has no clinical significance. It only has a “relative” significance.
The tables above disclose the risk for healthy white men and women ranging from 45 to 75 years old. As shown the MESA model is so much less sensitive than the other two.
The MESA risk for a 75-year-old man at 7.2% is lower than the Framingham risk for a 55-year-old man (7.6%). The MESA risk for a 75-year-old woman is the same as the Framingham risk for a 50-year-old woman (3.5%).
The ASCVD Plus and Framingham models generate age-related risks that are much more convergent (vs the MESA model).
The graphs below visualize the tabular data. These graphs clearly disclose the divergence of the MESA model. It is so much less sensitive to age-related ASCVD risk.
Total Cholesterol
Disclosure:
For the MESA model for efficiency purpose, I used the same method as before. I calculated the risk for total cholesterol of 180 and 260. And, I estimated the in-between value using an exponential function.
The tables above denote the risk for otherwise healthy 65-year-olds (using default inputs for blood pressure metrics).
As shown above, the very low sensitivity of the MESA model makes it an outlier vs the other models. The MESA risk for individuals with total cholesterol of 260 is much lower than for individuals with total cholesterol of only 180 in the other models.
Notice that women, regardless of the model used, appear nearly immune to cholesterol levels. Indeed, their risk increases very little with cholesterol rising from 180 to 260. And, at 260, their risk is much lower than for men at only 180.
These results question the merit of using cholesterol-lowering drugs for women of any age who have not experienced any adverse symptoms.
Even though men’s risk appears more affected by cholesterol level, their respective increase in risk is not as material as expected.
The graphs below visualize the tabular data. Again, the MESA lack of sensitivity stands out vs the other two models.
Systolic Blood Pressure
Disclosure:
For MESA, I used the same method as before.
You can see the same pattern emerging as for the other factors. MESA’s estimated risk is invariably much lower than for the other two models. Again, women are much less affected by blood pressure than men.
The graphs below show a pronounced differentiation in sensitivity. The MESA model, on a relative basis, has almost no sensitivity. The Framingham model is by far the more sensitive. The ASCVD Plus model is in between the two (but closer to Framingham).
Considerations
- The MESA model's low sensitivity is questionable.
- The ASCVD risk for women is a lot lower than for men.
- The above questions the need for preventive therapy for otherwise healthy women with prescription drugs associated with side effects.
