Medical Devices: Does Validation End?
Product Development: When Does Validation End?
A recent NY Times Editorial, 80,000 Deaths. 2 Million Injuries. It’s Time for a Reckoning on Medical Devices, presents three proposals aimed at improving medical device quality: Tighten approval standards; Fix post-market surveillance; and, Loosen industry’s grip.
Yes, “F.D.A. oversight is the gold standard the world over”, however, as a #seasoned medical devices professional, I believe our profession, both manufacturers and regulators, can step up to this challenge. It’s time we show each other and the world how to use an existing superpower that’s been too infrequently exercised, our decisions of validation.
Described simply, we know the typical phased medical device product life cycle starts with the definition of a product concept, proceeds with requirements and architecture, development and verification, and concludes with validation. Thereafter, a product is in the maintenance phase and a period of post-market surveillance.
I advocate our adopting a position of continuous validation — validation from cradle to grave — of our medical device products. Through continuous validation, we can resolve many of the remaining quality problems ailing our industry. Using the editorial’s proposals, I’ll explain how together we can make it can work.
Tightening Approval Standards
According to FDA Quality Systems Regulation (QSR) 21 CFR 820, “Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled”. A waterfall-based development tradition by manufacturers has inspired our verification then validation culture. Generating a mountain of verification results then coupling with a validation phase at the end of development results in an implied decision to validate by the manufacturer and, subsequently, by the FDA. Once a manufacturer validates a device and achieves FDA regulatory clearance or approval it can be made available for sale.
Embracing a protocol of continuous validation throughout the life-cycle inherently imposes a greater emphasis, i.e., tightening, of the already existing review-and-approve requirements of the QSR. If our industry adopts a consistent means to validate medical devices, perhaps using the MDIC’s Domains of Quality (DoQ) as an example, I contend it will yield products of greater quality while hindering products of inferior quality from entering the market. Employing the DoQ establishes a set of validation questions to be asked throughout the development of the device; asked of its main features, of its requirements, of its design, of its implementation, of its verification procedures, and, especially, of its defects discovered during development.
Employing the DoQ: A framework for consistent decision-making/validation:
Is Safety impacted?
Is Effectiveness impacted?
Is Reliability impacted?
Is Patient Satisfaction impacted?
Is Usability impacted?
Is Availability impacted?
Is Compatibility impacted?
An answer of Yes to one or more is an indication of increased risk and should trigger the question, “are we doing the right thing?”.
When done properly, continuous validation becomes a creative process of multiple participants who experience a state of flow and excitement much like an improv event.
In our medical devices culture, professionals at all levels should always be asking these questions. The questions serve to ask and inform us of all the known impacts and potential impacts of both expected behavior and potential side-effect behavior. By the end of development, if done properly, virtually every facet of a medical device’s behavior will have been uncovered, documented, and validated. Validation not only becomes a data set implying a device has been verified and validated, its documented decisions of validation objectively support the manufacturer’s final decision to validate the device. Validation becomes as the QSR definition intended, a confirmation, a subjective decision supported by continuously gathered objective evidence — data and decisions, that the requirements for a device designed for a specific intended use can be consistently fulfilled.
Again, it doesn’t take tightening of the approval standards, only a better method of practicing and executing a process of the already existing regulation. When our industry agrees on a consistent validation method, one embracing continuous validation, we as medical device professionals can practice an effective and transparent means to ensure better safety, efficacy, and overall quality; our industry earns the respect of the general public.
Fix Post-Market Surveillance
A requirement to perform post-market surveillance is also expressed by the QSR, an expectation of the manufacturer to track complaints. A “Complaint means any written, electronic, or oral communication that alleges deficiencies related to the identity, quality, durability, reliability, safety, effectiveness, or performance of a device after it is released for distribution”. Further, an “Appropriate statistical methodology shall be employed where necessary to detect recurring quality problems.” As such, a manufacturer must have a means for professionals to log and actively monitor product complaints in order to recognize impending quality problems and prevent them from escalating.
Building off the first recommendation, I propose we recast post-market surveillance as post-market validation, employing the same continuous validation process and criteria to both confirm expected behavior and confirm or deny defective behavior. Continuous validation gives us a qualitative assessment and quantitive measure of potential impact and harm, perhaps a DoQ-risk score. Defects occurring with frequency and/or having high DoQ-risk inherently become the ones that must be addressed; another decision of validation.
Fixing post-market surveillance by employing continuous post-market validation amounts to extending surveillance already established during development. Further, our industry is anticipating big-data, analytics, artificial intelligence, and the like to fuel Industry/Quality 4.0. In having a means to react to and track complaints employing DoQ-risk assessments enhances post-market validation and enables statistical models to more rapidly detect and recurring quality problems and, ideally, prevent unnecessary downstream effects to a greater population of patients and users. Tracking complaints can be done electronically, but it will require us medical device professionals, exercising appropriate validation judgment and process, to assess the complaints.
Loosen Industry’s (Manufacturers’) Grip
Both manufacturers and regulators have empathetic medical device professionals and we are united by similar cause: bringing innovative, life-changing, and timely products to market and making sure they are reasonably safe and efficacious; that is, valid and validated products. We have many good regulations, standards, and guidances already in place to help us effectively do our jobs. As manufacturers, we do our best to interpret and align our policies and procedures to ensure compliance and training to assure quality medical devices. As regulators, we are nonpartisan advocates for both manufacturers and the patients and users whose lives are impacted by medical devices. Working together, the risk of serious injury or death is not something we take lightly.
Up until now, our industry has predominantly measured quality via compliance (to regulations, standards, and guidelines), which essentially verifies we’re using a good practice appropriately and its artifacts can be measured quantitatively. Inspired by W. Edwards Deming, quality is measured via verification.
In applying a greater focus on validation, continuous validation, we gain the benefit from focusing on what Joseph Juran referred to as the vital few; the most critical 20% of criteria, e.g., MDIC’s DoQ, driving 80% of product quality. In measuring subjectively against a vital few criteria, quality is measured via validation.
Paradoxically, doing more validation results in the need to do less verification. Philip Crosby said, “quality is free”; I say “continuous validation provides a rebate” and enables saying “we’re done! It’s valid!” with confidence. With rapidly advancing innovation on many fronts is bringing more solutions and more players into the market, FDA budget and resourcing most likely won’t keep pace. Doing more with less becomes an ever-building dilemma. The way to greater efficiency is better-evaluation by way of better-validation. The primary, least-burdensome objective evidence in regulatory submissions is the product of continuous validation.
Loosening industry’s (manufacturers’) grip can be achieved through better alignment of validation expectations of medical device professionals on both sides of the manufacturer/regulator relationship. There are already plenty of eyes and minds employed in doing the thing right, verification and compliance. The same eyes and minds can be entrusted and empowered in doing more of the right thing, validation beyond quality.
Does validation end? Verification is finite. Validation is infinite.
Continuous validation is your superpower as medical device professional.
Gregory Keyes is a 35-year medical device professional and consultant focused on systems engineering, software engineering, and validation.Write to him: greg@conceptualintegrityconsulting.com. Connect with him: https://www.linkedin.com/in/gregorykeyescva/
