Time for The FDA to In-Validate/ Modernize Its Thinking on Validation?
On Monday, The FDA announced their plan to make significant changes to the 510(k) process, which is the most common pathway to approval for medical devices in the U.S. market. Coincidently, the announcement followed several recent articles in the general press questioning the FDA’s approval history, including an AP article scrutinizing the approval of devices that later incurred dubious safety records.
The FDA and its announced changes are further being criticized by the public in light of the published findings.
Manufacturers are open to FDA process improvements which enable greater safety and efficacy of their devices, but are wary of implications of proposed changes regarding comparison to predicate devices, the means for a new/updated device to indicate substantial equivalence to a previously marketed device.
According to 21 CFR 820, The Quality System Regulation (QSR), validation means “confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled”. (Note that consistently fulfilled leaves the door open for an occasional hiccup; valid does not mean perfect.)
According to FDA guidance Evaluating Substantial Equivalence in Premarket Notifications, substantial equivalence means that the intended use of the new/updated device is the same as the predicate and that “the principles of safety and effectiveness underlie the substantial equivalence determination in every 510(k) review”.
While there was an emphasis on modern and modernization (mentioned 32 times in the PR), the FDA has apparently not provided any hints as to their plans to update/modernize definitions of validation or substantial equivalence.
I suggest there’s modernization opportunity here as well. It is time for the FDA and the medical devices industry to modernize thinking on validation; that is, in-validate thinking on validation. The modern pathway, as suggested, continues with a narrow emphasis on safety and effectiveness (mentioned 15-times in the PR).
The need for validation is not new, but the industry needs to think differently about it. 21 CFR 820 states manufacturers must validate designs and validate design changes. Further, it states that validation must be done prior to implementation of the design/design change.
You may be asking, “Wait, isn’t validation done at the end as a confirmation?How can I confirm the change before I make the change?”
The terms verification and validation, despite being part of the common development vernacular, continue to befuddle some of the best engineers and quality professionals. The simplest way to understand them, according to Barry Boehm, is verification means “doing the thing right” and validation means “doing the right thing”.
Verification, doing the thing right, is a compliance activity, checking off a box that says, “yeah, it worked right”. Validation, doing the right thing, is a decision activity that says, “yeah, we did the right thing”. According to Peter Drucker “Management is doing things right. Leadership is doing the right things.”
By implication, managers are responsible for verification, ensuring compliance. Leaders are responsible for validation, making decisions.
In establishing the Case for Quality (CfQ), FDA has suggested there’s been too much focus on compliance and not enough on quality. Manufacturers are producing products that are “compliant” but still suffer from quality issues.
Working in collaboration with the FDA on the CfQ, The Medical Device Innovation Consortium (MDIC), has identified the domains of quality (DoQ) as follows:
Safety: device does not compromise the clinical condition or the safety of patients, or the safety and health of users
Effectiveness: device produces the effect intended by the manufacturer relative to the medical condition(s)
Reliability: device system or component is able to function under stated conditions for a specified period of time
Patient Satisfaction: device is perceived to meet or exceed patient expectations of usability and outcome
Usability: device minimizes the risk of user errors by patients or clinicians
Availability: device is available to fill first request orders
Compatibility: device is compatible with related devices or drugs, the use environment or relevant standards
I suggest the complete DoQ — adding to safety and effectiveness — provides the basis and a framework of consistent decision-making & validation whereby every validation decision and substantial equivalence assessment employs the following asks:
Is Safety potentially impacted by this change?
Is Effectiveness potentially impacted by this change?
Is Reliability potentially impacted by this change?
Is Patient Satisfaction potentially impacted by this change?
Is Usability potentially impacted by this change?
Is Availability potentially impacted by this change?
Is Compatibility potentially impacted by this change?
An answer of “Yes” to more than one of these is an indication of increased risk and should trigger the questions, “Are we doing the right thing? Is there a better alternative? Is doing nothing (for now) acceptable?”
When making multiple changes, the total qualitative impact provides an objective risk assessment whereby the overall decision to validate or invalidate can be made.
In the years 2009–2012, while at Ortho Clinical Diagnostics (OCD), then a Johnson & Johnson company, our devices were suffering numerous software defects, some resulting in field actions and recalls. While analyzing the root cause, I discovered many of the defects resulted from inadequate software validation, validation that did not occur prior to the design/code changes being made.
Our compliant software development and V&V processes were being followed and documented as prescribed, but software defects were still occurring. We were an example of case for quality.
After I instituted a software validation practice similar to the above, using OCD’s internal equivalent of domains of quality, our software quality improved markedly.
A minimal change to a manufacturer’s process, one that’s already a requirement of the QSR (validation prior to change), can have a significant impact on product quality. One of my colleagues told me, years later, “We hated you when you gave us more work to do, but today I’m telling you it was the right thing to do.”
I advocate the above simple, fast, cost-effective means to increase quality, quality beyond compliance. The process easily fits within the existing product development process — start, middle, and end. It achieves consistent and continuous validation.
Until next time, Be valid! Be an in-validator!
