Implementation of a QMS for Medical Software Development in an Agile Team
Introduction by Andrew Tremblay:
Before we talk about Quality Management Systems, or agile teams, or ISO standards, we should talk briefly about their history. We should talk about Therac-25.
Therac-25 was a radiation therapy machine built by AECL (a Canadian medical device manufacturer) in 1982. It was designed to use radiation to destroy harmful cells in cancer patients. Tragically, between 1985 and 1987 at least six accidents occurred with Therac-25, where patients were exposed to one hundred times the intended dose of radiation, resulting in severe burns and radiation poisoning.
After its recall, an audit of the Therac-25 showed that the main cause of the device failure could be traced to poor software design. AECL had reused software from previous medical device models, removing hardware failsafes in place of software ones. They neglected to independently review the newly written software and had no way to automate any of their tests.
While the events of Therac-25 were happening in Canada, a US congressional investigation on device safety was being launched. It found that from the years 1983 to 1989, 44 percent of all medical device recalls were due to faulty product design. In response to this (and other investigations), the Safe Medical Devices Act (SMDA) of 1990 was passed. Among many international restrictions, the Act required medical device manufacturers to have a Quality Management System (QMS) in order to be considered to follow Current Good Manufacturing Practices.
ISO 13485 came about as a result of the SMDA to define such a system. The FDA announced last spring that it would be adopting ISO 13485:2016 as its standard for evaluating QMSs. It is not only one of the primary requirements in the manufacture of medical devices for the USA and Canada, but for the European Union as well. It is the base test for the Medical Devices Single Audit Program (MDSAP), currently operated by Australia, Brazil, Canada, Japan and the USA.
Many of those mistakes of Therac-25 now seem impossible, but these risks are no less present now than ever as our software constantly integrates into our every moment. It is to mitigate these risks that a QMS is used, and it’s one of the reasons why we implemented one.
Quality Management System
At Rightpoint, one of our missions is to improve lives through technology. Medical applications are increasingly able to impact the lives of patients and help them onto a path of better health.
While we know we have the expertise to design and develop medical applications, a large part of building software in the medical device domain is to ensure the quality and safety of the software. This is accomplished by using a QMS which is an integral part of organizing and controlling our work to ensure that the products we build are safe and effective while consistently meeting customer expectations.
Adhering to ISO 13485:2016 standard helps ensure that we’re building safe software and that we’ve taken the precautions needed to reduce the likelihood of risk to the patient using these applications. This certification enhances our abilities to build an entire medical product for our clients and eases the regulatory pathway for them by being able to show that the software was built via processes that adhere to internationally accepted quality standards.
Preparing for the Certification
To determine the level of effort that would be required to obtain the certification, we first performed a Gap Analysis of our current processes and documentation. This Gap Analysis takes us item by item through the ISO 13485:2016 standard and identifies what components of the standard we already meet, what we have documentation for, and where there are weaknesses in our process. In Rightpoint’s specific case, many of the ISO 13485:2016 requirements were already being met from a process perspective, but we lacked the formal documentation and process controls required to claim adherence to this standard.
Building the Team
With the gaps in our processes defined, we now needed to identify those people in our organization would be vital to the operation of our QMS. These key players would need to help us not only build the QMS and obtain certification, but also remain available to help us continually improve the system. They would also need to go through various levels of training to show their competency with the ISO 13485:2016 standard as well as our own processes. We chose to have representatives across the various teams within Rightpoint, who are all vital to the development of a medical software product, including:
- Software Engineering
- UX/UI Design
- Product / Project Management
- Quality Assurance
- Operations
- Business Development
- Executive Leadership
This core team would be vital in participating in training sessions, reviewing process documentation, and participating in prototype development of medical device software under our new QMS.
Agile Approach to Risk
Due to the Agile nature of our development process, we took an Agile approach to risk analysis during the development process. Typically, we use the concept of “Stories” within a backlog or sprint to plan and execute on work items, each with acceptance criteria. Before any story can be considered “Done” — several new steps to our process needed to occur. These include:
- Failure Mode Effects Analysis (FMEA) Documentation
- Formal Test Protocol Authoring
- Test Protocol Execution and Documentation
The FMEA Documentation for each story includes a probability of failure occurrence, severity of failure occurrence and an overall Impact rating based on the probability and severity. Additionally, each risk must include our approach to mitigation of the risk as well as the description of the contingency of the risk.
Documentation
We approached the development of the QMS with this mindset: “We have a process that works — we have proven over many years that we are capable of producing high quality software. Let’s formalize the process and fill any gaps called out by the ISO 13485:2016 standard.” With this approach, we were able to maintain the way our product teams are used to working while adding minimal overhead to achieve compliance with the ISO standard. We still adhere to an Agile framework for product development, while meeting the requirements of ISO 13485 — AAMI provides guidance in their approach to agile software development in their TIR45 Document (Guidance on the use of AGILE practices in the development of medical device software).
While many of the processes for our QMS were already in place, there were gaps that needed to be filled. Among the major items that needed to be addressed were a process for Risk Analysis, Design History File, and Device Master Record (DMR) procedures and several other medically-focused procedures that we would otherwise not have focused on without a QMS.
Approach to Document Control
A key aspect of any QMS is Document Control — this includes tracking the version history of documents, control of who can view and edit documents as well as a review process for changes to process documentation.
Document control is not a foreign concept to Software Engineers, who comprise a significant portion of the team members at Rightpoint. Most engineers are familiar with using systems such as Git to manage the version history as well as review processes for changes to source code. Our question was “why can’t we use the tools we’re already accustomed to for document control of our QMS”?
It turns out, given the correct document format for our QMS, we could absolutely leverage the same tools software developers are used to using. Using tools like Github, document authors could create rich sets of documentation, and they would file “Pull Requests” — just like they would with changes to software components — to get approval on documentation changes in the system. Additionally, we can leverage GitHub’s powerful access control to determine who has rights to author and modify documents. We can also enforce rules, such as required reviews on pull requests, before a documentation change becomes final.
The Audit
At the stage when we had a good foundation of documentation written as the basis of our QMS, we scheduled our certification audits with an accredited ISO 13485:2016 Registrar (in our case, NQA). This is a two-part process where we have a Stage 1 audit which covers our documentation and processes. Later, we go through a Stage 2 audit which will look at records of use of our QMS and how it has been leveraged.
Stage 1 Audit
As a result of our Stage 1 Audit there were a few minor findings including missing procedures for Verification and Validation (V&V) of software as well as the absence of any records demonstrating effectiveness of the QMS — this was due to the fact we had not yet kicked off a project that would be built under these new procedures.
As a result of the Stage 1 audit we updated several of our procedures and thoroughly documented our V&V strategy. Some of these updates were performed as CAPAs (Corrective Action Preventive Action) to the QMS — showing that we adhered to our agreed upon process for making procedural improvements to the QMS.
POC Project
It was now time to put our QMS into practice, even though it was not yet ISO 13485:2016 Certified. We used a real Early Stage development project for one of our existing clients as first project within the new QMS — while also maintaining compatibility with our client’s QMS, since they were still on the hook for their own design controls. This gave us the experience of leveraging our own QMS to manage the lifecycle of the application components we were building and led to several more findings that could be used in the Stage 2 Audit.
Stage 2 Audit
NQA returned to perform a 3-day on-site audit where all our documented processes as well as records of use of our QMS were explored by the auditor. Many of the Rightpoint staff members who trained to use our QMS and the development of our first QMS based project were interviewed on their relevant areas of expertise relative to the QMS.
Certification
After three days with the auditor he was able to file his final report with the registrar: we had no major findings that needed to be immediately fixed before a recommendation for certification could be made. The result was a recommendation for Rightpoint to be certified to the ISO 13495:2016 standard.
Next Steps
Part of the focus of ISO 13485 is the concept of Continual Improvement — that over time, as we exercise our QMS and build medical products for our clients, we will continue to revise and improve our processes. We are now using our QMS to deliver high quality software components to our clients who are working on products that are Software as Medical Devices. We will be audited annually by our ISO 13485 Registrar to ensure that we are meeting all the required standards in the ISO standard and that our system is constantly improving.
Sources:
https://en.wikipedia.org/wiki/Therac-25
https://archive.org/details/medicaldevicesaf00unit/page/2
https://www.reginfo.gov/public/do/eAgendaViewRule?pubId=201804&RIN=0910-AH99
