Certifying a Medical Device as a Startup, Part 2: V-Model Doesn’t Mean Waterfall Development (IEC 62304)
How does your startup develop software? You’re probably thinking: Hm, our developers sit behind computer screens and write code. Simple, right?
Yes, unless you’re developing software which is a medical device. Then you need to put some thought into what you do during development — the process of how you develop software.
Let’s assume you’re a small startup building a dating app. How do you decide which features to build? Maybe a businessperson has an idea for a new feature and posts it on Slack. Now the developers should implement it. How will they know how it should look like? Will they have to improvise and interpret the statement made on Slack? And how will the businessperson follow up and check whether it matches her idea?
It gets trickier. Will that be documented somewhere? Where? In which structure? If someone visits your company one day and asks you “so what does this app exactly do?” you could demo the app but that would only paint half the picture. It doesn’t cover all the details, like which input is allowed, which errors are shown and which third-party APIs are used. These things are commonly referred to as specification. Do you have one?
Many startups don’t. For medical software, you need one.
Developing medical software means having a software development process which includes all requirements of IEC 62304:2006. As noted in the first part of our series, the IEC 62304 is unfortunately one of those standards which states the what but not the how.
And this is where it gets dangerous — imagine you bring in consultants to tell you how to do things and they say “yeah, you guys have to develop using the waterfall model from now on”! Never forget: Incompetent consultants can ruin your company. Especially if you’re a high-risk startup which has to ship a product.
Time to get IEC 62304-compliant then. What do you have to do?
The Impact of an IEC 62304 — Compliant Development Process
Let’s have a look at the IEC 62304:2006. In appendix B, table B.1, different development strategies are compared:
| Development | Reqs defined | Multiple | Distribute |
| Strategy | at start? | Dev Cycles? | test software? |
+--------------+--------------+---------------+-----------------+
| | | | |
| Waterfall | yes | no | no |
| Incremental | yes | yes | maybe |
| Evolutionary | no | yes | yes |IEC 62304:2006, appendix B, table B.1.
We appreciate the authors trying to give more practical advice in the appendix but unfortunately this still leaves many questions unanswered. What the hell is Evolutionary Development? It sounds like someone wanted to write Agile but wasn’t allowed to.
But luckily, we don’t have to understand the table. The main takeaway is that the authors don’t stipulate a specific development strategy. That’s great news!
Paraphrasing the sections surrounding the table, they state you can use whatever development strategy you want as long as your documentation is consistent with what you’ve produced and IEC 62304.
So, does this allow you to go on a nightly coding spree, developing whatever you want? Kind of. As long as you document it and adhere to your own development process which itself must comply with IEC 62304. You have to ensure that your documents are up to date and your software has been tested properly before releasing it (verification).
Great. We’re making progress. So what do you have to document?
Software Requirements
Features, allowed input, error messages, system requirements — all those details of your software need to be documented. These are called Software Requirements. Together, they form a Software Requirements Specification.
While this may seem unnecessary, it can be beneficial in the long run: Imagine a new developer joining your team. Instead of slowly onboarding her by letting her discover what your software does, you have your Software Requirements Specification which she can read in an afternoon.
There’s a trade-off here: In a perfect world, with one developer working on one project his entire life, this wouldn’t be necessary — all information would reside in his head. Once you develop software in a team, however, exchanging information becomes more critical and at some stage things need to be put in writing.
Implementations and Traceability
Once your software is done and all Software Requirements are documented, how do you ensure you actually implemented all of them? You need a way to link your Software Requirements to implementations. This is called traceability.
In practice, you could link each Software Requirement to a GitHub Pull Request which implements it.
Software Architecture
Based on your IEC 62304 software class (A, B or C; section 4.3 of the standard), you have to document your software architecture to varying degrees. Typically, this would include specifying the interfaces of your software but in the “worst case” of class C you’d have to document everything down to the methods of your classes.
Third-Party Libraries (SOUP)
If you’re developing software in the twenty-first century, you’re probably using third-party libraries. In the regulatory world, these are called SOUP (Software of Unknown Provenance). I don’t know why.
You have to list them, including their version number. Then, you have to document which requirements they shall fulfil and whether they actually do it.
This is arguably a lot of overhead but sometimes it makes sense to know why you chose a library two years ago.
Verification
All software needs to be tested. This doesn’t mean you need 100% test coverage — “testing” here is to be understood in a wider sense. We developers test software all the time, even without explicitly writing tests. Typically, we write some code, execute it and check whether its output matches our expectations.
The part of whether something matches our expectations is crucial. To be IEC 62304 — compliant, we need to document these things. First, which feature (Software Requirement) were we testing? Second, what did we expect? Third, what was the actual result?
How we do that is up to us — we could attach output of our automated test suite including unit and integration tests or screenshots of a QA person testing it manually. Also, traceability is important, again. We need to prove that all of our Software Requirements have been verified.
Waterfall Development?
So by now you may be thinking, “This really sounds like waterfall development”. What is the waterfall model?
It’s attributed to Winston W. Royce. He had great ideas regarding software development and published his seminal paper on the topic. Unfortunately, it was widely misunderstood and nowadays, the abomination of the Waterfall Model of Software Development is wrongly attributed to him.
When people nowadays hear “Waterfall Development”, they think of a group of people sitting in a dimly lit room, smoking cigars and drawing UML diagrams. When done, the group disperses and the diagrams are handed out to developers. Their task is simple: Implement everything according to spec. And the spec must not be changed — by anyone! Ever!
History has shown that this is not a good way to develop software. Surely, Mr. Royce didn’t have this in mind when writing his paper. Have a look at the second page:
Either the requirements must be modified, or a substantial change in the design is required. In effect the development process has returned to the origin and one can expect up to a 100-percent overrun in schedule and/or costs.
(Winston W. Royce, Managing the Development of Large Software Systems, 1970)
He went even further and proclaimed that a more iterative approach to software development was needed. But nobody bothered to read the second page of the paper. He didn’t even mention the term Waterfall Development!
After companies used the “wrong waterfall model” and noticed it was not viable, new ways of developing software emerged. Often, requirements had to be changed mid-development which led to problems — so why not embrace the fact that requirements change constantly? Henceforth, agile development was born.
The Right Development Model
Which brings us back to the question of which development model is the right model for IEC 62304 compliance.
There’s a subtle difference between IEC 62304 compliance and the waterfall model: While IEC 62304 focuses on what you should document, the waterfall model describes when each step should happen. This distinction is important.
And it’s great because it gives us the flexibility to use whatever development model we want as long as we document everything the IEC 62304 dictates.
Again, this doesn’t mean documentation as developers understand it (doc string in source code) but instead having written Software Requirements, Implementation, Software Architecture and Verification, always ensuring Traceability.
Let’s go through these steps with an example.
Example
Our imaginary software treats patients with Nightmare Disorder which means they have a lot of nightmares. It’s a mobile app which displays friendly dog pictures before patients go to sleep. This is a proven way to ensure sweet dreams, right?
Our Software Requirements might look like this:
- Display a full-screen random dog picture from a database.
- Run on iOS and Android.
We’d also document our SOUP. This will not be very fun because native iOS and Android apps often include tons of dependencies.
Being conscientious developers, we’ll use GitHub for version control and create Pull Requests for every new feature we implement. This lets us keep track of our implementations where we simply link each Pull Request to the Software Requirement it implements. Nothing prohibits us from having multiple pull requests implement one Software Requirement, of course — maybe it’s a very challenging task to display a full-screen dog picture (1.) and our developers have to create multiple Pull Requests to achieve it.
While doing that, we also document our Software Architecture. This depends on the classification of our app but could include documenting all our interfaces. Where does data enter our application? Maybe we’re retrieving our dog pictures from a dog picture API. What’s the syntactical format of our data? Maybe it’s JSON. Which keys do request and response have? We could create a document in which we describe those interfaces — or we could auto-generate it with tools like Swagger.
Next, we take care of verification. We have to prove that we correctly implemented all our (two) Software Requirements. How would we do that? If we’d want to keep it really simple, we could make human testers launch the app a few times and check whether a random dog picture is displayed each time. They could also use both iOS and Android for that and take care of verifying the second requirement, too! As we need some sort of proof for our documentation, we take screenshots.
And we’re done. You get the idea. A lot of this documentation may seem like unnecessary overhead. But when you come to think about it, a good team of software developers documents some of these things already. The main work is about ensuring that it’s complete and in one place.
Again, this doesn’t mean we have to develop using the waterfall model. It only specifies what we have to document. We could implement this by defining that our documentation must always be done after a sprint cycle, say, every two weeks or so. Or maybe only before each release. This should be defined in your software development process.
Nitty-Gritty Details
For simplicity, I’ve skipped a few things. You’ll notice that in regulatory work, the devil is in the details. So, for the sake of completeness, here are a few more things you have to think about:
- Write a Software Development Plan.
Write how you plan to develop your software. This includes listing the results of development but also how you plan to accomplish them and which tools you plan to use. - Write a Software Maintenance Plan.
Describe the long-term maintenance of your software. How long will it be in the market? Who is responsible for providing bug fixes and updates? Ideally, this could be part of your Software Development Plan. - Have a Software Release Process.
Before you release, you need to make sure that your documentation is complete and you have a maintenance plan in place. Also, you should archive your software so that you’ll be able to retrieve old versions at any time in the future. Like always, you need to document this: Who checked that everything was complete? When did you sign off the release?
Which Tool for Documentation?
Now you know what to document, but not where. The simplest way would be using a spreadsheet. While spreadsheets are accessible and simple, they aren’t perfect. The main problem is that they’re bad at modelling many-to-many relations. Imagine you have a Software Requirement which is implemented in multiple Pull Requests — and those Pull Requests also cover other Software Requirements. You’ll end up with spaghetti spreadsheet rows referencing each other, a big mess!
It gets worse: If you decide to remove a Software Requirement, you have to manually remove all references to it in other spreadsheets. For large software projects, this becomes nearly impossible. Instead, it makes a lot of sense to think about using a tool for this. Unfortunately, most of them are crappy as noted in our first article.
It’s Overhead, but it’s Safe
Wrapping up, documenting things in an IEC 62304-compliant way does create overhead. It’s our obligation to implement those requirements in a way which is practical and realistic for our companies. Many of those things make sense and lead to developing better software.
Well, at least that was the goal in the beginning. The more complex your software development process becomes, the slower you get. And slow development speed is one of the strongest inhibitors of innovation. Sometimes, in the evening, under the shower, I wonder: If everything would be less regulated, which innovations might we have today?
There’s always a balance to strike — and for medical software, our society has decided to heavily prioritize safety over innovation. And that’s a good thing. So let’s channel our efforts and share best practices on how to implement those regulations in a sane, technology-compatible way. Only then can we enable new companies to enter the market and help patients benefit from products which are truly innovative.
