Continuous Integration in Product Design
What would happen if your documentation directly tracked design changes to your product?
Get Through The Gates
Traditional hardware development is planned with breaks in the action. A common method is called the “Phase-gate model”. In these ‘project gates’, hardware companies stop and evaluate if a project is ready to move to the next step in the product development process. This is especially important as a project moves towards New Product Introduction (NPI) — transitioning to manufacturing. A group of stakeholders sit in a conference room, review documentation and test results and planned development and make a decision whether to move forward. It can be a stressful, messy and very long meeting.
Why is it done this way?
One reason is the level of complexity of a hardware product. Evaluating each piece of the process is necessary because the cost of failure is high. In a multi-board system with mechanical elements and cables and many other things going on…each interconnected piece can cause an error in the final product. There is the idea that when moving things into production, a “freeze” on brand new features will reduce complexity, though it doesn’t always happen in practice.
Another reason is the large amount of documentation required in order to move the project forward. This may seem cumbersome but benefits future stages. The mechanical drawings, schematics, layouts, software packages, firmware packages, test suites and all of the initial customer documentation needs to be verified. In order to have everyone on the same page (sorry), the documentation needs to be approved at each individual level and then “rolled up” to the top level documentation. This can be a final Bill of Material (BOM) that lists all of the components and sub assembly documentation or it could be a large requirements document. Whatever document format that is used, the top level document is meant to tie together the project in a large hierarchical manner.
The final reason to wait until this “gate” meeting is because of the huge amount of uncertainty during early stages of the process. Sure, the individual departments are meeting and agreeing upon certain items throughout the process. But unless there is consistent updating of documentation, it’s possible for small changes to derail larger integration systems, which then also “roll up” to the higher level documents. Each of these issues can potentially delay the schedule, especially if they’re not found until the gate meeting (unlikely).
What if the gates were tiny?
Like in a mathematical limit problem, what happens if the time between one gate meeting and another starts to shrink? What happens if it shrinks to the point where gate meetings don’t happen every 6 months…but every month? Every week? Every day? Every hour?
What if the “gate meeting” happens every time someone makes a change to anything?
That sounds crazy, right?
This is really the concept behind Continuous Integration (CI). Every time there is a change in a CI software system, the build of the final software is kicked off. If that completes successfully, a battery of tests cases are run. If those all pass, the change didn’t break anything!
In software, like in hardware, the complexity of the project scales the difficulty of running continuous integration. Each new output case needs to be tested, as each individual component has a series of inputs and outputs that creates more possible outcomes when changed.
Software is different than hardware
I gleefully steal (repurpose) concepts from the software world and apply them to the hardware world. Two examples would be trying to do revision control and agile project management in hardware. But there always needs to be a conditional statement about how different hardware and software are. There will be practical differences in how things are done. And while “hardware CI” seems an interesting idea, I should point out it is very much still just an idea.
Let’s look at an example scenario.
I’m imagining a small electronics device enclosed in plastic. It has firmware programmed on it and let’s pretend it is connected via a bluetooth module to an app on a cell phone that is custom written. So for simplicity, let’s say there are four sets of design files:
- Mechanical design files, including final assembly drawings, packaging for the final product and design files for creating the test stand.
- Electronics design files, including the schematic, gerbers (board design files) and Bill of Materials (BOM) for the circuit board.
- Firmware files, including a bootloader image that needs to be burned to a ROM that gets soldered to the circuit board during PCB assembly and a final application firmware that gets programmed during integration testing.
- Software files, including test outputs.
- <BONUS> User documentation, including guides on how to reprogram the device with new firmware and how to use the app.
Already you’re seeing the layers and layers of documentation required. A seemingly simple IoT product has multiple interdependencies and potential pitfalls if there is insufficient communication between stakeholders/groups.
A change occurs
OK, so let’s go through some of the things that might need to happen as you change some of your design files and what should happen as a result. Unlike a pure software continuous integration system, there will be more “documentation” steps in order to keep the product team informed and working with the freshest information.
- The ME decides to change the test fixture— The EE needs to know if programming pins connecting in the same way, the firmware team needs to understand if the programming of the boards is the same and the software team needs to hear if the test output format is the same for their integrations with test data. Also the ME would need to regenerate the assembly/test instructions for the manufacturer.
- The EE adds a new capacitor the circuit board — This seemingly small change should trigger looking into whether the new part is available in the marketplace (perhaps quoting the BOM), adding the new supplier if they’re not in your supplier database (and checking their track record on lead times), updating the mechanical 3D model to include the new capacitor (checking component height) and informing the firmware engineers if anything has changed that impacts their firmware image (if the capacitor was in the filter for an ADC, perhaps).
- The firmware engineer changes an internal register function — The EE needs to know if this will affect hardware function (if it is controlling some part of the hardware), the software engineer needs an updated API or an explanation of how to access this new register and the ME may need to hear if the programming fixture needs to change.
- The software engineer changes the update access rate from the app — The EE needs to understand if this changes the energy usage of the device, (if so) the ME needs to understand how the heat output from the increased activity impacts the overall heat of the device, the firmware engineer needs to understand if the small processor can handle a higher access speed and run tests on their firmware images.
None of these necessarily represent reality. I just wanted to illustrate simple examples of how small changes could flow in both directions for product development.
It’s also important to note that as these changes happen later and later in the development process, the impact is much greater. More things are already set in the device and more things can be impacted by small changes. Relevant to that, notice that the changes listed above are tied mostly to design elements. When this becomes a “Design for Manufacturing” exercise, things get even more interesting. We will write about that in the future here.
How tiny is too tiny of a change?
If each small change kicks off a range of new documentation and notifications to involved team members, things could get very “noisy”, very quickly. In the EE example above, I imagine that a change such as adding a new part would warrant a trigger, whereas changing the silkscreen on a PCB would not. But each case would be different. Rules and tests could be used to determine when to really kick off a tiny gate review. These could be prototyped throughout the process as checklists.
More Coming Soon
While this is likely not the first time the concept of Continuous Integration has been applied to non-software, we are interested in the idea of iterating on documentation over time and keeping documentation in lock-step as a team updates their projects. This is also the subject of many PLM discussions, but we envision a different way of doing things than a complete top-to-bottom PLM implementation. We will try to show the differences in future articles.
We’d love to hear from you! Have you ever experienced a system like this? How did it work out? What were the mechanics? Please let us know in the comments below or write in to firstname.lastname@example.org
Thanks to Charles Aylward for contributing to this post