Hi Dustin, fortunately, in the highly complex systems I’ve worked on most of the risk is mitigated by running elaborate simulations. This means we have to create, maintain, and thoroughly vet a simulator. This is the Alamo for quality. Over time, all the knowledge we gather about a real life scenarios go in to the simulator. Even this is not an acceptable replacement for real life testing. In rail, there are actually real track circuits set up to test control software. And after all that happens, we hope nobody dies due to some bug that managed to evade all the testing scenarios and apparatus.
Complimentarily, unit tests, which I call specs, serve different purposes I think. Firstly, when using TDD, they help us build the initial component or unit. A starting point where we can define inputs and expected outputs. Secondly, I think a good spec will thoroughly document a component, it’s purpose and functional specifications, even tolerances. Thirdly, specs are a first line of defense against bug introduction. A tripwire. It gets scary, though, when someone breaks a component, gets failing test(s), then cripples those tests. We’ve all seen that horrible scenario before. Specs are not good at vetting elaborate, real life scenarios. A bunch of passing specs does not a safe, well functioning system make, hence the use of simulators.
From what you have said, I think you work on a large, complex system too. I’m curious what testing strategies you use. Full disclosure, I’m obsessed with testing :).