Testing Resiliency

Testing Elixir — by Andrea Leopardi, Jeffrey Matthias (35 / 80)

👈 Testing Singleton Resources | TOC | Wrapping Up 👉

Let’s start this section with a bold statement: we are not fans of testing process crashes. We’ll get philosophical. Why does a process crash? Most of the time, because of an unexpected error. In some cases, the best thing to do might be to raise an exception or exit from a process for an error condition that we know can happen (an expected error). However, we find that tends to be the exception rather than the rule, because if the error is expected then you likely want to handle it gracefully (think of TCP connections dropping or user input error). In cases where you’re raising or exiting on purpose, it might make sense to test that behavior.

Regardless of that, one of the most powerful features of the OTP architecture is that if a process bumps into an unexpected error and crashes, there will likely be a supervisor bringing it back up. That’s not behavior we want to test; supervisors work and have been tested with automated and on-the-field techniques for decades now. At the same time, we don’t really want to test that processes can crash. But if they crash because of an unexpected error, how do we test that if the error itself is unexpected? If you can test an unexpected error, we believe that error is by definition not unexpected. There’s your philosophy right there.