Decoupling With Benefits
Different Times, Different Priorities
At first, shipping fast was the top priority. In our early days, it took us just a few months to go from a pen and paper insurance application form to a more modern online questionnaire boasting several features: validation, the capacity to show/hide questions, on-the-fly save, electronic signature, etc. As a result, hundreds of financial advisors were suddenly able to sell insurance products using our online questionnaire with a significantly improved customer experience.
Nowadays, the top priority has changed. Extensibility and maintainability of our platform is what we care about. Concretely, it should be easy to add or extend current functionalities. For instance, we regularly add a new type of question in the questionnaire, or we might want to make it possible to answer the questionnaire using another navigation system such as a wizard.
As the company grows, the importance of software architecture increases significantly. And a great architecture is one which makes decoupling a top priority. Robert C. Martin says it well in his book Clean Architecture:
The architecture of a software system is the shape given to that system by those who build it. The form of that shape is in the division of that system into components, the arrangement of those components, and the ways in which those components communicate with each other.
The purpose of that shape is to facilitate the development, deployment, operation, and maintenance of the software system contained within it.
The strategy behind that facilitation is to leave as many options open as possible, for as long as possible.
So here’s the story of a major transformation of our online questionnaire. How it went from a small monolith dealing with several operations to decoupled software layers with their respective responsibilities. We’ll tell you why this is relevant and what the benefits are.
One Big Questionnaire to Answer Them All
As we briefly explained in a previous blog post, we use a configuration-based approach. This means that we have a “questionnaire configuration” defining hundreds of possible questions in a given questionnaire. Each question has its own validation and visibility rules. Here is a simplified question coming from this “questionnaire configuration”:
At first we had some React component (empowering Formik) which would take the whole questionnaire configuration as a prop and render all the relevant questions. I say “relevant” because many questions are shown to the user only if they meet certain visibleIf conditions (you have 2 examples of such conditions in the code snippet above). It would also validate the answers and display error messages if needed. All this seemed to work fine.
Unfortunately, we had many sneaky bugs. For instance, some questions would not be shown when they should be, or some answers from hidden questions would not be removed from the component’s state. So, we fixed those bugs, but some more crept up. We got tired of it and we decided to rethink our approach. Of course, we could have added a ton of tests using react-testing-library, but it would have taken a lot of time (the total number of combinations of visible/hidden questions is somewhere in the millions) and it would have been hard to maintain. We needed a better plan.
When You’re Too Smart, You’re Buggy
We decided to extract all the business logic dealing with visibility rules and validation rules out of the React components entirely.
Before the refactoring, our React component responsible for rendering the questionnaire had a few responsibilities:
- Determine the visible questions, based on the visibility conditions and current answers.
- Process the answers. In other words, change some answers based on other answers in the questionnaire.
- Validate all answers. And display error messages where applicable.
- Render the appropriate questions with their answers.
After the refactoring, our React component responsible for rendering the questionnaire ended up having a single responsibility, which is the last one listed above. It now solely renders the appropriate questions, based on some information provided as an input prop. We could call that a “UI description”.
Note that a “UI description” is deprived of both visibility rules and validation rules. Those details have been sorted out before and the component is now effectively dumb. It can simply read the UI description and dedicate itself to rendering stuff on the screen.
Naturally, when the user adds or changes an answer in the questionnaire, we still need to evaluate the visibility rules and the validation rules. Before the next render(), a separate module evaluates the visibility and validation rules (as defined in the “questionnaire configuration”) and ultimately generates the questionnaire’s UI description. Here is a high-level perspective of the render() function:
Benefits of Extracting Business Logic from the View
Probably the most pragmatic benefit for us is fewer bugs. This is made possible because now we have the capacity to test all our business rules, independently of any UI concern. We can provide a set of answers, and make sure the generated UI description is what we expect.
Conversely, we can test all of our UI components which are now agnostic to our business rules. We have a UI component for each type of question, e.g. textfield, radio button, textarea, etc. And we have tests (using react-testing-library) for each of our UI components. Those tests exist to make sure each UI component will:
- behave properly when provided with a UI description (e.g. display validation errors when present or be selected/checked when they need to)
- propagate the onChange callback
- fire a tracking event when the focus is lost
Another great benefit of this separation of concerns is improved readability. Back in 1999, Martin Fowler eloquently wrote something which is now famous:
Any fool can write code that a computer can understand. Good programmers write code that humans can understand.
After our refactoring, it is fair to say that the next developer reading the code will grasp things more rapidly, because there are fewer things to consider at once. Smaller units of code with fewer responsibilities means a smaller cognitive effort to interact with the codebase.
Coming full circle with the beginning of this adventure, it also makes for more maintainable code. It is now easier to find the origin of bugs. If something fishy comes up, it is easy to determine if the issue lies with the rendering or with the business logic. Once we’ve determined that, we could say there is now about half the amount of code to consider in order to pinpoint the source of the bug. And if the list of benefits was not convincing enough for your taste, we will mention that it is easier to profile and improve performance.
Leaving Formik Behind
There are a few libraries out there when it comes to dealing with HTML forms. At first we decided to go with Formik (see one of our previous posts). It did the job, following a simple MVVM pattern (Model / View / ViewModel ) — providing data binding (between the form’s state and the form’s field values) as well as helping with form validation. The following diagram depicts how Formik incarnates the MVVP pattern.
As our codebase evolved, Formik went from a tool to an obstacle. From the get-go, Formik has been conceived to encapsulate the form state. And it does an excellent job at it, for simple or moderately simple forms. However, with more complex forms comes additional requirements, like hiding fields or changing arbitrary field values based on some business logic. In order to address those kinds of requirements, many people proposed a solution in order to have an onChange() at the form level, a sort of form state reducer (suggestions here, here and here). But from what we understand from the project maintainers, it is not in Formik’s DNA. Bottom line — and as was mentioned during a colleague’s presentation at Chain React 2019 — Formik is not meant to be used with complex forms relying on much business logic determining what form elements to render.
Additionally, with Formik it is not possible to have 2 field values bound to the same value in your model. For us that was not an uncommon use case given we show or hide whole questionnaire sections depending on some answers, and those sections might have fields in common. The reality is that Formik’s field values are not the same as your application model. They are essentially a set of key/value pairs tied to an HTML form.
For all those reasons, it meant that Formik was in the way, preventing us from building adequate software. We wanted to have more control over the state and deal with our extensive business logic differently. Note that we decided to keep Yup, the simple and awesome validation library. That’s a good example of a software library which does one single thing, and it does it reliantly.
From MVVM to MVP
While contemplating the issues we were facing due to tight coupling, we formulated new objectives. As you now know, we wanted to have:
- A module capable of generating a specific set of questions from the questionnaire, based on business logic and current answers
- A view entirely deprived of business logic and dedicated to rendering UI components
Leaving Formik behind and implementing our own solution meant that we moved away from the MVVM paradigm to an MVP paradigm.
Doing this allowed us to decouple the view concerns (the rendering, the UI) from the business logic concerns (the questionnaire’s validation and visibility rules). The presenter layer is clearly the smartest layer among the three layers. It knows about the model (the questionnaire and the answers), but it also knows about the business logic (located in the questionnaire configuration). With this knowledge, it is capable of determining the adequate set of questions and generating the UI description. In turn, the view follows this UI description and renders the necessary UI components.
We described this flow in different terms in the course of this article, but the MVP paradigm is probably the best conceptual representation to describe how all the relevant units involved interact with each other. It shows where responsibilities lie.
Parting Words
Decoupling might be the most important thing in software design. Having some code dealing with a small set of concerns translates into many benefits, like the capacity to easily extend existing product features and quickly locate bug sources.
With hindsight, one might wonder why we did not follow those basic design principles in the first place. The answer involves the lack of familiarity with the problem domain back then and the always evolving nature of product requirements.
We could also invoke Kent Beck’s 3 steps in building software (the quotes are from Kent Beck, the remarks following the quotes are from Robert C. Martin):
1. “First make it work.” You are out of business if it doesn’t work.
2. “Then make it right.” Refactor the code so that you and others can understand it and evolve it as needs change or are better understood.
3. “Then make it fast.” Refactor the code for “needed” performance.
That list sums up our adventures to date. First, we successfully delivered a working prototype. Only after that did we begin the refactoring to decouple the view logic from the business logic. Doing so allowed us to test each of those concerns separately, leading to more readable, more reliable, easily testable code. Clearly producing a win for everyone!
That’s all for today, thanks for reading! If you have any question or comment, don’t hesitate to reach out to us at hello@breathelife.com.
