Treating Technical Debt
How We Handle Forms at Doctolib Then and Now
As your codebase grows larger and you begin to accumulate legacy code, it naturally becomes more and more difficult to work with. This concept is called technical debt, and the unfortunate outcome is that it requires extra work from the developer to implement features because the code is tough to understand and difficult to expand.
Racking up technical debt is unavoidable, yet treatable. The following article explains how we unified the process by which we build forms in an attempt to reduce our technical debt at Doctolib.
Looking back on our legacy
Over the years, we have gone through multiple ways of building forms either by using external libraries or developing internal tools to do so. For example, we had an abstraction on top of formsy-react which we eventually dropped in favour of a custom implementation using RxJS.
The observable hell
Most of our forms were making heavy use of RxJS observables (one for each field) and being injected into React’s context using Recompact. More fields on a form means more observables, which tend to make the form declaration cumbersome and difficult to maintain. For example, one of our most complex forms is one that manages an appointment; it became increasingly more difficult to maintain this form with more than 30 observables provided to manage fields:
There are multiple issues with the above code:
- It became difficult to browse because the observables provided through the context reduce the links between components
- It was duplicated a lot, especially between the web and mobile parts, making it sometimes difficult to debug the code
- It was challenging to read, mainly because we over-engineered it by over-composing it using Recompact
State of the art: Forms using RxJS
At this point, we developed an API in an attempt to reduce this observable hell. The idea was to contain the use of RxJS in a few helper functions. The base API was composed by a set of two Recompact HOCs:
It was pretty straightforward in the beginning; we needed only to call connectField on our field and use the value and onChange props:
This implementation did provide some clarity, however, it still had a few issues including: a dependency between the fields, problematic errors and validation management, and always having to give the value and onChange props to our display components which should have managed it automatically. All of these issues ended up resulting in the production of boilerplate code that we could have avoided.
Because of those trade-offs, we decided to move to a more maintainable and easy to read implementation.
Introducing react-form
We could have chosen to improve our own API, but instead of creating an inhouse set of helpers, we decided to switch to react-form in order to unify our codebase.
React-form is a cool tool-set used to build forms with React:
- It provides some basic input components (like Text, Select, Checkbox…)
- It gives a
formApiwhich is used to manage the state of our form - Its
fieldApilets us create custom inhouse inputs - It is bundled with a validation API for errors state management
Building custom components on top of react-form
However, we were not completely satisfied with react-form’s basic APIs. Being heavy users of the great Rails Simple Form library and admirers of its simplicity, we built our components on top of the react-form API based on its principles in order to obtain the best developer experience possible.
To do so, we first wrapped the Form component into a custom one in order to avoid boilerplate code. As it turns out, we seldom use the formApi, except when passing submitForm to our <form> tag, so we wrapped it up:
Next, we had to migrate all our custom inputs in order to use fieldApi. Fortunately for us, the third version of react-form introduced a wrapper component Field which does almost all of the work for us. It gives us the API we need to manage the state of input regarding the form. Here is a simple example of the code of our Selector container:
We had some issues regarding the fieldApi.value that was not up to date in some cases. If your component is not re-rendered when the value mutates (e.g.: if you use Cleave.js), the fieldApi won’t be either. This means none of its data (value, error, warning, success, touched and fieldName) gets updated. Our current workaround is to retrieve the formApi from the context and use its getValue method:
We also created an Input component, which accepts a type prop. However, if type is not provided, it will try to match the field name and props to generate its type in the same manner as Simple Form. We could write a whole separate article about this magical component:
The resulting API is something pretty simple to use and easy to read:
Killing the legacy: Migrating all of the code
In order to handle this massive refactoring, we created a cross-teams task-force with one single goal: kill legacy code. As the tech team grows (we are currently at more than 25 developers and we plan to be 40 by the end of the year), it is very important to work on simplifying our codebase. No one is dedicated full time to refactoring forms, but we manage to dedicate roughly half a day each week to cleaning the codebase.
Technical tasks
“Tech tasks” at Doctolib are tasks with no direct product value (they’re not a new feature), but instead are those which have technical value (e.g.: improving developers efficiency, enhancing the platform performances, etc.). These large projects are prioritized in a tech roadmap which is maintained in parallel with the classic product roadmap. This is not only a refactoring time, it is also time allocated to experiment with new technologies, to keep ourselves up to date and so on.
Four of us worked on this particular task. We plotted a six month long roadmap for the transition to the new forms standard which started late December and the plan is to have the last form migrated by June.
How we kept the product evolving in parallel
The idea was not to jam the development of new features while refactoring the forms. To avoid doing so, we ensured that each form we migrated had no changes scheduled within the next few weeks. It is very important when you start such a refactoring to know the next product focuses and to work with the parts in which no evolution is planned. Still, as we went we did have to make some minor changes and bug fixes from the main trunk to our feature branch.
We also decided that this was a good time to update our forms’ graphic design, and to ensure its unification across the rest of the app:
How we kept the same great quality through the refactor
We were able to perform a quick and safe refactor with the confidence that the forms we were shipping worked just as well thanks to two things : a whole lot of unit tests, and even more integration tests.
We love tests at Doctolib: our test suite includes more than 6000 unit tests and 1000 end to end tests which run on Heroku CI for each commit we push to Github. It took a while, but we finally got a green build on our refactor.
For some of our forms which do not change graphically, we also have visual regression tests that helped us a lot (using a custom version of Argos CI).
Transmitting knowledge to the other developers
Since making developers’ lives easier was one of our main goals, sharing the knowledge to the rest of the team is a crucial part of the process.
We have a few channels to do so:
- By providing clean implementation examples: we have some forms which are tagged as “state of the art” which other developers can read and tinker with
- During “tech time” meetings: a weekly gathering of our technical team, where anyone can share something great they did that could be useful to others.
- Being a cross-teams project, there is a point person on each team that can help his team mates using the new API
- On an internal wiki
- Hopefully through this article :)
Conclusion
Gathering technical debt is unavoidable. When your business is growing fast, you don’t always have the time to keep your codebase perfectly clean and it’s probably not a good idea to spend too much time cleaning your code anyway.
You have to find the balance between over-engineering and mess to continue to expand your application and your business. Good development practices can help avoid technical debt, in particular, code reviews, but never forget to take a step back and figure out if you are still heading in the right direction.
