From Confluence to Automated Development Environment
I joined Doctolib as a Software Engineer in late 2021 with the goal of joining a special team that would be created at the start of 2022. This team, called DevTools, had one purpose: to empower developers and provide them with best in class tooling. When I joined the team, developers had to follow multiple pages of documentation to set up their development environment. Considering our recruiting pace and how fast our stack was evolving, we started to automate this setup.
We will see here how the team managed to automatize development environment management, from the installation for a new joiner and the update for the whole tech organization.
The Doctolib application development environment
Doctolib is a monolithic application with a Ruby on Rails / Typescript & React stack. Around 400 developers are working on this application and need to have this stack alongside with all the requirements for local testing (docker/compose custom ca certs ruby version management, node version management) installed on their laptop (which come in two flavors: Linux and macOS).
What was the previous setup?
At Doctolib, every new member attends what is called the Doctolib Academy. During their first month, the new joiner follows a hands-on session to ramp up on their future role in Doctolib. For new joiners in tech org, a dedicated session is set up in the Doctolib Academy so that someone can help them install and debug their dev env, and each time there were people attending, lost in the docs.
Until early 2023, there was a confluence page explaining how to set up the dev env. This page contained:
- Scripts samples that users had to copy and paste (and sometimes adapt)
- Links to some documentation to perform standard installation (like the commit signature)
- Links to scripts that were directly maintained in the monolith
This was very error-prone.
It got more complicated when we had to update the stack, we needed to:
- Update the Documentation
- Send an email to the tech team advertising about this update
- Send a Slack message to the tech team advertising about this update
- Provide support to the users that did not read the email or the Slack message (or the doc)
The result of all of this was that there was no homogeneity in the different workstations, and it was very difficult to provide support to developers when their development environment was broken.
Creation of DevTools Team
In 2022, a DevTools team was created in the Tech and Product (T&P) org. The goal of this team was to provide best in class tools to increase developers productivity and quality of life.
Very naturally, developers took the habit of contacting the DevTools team when they had issues with their dev environment. It was quite difficult for the team to handle support requests for such heterogeneous environments. Furthermore, the DevTools team was not working directly on the monolith, and it took time for them to reproduce and fix the issue of the developer.
The team decided then to create a tool that would install the whole development environment.
What would we use?
Reading the documentation made the DevTools team feel like everything could be replaced by a script. But the team wanted to make this maintainable and idempotent, so a choice was made to use Ansible to install the development environment.
As Ansible was not necessarily a techno known by the developer (the stack of the monolith is Ruby/React), we needed to find a way to launch the playbook easily. Fortunately, the DevTools team had just created an internal developer CLI. The Ansible playbook would be launched by this CLI.
At the end, developers would just have one command line to install, update and repair their environment.
Splitting into different blocks
The team decided to split into different blocks (using Ansible roles), from the most generic to the most specific.
Setup
The Setup block installed all the configuration needed for the users such as:
- Git config
- SSH key creation for pushing on GitHub
It also installed a package manager. We decided to use Homebrew for both OS in order to have only one package manager configuration to maintain.
Dependencies
This block installed all the dependencies needed in our stack. It mainly installed Rbenv for ruby versions and NVM for node which are the tools used to manage respectively Ruby and Node.js versions.
Cloning the monolith
Obviously this block cloned the monolith if it did not exist. It also installed the Ruby version and Node.js version needed by using the version declared in files from the repository.
This is not an exhaustive list of the blocks that the team set up. The idea of this decomposition is to let the team provide different kinds of installation in the future (depending on team profile).
The special Case of the Bootstrap
As mentioned above, we wanted to use the CLI created by the DevTools team to install the Dev Environment. But, this CLI (installed with Homebrew) required the user to have an SSH key allowing him to get the package from our private GitHub org. We also wanted those SSH keys to be created by the Dev Env. We had to find a solution.
The solution was to create another playbook named the bootstrap. This playbook ran the setup block described earlier and install the CLI.
Then we created a script that:
- Installed Homebrew and Ansible
- Ran the bootstrap playbook
- Ran the command installing the dev env when the CLI is installed by the Dev Env
With this, the user only needed to run this script for the first run.
Updating the Dev Environment
The Dev Environment is distributed using Ansible Collections. Every time an update is done in the Dev Env, the collection is updated. As the dev environment playbook is launched with the CLI, there is a step in the command that updates the collection on the user laptop if it has been updated before running the playbook. With this, we are sure that the user always has the latest version of the Dev Environment.
Conclusion
Here are some key improvements we observed after the release of our new development environment:
- A better ownership of the dev environment from the DevTools team
- Faster environment repairs
- Faster setup for new joiners
- Easy maintenance of the development environment
It has been 8 months since the tool installing the development environment was released. The dedicated session for helping new joiners to install their environment has been removed because a few developers came at those sessions, and everything was working fine. The DevTools team has now a better ownership of the Development environment, and it has become easier to help users to repair and maintain their environment.
