Design Decisions for Project Template

This write up goes through some of the thought process that guided our project template that was discussed at a higher level in a separate post:

Technology Agnostic: Most of our project utilizes different technologies, such as ruby, python, nodeJS, or others. So our project template needed to be agnostic to these technologies. This was the predominant reason why the template utilizes mainly shell, docker, and docker-compose. With containers, we no longer needed version management tools like rvm, rbenv, chruby, pyenv, virtualenv, nvm, etc, because we just needed to change the version in the Dockerfile.

Shell Scripts: The terminal is the common dependency of all of our services. Hence, we scripted all of our typical tasks in shell rather than any particular language, even if the shell script is just calling a ruby or python task. Our common tasks includes:

  • bootstrapping the project: This task helps new developers bootstrap the project. Additionally, it ensures the project is clean of artifacts from a previous task/feature so when starting a new task/feature, the project/container is in the expected state.
  • setting up the application: This task involves things like creating and migrating the database. It can also seed the database so developing user-interface features will be easier since there is existing data to test against.
  • starting the application: This task starts the server. Details like specifying ports and removing previous server process PIDs can also be useful in this script.
  • testing the application: This task runs the tests and can also run linters, style/format checkers, and other static code analyzers.
  • backing up the application: This task backs up the data and schemas required for the application. The application might be using multiple databases so this script will take care of these types of things.

To share common subtasks between each of the scripts, we defined shell functions. Things like waiting for the database server to be up would fit into that file.

Another benefit of having these common tasks scripted is they can be handled by a CI/CD pipeline quite easily.

Reusable Containers: Our approach is to use defaults as much as possible so we stay in line with the assumptions made by the tools’ creators. So for example, our main Dockerfile and docker-compose.yml files are located at the root of the project. This makes it easier when using commands like docuer-compose up.

However, to be able to reuse the containers when another dependent project requires the current project, we extend docker-compose.common.yml, which contains the common configurations that are required to instantiate the service. This allows dependent projects to include the current project as a git submodule and extend from the same docker-compose.common.yml.

Another benefit of using containers with the shell scripts, is that the application is bootstrapped and setup the same way each and every time, hence, avoiding the “it works on my machine” problem. Operating system dependencies are specified in the Dockerfile. Environment variables are specified in development.env, which are sucked into the containers via docker-compose).

And that’s it. That’s the core of the project template that we use. It helps us pick up and be productive quickly on each of our applications as well as avoids bike-shedding. I hope it is useful to you as well and if you know of ways to improve on the template, please do let me know.