Journey for a home-made back-end, part I: where do we want to go
At the end of a previous post, we discussed the decision to build our back-end and pay back our technical debt. A couple weeks later, the first version is up and running and ready for integration within the Equisense Motion application. There will be a series of posts about the steps along the journey and the take-away advice we would give.
The first step was defining what the back-end was going to change in the current architecture and the features with the highest priority. These were prior to addressing other questions.
Then came the dreaded choice of a technology, dreaded because it can turn into an endless source of debates. To avoid this, everyone could list technologies they saw as plausible to use. After a quick talk and some research from all people from the tech team, we kept a shortlist of 3 stacks (language + associated dependencies) which could suit our case:
- Python with Flask or Pyramid
- Elixir with Phoenix (or Plug)
- Go
Five of us used 3 hours to test one of these stacks they had not worked with previously, on one key aspects we had to consider (authentication, ease of use, project modularity, tests, database access, …). Once well-informed, the last step was to find the right way to work as a team in each case. Since there would be no one dedicated to the back-end exclusively, we could not afford building a system with a high barrier to entry (no high expertise required to read the code base not contribute). That’s primarily how and why we went with Go.
Elixir would have been a great choice too, but one characteristic of the tasks given to the back-end is that we don’t have that many connections per unit of time compared to other startups: horse-riding does not involve as many users per day as a social network, a search engine or a messaging company.
However, it does involve heavy computations and data structures significantly bigger than just a message (imagine recording data from a sensor every two seconds for up to several hours, and then sending it to a server for further processing). The second point with Elixir was the smaller community, which is not a problem in itself, it just defines who Elixir developers are.
I see the Elixir community as craftsmen building their expertise on the language and the Beam VM, with a deep expertise of what is going under the hood.
For a more complete view of the differences between Elixir and Go, check out this excellent article from Codeship.
As for Python, the dependencies we looked at for API building are still in early development and we could not afford watching every release with anxiety. The syntax itself was great, but as people using it quite often for Machine Learning purposes, we know it can be limited in computational speed. A bit more than a year ago, I would have argued that typing is another limitation when scaling the team working on projects, but Python 3.5 and 3.6 have brought great possibilities for just enough types when needed for readability.
Now that the choice was made, we had gathered so many thoughts about how the core API had to be defined that the initial development was rather straight-forward. The key resources we are dealing with are users, horses, sessions and sensors, the mobile client applications have to be able to perform standard CRUD operations allowed per user.
At this point, here are my two cents to define your workflow when starting in a similar configuration as Equisense:
- Involve the product manager as much as possible for the what: deciding what resources are exchanged, what are the access rights of each application, what are the possible scenarios?
- Involve the client application developers as much as possible to define the how: make them write and keep up-to-date formal specifications on how are the routes defined, the HTTP verbs for each, the models for resources they are already using.
This will win precious time and avoid friction and slow-downs in the early iterations, and minimize changes in the API once it is defined. For that part, choose your tools carefully. Keep a document or board up-to-date with everyone and allow for no oral request or changes. Make everything go through the document/board. We use Jira for all development projects and it saves precious time: no unnecessary reporting of the weekly progress, no more meetings for things that don’t have to be discussed.
If you are just developing a back-end, your company is too young for the tech team to loose time chatting just for social purposes. Save that for the lunch break.
Now, of course you will need to discuss some topics, like which task should be kept on the mobile applications and which should migrate to the back-end. But these discussions will be even more fruitful as you spend them just on the critical and non-trivial aspects.
I’d love your feedback in comments or on Twitter. In the next post, we’ll take a look at the technical side of the early development within the Go ecosystem, stay tuned!
By the way, we’re looking for more awesome people to join the adventure: jobs.equisense.com
