Building a backend for mobile apps in Go

In this article I will share some experiences from my latest project. A colleague and I created a new back-end service for the iOS and Android apps of a video streaming service, using the programming language Go. It went surprisingly well considering that we had no previous experience in Go.

The Big Project

For the last 18 months or so, I have been consulting for a video streaming company. I was heavily involved in this Big Project where we built the next generation of client apps for our Video On Demand (VOD) service. We had four client apps, one middle layer and one Content Management System (CMS) in the Project. We put a lot of effort to make things robust and well structured, we were proud of what we had done, and we would soon be ready to release it. My role was to be a solution architect for the client layer, so I had a close collaboration with all the teams and roles within the Project. It was intense and challenging, and we had a lot of fun.

But then one day, we got the news that the Big Project was cancelled. And that all us consultants were to leave the company within 30 days.

This was of course a shock for everybody in the Project! An interesting thing with a major change like this is how different people react to this kind of news. Many were devastated, this was probably the biggest, most significant project of their careers so far. Some immediately started looking for opportunities elsewhere. Some spent their days on social networks. Some became sad, even depressed. And some started to figure out what we could do instead, leveraging the vast amount of experience that we had gained so far. I was in this last category.

Best Friends Forever

As the Big Project was now cancelled, we had to come up with a plan for what to do instead. The previous generation apps for Web, iOS and Android had been developed and maintained by external suppliers. So the directive from management was that the internal developers should take over these existing apps. These apps would then be gradually improved to fulfil the visions that we had for our next generation apps in the Project.

When we received the code for the previous generation iOS app, we saw that it was written in Objective-C and not in our preferred language Swift. Our iOS developers (the ones that were not consultants) felt that it was demoralizing to have to go back to an old language like this. They wanted to keep working on our new Swift app!

This is where I find myself suggesting this crazy idea that I had. What if we would create a new thin middle layer that would imitate the middle layer that we had in the Project? Let’s call it a BFF (for Backend For Frontend, which is an emerging pattern). This BFF would not have any data sources of its own. The BFF would use the same data sources as the previous generation apps, but convert the data structures to the way that we had designed them in the Project. This way, the new iOS app could co-exist with the other previous generation apps.

Now, what technology would we use for this BFF? The middle layer that we had built in the Big Project was written in Java. But there had been a lot of debate about this, and we had been struggling with convincing the employed backend developers that this was a good choice. What would they have preferred instead? One language that I had heard a lot of good stories about was Go. This was also one of the preferred languages of the organization, so it made sense to take a look at it. I read some articles and realized it looked promising.

All this happened on the day after that the big project was cancelled. All consultants had a 30 day period where we would close down the Project gracefully and document our experiences. When we discussed this BFF idea in the iOS team, we realized that we could use those 30 days to start building this thing and see how far we could go. The BFF developers would be Tord, one of the consultant iOS developers, and myself. As long as we also fulfilled our duties during the 30 day close down period, nobody could really say anything about it. So we decided: let´s do this!

Learning (to) Go

Tord and I started to read various articles to get going (for example this one). Tord started working on an API for getting metadata about video assets, while I worked on APIs for Login and video stream progress (i.e. how much of a video asset the user has watched).

One of the first obstacles when programming in a new language is to figure out what tools to use. Since the language is unfamiliar, it is important that the tools are familiar. So, for myself, I immediately started looking for plugins for IntelliJ. I have been using Jetbrains products for about 10 years now, and they usually make me very productive. And luckily, I found a working plugin and I got some autocompletion and intellisense going. Tord started out with more simple tools, like the terminal and a plain text editor, but when he saw the things I could do in IntelliJ, he switched over. Everything is still not perfect in IntelliJ, e.g. debugging doesn’t work well, but it’s still better than plain text.

Initially, we only ran the code on our own computers, we didn’t have access to any server or cloud environment where we could deploy it. So when we wanted to test our code with the iOS app, we had to run XCode locally too.

As I remember this phase, it was fun and energizing for us. But almost everybody around us was sad and had problems with motivation, due to the cancellation of the Project.

Getting management acceptance

Even though we were more or less free to do whatever we wanted, we realized that we’d better seek some support among the managers. Since there was a quite a lot of experience with Go in the organization, we wanted to be able to ask them questions and we needed them to want to help us. So we sold the idea to the relevant managers and got access to one of their best Go developers, Peter. We also discussed what goals to set for the rest of the 30-day period, and that we would have some code reviews to push ourselves in the right direction.

One thing we realized during the code reviews was that we had a tendency to create our own bread and butter utility classes. But they were mostly redundant. I suppose this was due to our developer experience with other languages. In Go, most of the built in packages are good enough to use straight away, and there is not really a need to create too many abstractions. It will just make the code less readable for other Go developers.

Once we had the management acceptance, we could also deploy the BFF in the cloud, and the rest of the iOS team could start playing around with it too.

Around this time, the Android team had also come to the conclusion that they wanted to keep working on the app that they had built in the Project rather than taking over the maintenance and development of the previous generation android app, so they jumped on the BFF train too!

Structure

At this point, it became obvious that Tord and I had different views on how to build things. We worked on separate parts of the code base, and it wasn’t very easy to contribute to the other persons parts.

That’s when I started to think about the internal structure of this application. I realized that we needed some kind of reusable internal object that would abstract the connections to underlying systems and APIs, and make them functional. We call these Services. Then we needed some kind of objects that would take care of the requests from the client apps, we decided to call them Handlers. This gave us an internal structure of two layers with Handlers and Services.

BFF structure

Each Handler takes care of the requests of a given endpoint in our API. For example, the AuthenticationHandler takes care of the login requests from the client and returns a session token. The client will then include the session token in later requests.

A Service on the other hand, should perform a well defined task and have an internal interface that the Handlers can use. A good example of a reusable service is the AssetDetailService that gets asset metadata for a given video asset. This is used by the AssetDetailHandler, but also in other situations when we need asset metadata. For example when presenting the videos that the user has started watching.

In my experience, it is important to distinguish between the underlying system’s data models, and the ones that we expose to our clients. In fact, I believe that the underlying system’s model should be private within the Service. This is to ensure we have control over all communication within the application. This approach makes it much easier to adapt to changes, and you never have to think “who owns this model?” and “can I extend this model?”.

Another important design aspect is that a Service may not use another Service. In those cases where we need to combine data from several services, it is the responsibility of the handler to that.

Tord and Peter agreed with this structural approach, so we refactored the code so it would comply to the architecture.

End of 30 days

By the time that our 30-day period was about to end, we performed a demo of the BFF to our stakeholders. The demo went well and we were able to show most of the features that we had implemented, both with a raw REST client and with the real iOS and Android apps.

We had identified 17 API endpoints in the middle layer of The Big Project that the mobile apps were using. We had already implemented 5, and 3 were almost done. So about half of the endpoints were left to do. Pretty good for two newbie Go programmers in 30 days!

A new hope

On the same day as the demo, there was a management decision meeting about the future of the mobile apps, and they decided in our favor! They wanted to keep our BFF and release it with our iOS and Android apps. But who would do the remaining work? Tord was leaving us anyway since he was relocating. I got a new assignment to work with the architecture of the BFF and the client apps. Peter and Mikael, another Go developer, were assigned to continue the BFF development.

Four months later

Let’s fast forward a bit. Here we are now, four months later, what has happened?

We have been able to continue our work with both the BFF and the mobile apps, and we have had access to a few more developers. A product owner, a UX designer and a tester was also assigned to the project.

The code base of the BFF has grown and instead of two layers, we now have three. Some problems have required us to think outside the box and solve in unusual ways. And we can now clearly see that we are going to need to move parts of the BFF to another separate service. All this is part of the natural evolution of a software project.

We have reached a point where we have implemented all required functionality and the product is stable. Last week we released a Beta to a group of invited test users. The receiption has been very good, and we expect to release the apps to App Store and Google Play within two weeks. I think this is really exiting! Just the thought that I was part of this doomed project that crashed and burned, but managed to survive.

Don’t get me wrong, of course we have had problems. Misunderstandings. Performance issues. Design issues. Disagreements. But in my opinion, these things are natural and nothing to be afraid of. By working together, we have solved most of the issues.

Learnings

So what are the learnings that I will carry on to my future projects?

• A small project group is vastly more effective than a large project group. We managed to produce a releasable product with 10 developers in five months. Compared to the Big Project that worked over a year with 40 developers and never finished.
• Go is a great language for creating REST API applications, and I hope to be able to use it in future projects.
• Before making decisions about technologies, talk to the people that will end up maintaining the product. And involve them in the project. Let them do most of the coding and let them take the ownership.

My name is Anders Romin. I work as Solution Architect at Apegroup: a design, technology & communication agency in Stockholm, Sweden. We help brands, enterprises and organisations build great digital products and services with engaging content. Want to know more about how we work? See our website.

If you enjoyed this case study, don’t forget to press ♡ below! And if you have any questions please, bring them on!