Lessons Learned Using Server-Side Swift in Production
From a Lean Startup Perspective.
Server-side Swift is often just seen as a gimmick for your fun side projects, but nobody really seems to be using it in production. We at Courtastic challenged that and built our entire backend with Swift.
My goal with this article is to help you decide whether building your own backend with Swift might be a good idea. Not everything of it is related to Swift though — there are also some general learnings. After being in production for about 3 months (with a few clubs using our service), I thought you might profit from some lessons we’ve learned along the way.
First Some Technical Details
Our booking system backend is accessed via a JSON REST API by our iOS app, Android app and our ReactJS WebApp. We use the Perfect Swift framework and our backend is deployed on a DigitalOcean Linux droplet.
The primary reason we chose to use Swift for our backend was that we (as frontend developers) lacked the time to completely learn a new language and framework for this purpose to a level that we could use it in production. The summer season was coming and we needed a backend, fast.
Our iOS app was already very well developed and the idea of sharing Swift code with the backend as well as using my favorite language in the backend, too, seemed very compelling, so we decided to give it a shot.
After a few quick trials of different server-side Swift frameworks, among them Kitura, Vapor and Zewo, I chose to use Perfect, because it provided the most features for our particular use case (it perfectly fit us). Here is a nice article outlining the strengths and shortcomings of each framework.
Once the framework was chosen, I immediately started to build the foundation of our backend.
Hello World applications are not just there to give you a good feeling in tutorials. They also serve a purpose: simulating the process of going from writing the code to production deployment before you really start building.
You don’t want to spend hours writing the first few features of your application only to then learn that how you thought it would work in the real world (i.e. the internet) is not the way it actually works. Thus, after setting up your development environment (which, in case you use Perfect, can be automated beautifully with their assistant) you should try to get it deployed immediately.
Get it out there. Make sure everything works as expected. Only then start to build your application.
And once a new feature is built, deploy again and check.
It is also wise to think about security early in the development process. Get SSL certificates (we use Let’s Encrypt), add them to your server and make sure you can (only) access your backend via an SSL-encrypted connection.
The Database Connection
The next step, after making sure the deployment to DigitalOcean works correctly (good tutorial here), is to set up the database connection. Perfect Swift offers adapters for many different databases, among which we chose MySQL. We decided to let our database run locally on the same droplet as the backend itself, as this would allow us to make it more secure in the beginning by disabling any non-local access (only via localhost). If your application needs to be scaled quickly, a separate database server might be a better choice though.
After the correct ports were set up and I could make queries from Swift, I built a wrapper around it and never worried about it again.
What we also learned: make sure you really create a new database handle for each query (in Perfect, it’s just a separarte MySQL object). We had problems with simultaneous requests until we found out that we actually had to create the new handles ourselves. Suddenly, thousands of requests per second were possible (my cofounder wrote a script for that) where previously only two could lead to conflicts.
Sidenote: Perfect provides its own ORM (object-relational mapping) but as it didn’t provide the individual features and interface design I wanted for our backend, I chose not to use it. Before you use it, make sure it supports all features that you need right now and you might need in the future.
Our booking system is entirely based on users being logged into their respective clubs, so authentication was clearly the next important fundamental feature.
First of all, you should design the exact flow of authentication upfront. In Perfect Swift, there are several options:
- using Stormpath’s Turnstile framework for Perfect and build the logic yourself
- using the new Perfect local authentication features as specified here
- using 3rd-party OAuth providers like Facebook, Google or GitHub (all these are supported by Perfect)
As the ‘native’ Perfect authentication option wasn’t there when I started it, I chose to use Turnstile. Today though, the second option will probably be the best choice.
Using Turnstile, I was provided the basic structure and request parsers and only had to implement the connection to the database (saving tokens, signing up users, etc.). For me it wasn’t so obvious on how it works, so I will outline it here quickly:
- Turnstile processes the request and tries to find an authentication token.
- If it found one, it will call a protocol method that you need to implement in which you get the authentication information (an API token, for example) and need to return the corresponding user object from the database.
- Once you have implemented the above functionality, in every request handler (back in Perfect) you can access the user’s properties and the token.
- It works similarly with signup and logout: you are provided the user input parsed from the request and need to hook it up to the database.
Once you have your authentication all set up you can already build a fully functioning login, signup and logout frontend for all your platforms. Again, I advise doing this upfront as it might point you to problems in your authentication workflow that are harder to change once a fully-implemented backend is built around it.
Also make sure you encrypt passwords before sending them to the API and maybe once more with a random salt when you save them in the database. We oriented ourselves at this great article on security and hashing.
One core feature of our app is also that it is realtime. If something relevant for a user changes (e.g. a new booking has been created in his club), we want all currently connected clients on all platforms to reflect this change immediately. That’s why we use Perfect’s WebSocket support framework and have a separate port on our server that handles websocket connections to all clients.
It was very difficult to get the socket to work reliably all the time on all platforms. Some clubs have a client running 24/7 in their clubhouse that serves as a dashboard and shows the current bookings on all courts of this club. In my experience, WebSockets are generally not very reliable if you just have occasional updates and are running it 24/7.
That’s why, in the end, we solved it by sending keep-alive messages every 60 seconds. We don’t feel very good about this solution though and are still searching if there’s a better way to do it (but currently it seems to be the only option that provides the desired reliability).
Designing the Main API
Decide on a Set of Guidelines
When you’re working with someone else and you build an API that their client app will use, make sure the structure you use is
- easy to work with on every platform
- documented very well (we just have a private GitHub repo that documents exactly how the backend works)
You might not know how the responses from your API will be handled in a client app that you don’t implement, so agree on a structure with everyone involved.
Clearly Specify Errors
In Swift, you can write wonderfully extensible enum types. In our particular application, errors might not just be codes but also have associated objects (like the booking, that prevented your booking). Here’s an example:
Then, document clearly which errors come with associated objects and which don’t. Make sure you stick to it consistently throughout the whole application.
Design for Extensibility
When you’re a lean startup and are likely to add many new features in the future (as we did extensively), make sure your backend code allows to do this easily.
Some ideas on how to accomplish this which I used:
- Create a protocol for all data entities you deal with. It can encapsulate common properties (id, createdAt, updatedAt) and provide an interface for the database as well as the JSON API. You can then create Swift protocol extensions that already implement all the logic on how to do most of this functionality, so if you want to add a new entity, you can focus exclusively on the new functionality, not the standard stuff. The protocol I use looks something like this:
- As mentioned above, encapsulate all database logic in one place. Hence, if you change from MySQL to PosgreSQL for example, you only need to change one file.
- Create little checklists and guides for yourself. If you revisit some part of your application later in order to add a new feature, you might not know the details on how to work with your own services anymore. Of course, you should always add documentation and comments to your code (says my software engineering professor), but an extra little step-by-step guide for yourself or others when you work in a team like ‘How to add a new data entity’ might prove invaluable later.
About three months ago (April 2017) we went live and clubs started to actively use our service. And… not everything worked immediately. As stated above, the socket issue for example took us a while. Other problems like platform / browser incompatibility were also things we had to deal with as they were hard to test upfront. Here are some things we learned along the way:
Have a Development Server
We soon created a separate DigitalOcean droplet that‘s used by ourselves when we roll out new features. You should try to design your code so that you can deploy to both backends (production and development) without any changes in your code.
The other day we added new features that required some changes in the database. Because we have an exact clone environment, we could first perform the changes on our development server, record the steps, check that it works correctly and then confidently do the same with the production server.
If you use DigitalOcean, you can just make sure your production droplet works perfectly and then clone it (or restore from a backup). What we did is we just assigned a new subdomain ‘beta.’ to this new droplet, so the clients just had to switch this one url if they wanted to use the development backend.
Logging and Analytics
We received some emails saying, in effect, ‘I tried X and it does not work’. Of course, when you don’t have many users, you can ‘do things that don’t scale’, and have a conversation with each user individually, but keep in mind that often your users are not technical. We wasted much time and effort just trying to understand what exactly was not working.
It would have been much easier if we added better logging and analytics, so we could see exactly what went wrong and which part of our system is the problem. Perfect provides a very good library for the server-side logging: Perfect Logger. You can choose individual files (on the server) for different kinds of logs. Make sure you log the time, device info (if you have it), concrete request and logged in user (i.e. state of the backend). Later, if there’s a bug report like the one above, you simply pull out your logs and check what actually ‘does not work’.
On the client-side you can also add an analytics or crash-logging framework to monitor where the problems occur.
Dealing with Framework-Related Problems
At some point, you might encounter a problem that you think is related to the framework you’re working with. Or you might wonder how you could implement a special feature that is not explicitly supported.
It did happen to me, but fortunately there are great server-side Swift communities ready to help you anytime. Perfect specifically has a public Slack channel filled with employees and other developers that provide answers in minutes. The other companies often provide similar platforms too.
The great thing about these server-side Swift frameworks is also that most of them are completely open-source. Hence, if you really find a bug and nobody happens to be online in the Slack channel, you could just change the reference in your dependencies to point to your forked version of the framework.
The Future & Conclusion
As the server-side Swift communities grow and more examples of successful production apps are out there, I firmly believe server-side Swift will become more and more of a viable option.
Especially in a lean startup environment or if you also have an iOS or macOS app, server-side Swift helps you build a solid backend, fast. If you architected your iOS app well, you might even be able to copy-paste parts of your code. Either way, I had the experience that Swift is just so flexible that you can connect different existent functionalities to create whole new features in an instant. In the last few weeks we have added so many features I can’t even count them — and even though we have never thought about them when we started, they were very quick to implement.
The server-side Swift frameworks are constantly improved and expanded. It’s been only a few months that I built the main part of our backend; and now when I revisited the resources for this article, I was impressed how much has happened in the meantime. Perfect now even has a wrapper around TensorFlow so you can build machine learning into your backend!
I hope you enjoyed this article and you learned something from our mistakes and our process. I was inspired to write this article by the amazing book ‘Show Your Work’ by Austin Kleon. Agreeing with the author, I believe if you work on a project and in the process learn something new, you should share it so others can learn from your mistakes and insights.
- Perfect.org provides many great tutorials and example applications to get started. Check out their docs on GitHub too.
- The guys at Raywenderlich.com also have a nice free video tutorial about server-side Swift with Perfect.
- This talk presents some more arguments and goes into the details on how to implement a Swift backend (with Swift@IBM’s Kitura framework though): https://news.realm.io/news/tryswift-chris-robert-end-to-end-application-development-swift-backend/
This is my first Medium article. If you have any suggestions on how I could improve it or find any mistakes, please tell me! Any feedback is very welcome.