Server-side Swift: Making Canopy (3/6)

8 min readDec 9, 2018

Last week I talked about how how I chose Perfect as the base for Canopy’s Server. This week I’ll be talking about cloud providers, Swift on Linux, the development process and deployment.

What is Canopy?

Canopy has apps for macOS and iOS, and it’s available here.

Cloud Providers

If you chose Vapor, they provide free hosting for your server-side app, up to a limit. But it can make it a great choice for getting started, and depending on how much load you expect, it may be fine for good.

Kitura (by IBM) also offer simple deployment and hosting solutions.

One of the reasons I liked Perfect as it can run anywhere Swift could, there is no magic going on, you simply deploy a single binary and run it.

Thus I had choice of any cloud provider I wanted.

I had experience with AWS, so I went with it. Honestly I don’t think there is a great deal of difference between the various options. They all run Linux and they all run in the Cloud. AWS is the oldest, the best known, the choice with the most answers on StackOverflow and getting AWS experience looks great on your résumé.

Swift is not that portable on Linux currently, many distributions do not yet provide packages, which is a little surprising and sad. Ubuntu however has first class support from Apple. So I spun up an Ubuntu 16.04 instance. Amazon’s EC2 makes this really trivial, simply tapping a checkbox. They also have instance types that are free up to a certain amount of usage. I started with a free instance.

Amazon give you a private key that you need to add to your ~/.ssh so you can use ssh and have terminal access. Which is key, if you are not familiar with the command-line you are going to suffer, fortunately the macOS command-line is very similar to Linux, so if you have used it, and as a developer, that is probably true, then you should be fine.

You will need to amend your SSH configuration so it uses this key when you ssh to your server. On the left is an example. The final line is only required if you want to use the super handy rmate command so that you can edit remote files with your local instance of TextMate.

Now you can type justssh canopy and you’re in your new instance (obviously pick a sensible, different name).

I suggest configuring an Elastic IP straight away (it’s free) so you can change instance type without its IP changing. I changed instance type several times trying to find the right combination of capabilities and price.

Which Ubuntu?

I chose 16.04 even though 18.04 is also a fully Apple-supported option. Initially I did this because the 2 years 16.04 has been available is long enough that any other support issues you may have will have discussion posts somewhere on the Internet. I did at one point try to upgrade to 18.04 because I needed a newer version of libcurl than 16.04 provides, but I had problems I couldn’t easily solve, so it confirmed my initial reasons for choosing 16.04 and is what I’d recommend. 16.04 still receives crucial security updates, so it is fine.

Docker?

Docker is super popular nowadays for making it trivial to deploy images of your server-app and the Linux system configuration it needs, as well as run a local instance for testing. I chose not to do this since I hadn’t done any server work in so long and I wanted as few potential points of failure as possible. Either way it doesn’t change the rest of this document.

Configuring Your Instance

You should configure Ubuntu to automatically install security updates unattended. You’re running a server now, bad people may well try to hack it and install crypto miners or (much) worse, steal your user’s data.

Doing this is actually just a one-liner, here’s the documentation.

You should also configure your server to automatically keep it’s system time accurate. Many things a server does depend on your server time being accurate within a few seconds of actual time and will subtly break otherwise. Avoid the hassle of strange bug reports and errors and do this now.

Installing Swift

When getting started it’s easiest to install Swift on the server and build your app there. But let’s be clear: you should not leave this in production. One of the beauties of Swift is that it builds binaries and does not require Swift, a runtime or any build tools to be installed on your server. The build products are entirely self contained. A secure server has the bare minimum tools installed that are required to function, this makes it less susceptible to exploitation by bad people.

The process for making a stock Ubuntu instance work with Swift is quite painless (though you can’t yet use the system package manager (apt), update: I’ve been told that it is there, I just couldn’t initially find it).

Install the dependencies Swift requires (sudo apt-get install clang libicu-dev)
wget the latest Swift (Ubuntu) tar.gz from swift.org
Uncompress somewhere sensible, I suggest straight in your user home directory since this is not a general purpose machine
Amend your PATH in .bashrc to include the Swift’s bin directory

That’s it! Type swift and check the REPL works.

Development

You will be using SwiftPM to build your server-app. This requires a Package.swift in the root of your project. Here’s Canopy’s:

Now typing swift run will run your server. I suggest working with Xcode though, to do this type swift package generate-xcodeproj and you’ll get an Xcode project that will run your server when you ⌘R.

During development you’ll want to run a local instance of your server-app on your Mac and have your apps work against that. This heavily reduces debug cycles. As I said in part 1 of this series, if you have a single project for both client and server, you can share code and reduce the severity of rev-lock.

However, you inevitably will want to deploy these development builds, and that’s easy using rsync:

The script synchronizes your sources with your server, builds the app and then asks systemd to restart it. In the script I used foo where you would change foo to the name of your server in your .ssh/config.

I‘ll be talking about systemd and other production concerns next week.

Notably with the cheapest instances compiling your code takes forever. Which may be reason enough to upgrade your instance, or get straight into cross compiling locally, I talk about these options next.

Production

At some point you will decide the time is now: time to go live! Production seems scary, but really you just have to be prepared. At this point you have a tonne of practice and you will know your app and its code really well, so have confidence in yourself.

Firstly configure a password for your user so that sudo requires a password (by default AWS instances are password free for sudo)

You should setup cross compilation locally or a Docker Ubuntu instance so you can build your app locally and then scp it to your server. You can now delete Swift on your server and strip the remote package installation down to a bare minimum. Swift binaries require a few things, notably libicu and libcurl to remain.

I found getting cross-compiling did not work due to a number of curious decisions by the SwiftPM team: specifically #if os(Linux) does not work in the Package manifests and it turns out my dependency graph is full of such statements. I understand why these statements are there: it’s a language feature. I don’t understand why Apple have decided not to support this language feature in Package.swift files. Instead they intend to force users to specify platforms as part of manifest declarations. IMO this will bloat every single place platform specification may be required (ie. every subpart of every subpart of the DSL), which will take years (based on current progress) meaning the feature will be impossible to make work (thus forcing #if os because there is no alternatives as yet). And after all that #if os is still way more flexxible. Why are the SwiftPM team fighting against the language? You tell me.

To summarize, I had no option but to use a Docker instance to build ,my Linux binaries because SwiftPM cannot cross-compile my dependency graph. Fortunately Docker just works.

You can use a simple script to build locally, scp the binary and call systemctl restart foo to restart your server-app.

Note, specify --static-swift-stdlib to swift build or the resulting binary will not work without Swift installed (well strictly, it won’t work without libswift.so and libFoundation.so in the LD_PATH). Also don’t forget to build it with release optimizations (-c release)!

Upgrading Your Instance

With AWS upgrading your instance is trivial. The storage for (most) instances is not attached to the machine, so you turn it off (tap a checkbox) wait for it to finish, change the instance type and then boot it up again. This is a little scary the first time, but it is actually painless and ssh foo continues to work fine.

Once you go live I suggest a higher tier, but since Swift is high performance (relatively) you can stick with a T3 IMO, at least for starters.

Scaling

I don’t have anything to say on this as Canopy has not gotten to the point where it maxes out the CPU on my instance, and I have a few tiers I could still upgrade if that happened. But it’s something to consider, however making a system that can run on multiple machines in the cloud is a much bigger deal than anything I have talked about here. It is by no means a trivial change and as the Swift eco-system stands there is nothing I know of to make this easy. So: be warned!

Part Four

Part four here.