Using Swift in AWS Lambda

Recently Apple open sourced the Swift compiler and standard library. This makes it possible to run Swift server side on Linux where previously they could only be compiled for iOS and OS X. Along with the code, Apple has made available binary packages for Ubuntu. While this is great for running on an existing Ubuntu server it doesn’t make it easy to get Swift code running in AWS Lambda which runs on an Amazon Linux AMI.

If you don’t know, AWS Lambda is an offering from Amazon that allows you to run abitrary code without managing servers or tuning scaling properties. It will automatically provision more computing power as needed and will only bill you for actual CPU time used by your code. This code can start from AWS events like Kinesis streams, DynomoDB changes, or call the Lambda function directly using either an AWS SDK (from a mobile or web app for example) or using the Amazon API Gateway to create a more traditional REST service.

Officially Amazon only supports Javascript, Python, and Java as languages for AWS Lambda. However as Lambda uses node.js for its Javascript runtime it’s possible to execute any program that we can bundle in our zip upload. Amazon even has a blog post about how to accomplish this.

It would seem just compiling Swift into an executable and running it from the node.js file is all we would need to do to get it running in Lambda. To test this let’s spin up an Ubuntu 14.04 EC2 instance and install the Swift compiler (if you are following along I’m using ami-d05e75b8 for this instance).

First we’ll create a directory to hold our swift compiler and a directory to hold our swift code.

sudo mkdir /usr/local/swift
sudo chown ubuntu /usr/local/swift
mkdir ~/lambda

Next we will download the Swift compiler from Apple and install it.

# Install dependencies
sudo apt-get update
sudo apt-get install clang libicu-dev
cd /usr/local/swift/
wget https://swift.org/builds/ubuntu1404/swift-2.2-SNAPSHOT-2015-12-01-b/swift-2.2-SNAPSHOT-2015-12-01-b-ubuntu14.04.tar.gz
tar xfz swift-2.2-SNAPSHOT-2015-12-01-b-ubuntu14.04.tar.gz --strip 1
export PATH="/usr/local/swift/usr/bin/:$PATH"

With this in place let’s create a simple Hello World swift app to try and run in Lambda.

cd ~/lambda
echo ‘print(“Hello World!”)’ > hello.swift

Running it via the swift runtime you’ll see it works.

swift hello.swift
Hello World!

So let’s compile this program so we can upload the binary to AWS Lambda.

swiftc hello.swift
./hello
Hello World!

Now we’ll create an index.js file in our lambda folder that should allow Lambda to run this swift binary (use whatever editor you are comfortable with to paste this into ~/lambda/index.js).

var exec = require('child_process').exec;
exports.handler = function(event, context) {
  child = exec("./hello", function(error) {
    // Resolve with result of process
    context.done(error, 'Process complete!');
  });
  // Log process stdout and stderr
  child.stdout.on('data', console.log);
  child.stderr.on('data', console.error);
};

Now zip this up so we can try it in Lambda.

sudo apt-get install zip
zip swift.zip hello index.js

We can now use this zip file to upload to AWS Lambda and when testing the function in the console (the input doesn’t matter as we ignore it) we should see our “Hello World!” in the log output.

...
"errorMessage": "Command failed: ./hello: error while loading shared libraries: libswiftCore.so: cannot open shared object file: No such file or directory\n",
...

Doh, of course if we look at the hello binary we created it’s only 16062 bytes which means none of the libraries are included in it. We can’t have our Lambda code install the Swift binaries everytime it initializes itself so we have to come up with another solution.

My initial thought was to statically link the libraries inside the binary.

# Install libbsd as we'll need it
sudo apt-get install libbsd-dev 
swiftc -L /usr/local/swift/usr/lib/swift_static/linux/ -lpthread -ldl -licui18n -licuuc -lbsd hello.swift

Great, we now have a hello binary that is 4MB so looks like it has everything we need, so let’s try and run it.

./hello
String(String(String(String(String(String(String(String(String(Strin...String(String(String(String(String(String(String(String(StringBus error (core dumped)

Well that didn’t go as planned. I have no idea what happened here but this obviously isn’t going to work. So we need to figure out a way to include the dynamic libraries our previous swift build was looking for but bundle them in our zip file.

Through trial and error here are the libraries we need with the proper naming convention Lambda is looking for.

swiftc hello.swift
mkdir lib
cp /usr/local/swift/usr/lib/swift/linux/*.so lib/
cp /usr/lib/x86_64-linux-gnu/libicudata.so.52 lib/
cp /usr/lib/x86_64-linux-gnu/libicui18n.so.52 lib/
cp /usr/lib/x86_64-linux-gnu/libicuuc.so.52 lib/
cp /usr/lib/x86_64-linux-gnu/libbsd.so lib/libbsd.so.0

Let’s update our index.js code to use this library path.

var exec = require('child_process').exec;
exports.handler = function(event, context) {
  child = exec("./hello", {env: {'LD_LIBRARY_PATH': __dirname + '/lib'}}, function(error) {
    // Resolve with result of process
    context.done(error, 'Process complete!');
  });
// Log process stdout and stderr
  child.stdout.on('data', console.log);
  child.stderr.on('data', console.error);
};

Now we need to rebundle this into a zip for Lambda.

zip -r swift.zip hello index.js lib/

And once we upload it into our Lambda function, success!

START RequestId: 1e4f8150-9f8d-11e5-9d12-eb0f0cb63aec Version: $LATEST
2015-12-10T22:26:55.319Z	1e4f8150-9f8d-11e5-9d12-eb0f0cb63aec	Hello World!

END RequestId: 1e4f8150-9f8d-11e5-9d12-eb0f0cb63aec
REPORT RequestId: 1e4f8150-9f8d-11e5-9d12-eb0f0cb63aec	Duration: 52.94 ms	Billed Duration: 100 ms 	Memory Size: 128 MB	Max Memory Used: 10 MB