Serverless Swift

Running Swift Code on Amazon Lambda

Claus Höfele
Nov 20, 2016 · 5 min read

Out of the box, Lambda only supports code written in JavaScript (Node.js), Python and Java. However, it’s easy to use Node.js to run an external process written in any programming language you want. In fact, Amazon already provides a template called “node-exec” in Lambda’s Management Console for this purpose. Thus the key to running Swift code on Lambda is to compile your app into an executable that can be invoked by a Node.js shim.

Hello Lambda

As an example, the following code simply echoes input data provided at the command line back to the caller:

import Foundationlet inputData = FileHandle.standardInput.readDataToEndOfFile()
FileHandle.standardOutput.write(inputData) 

Combined with a basic package file for the Swift Package Manager, you can build this program and try it out in a terminal window:

$ more Package.swift
import PackageDescriptionvar package = Package(
    name: "Lambda"
)
$ swift build
$ .build/debug/Lambda <<< "Hello Lambda"
Hello Lambda

Building for the Target OS

If you are new to Docker: it’s a tool that allows you to easily run pre-packaged images that contain everything needed to execute a piece of software — in this case the Swift compiler for Linux.

$ docker run \
    --rm \
    --volume "$(pwd):/app" \
    --workdir /app \
    smithmicro/swift:3.0.1 \
    swift build -c release --build-path .build/native

The Docker command above uses Swift version 3.0.1 (indicated by the label of the Docker image after the colon). Since the container shares the build directory with the host, the build-path option places the compiler output files in separate folder so they don’t interfere with builds on the host.

Making a Swift Executable Self-Contained

The following scripts and ideas are based on Alexis Gallagher’s SwiftOnLambda project which provided the initial working code to execute Swift programs on Lambda.

What I haven’t told you when building your Swift code in Docker: the Swift image above uses a different Linux distribution (Ubuntu) than Lambda (Amazon Linux, based on RHEL). The reason is that Swift.org currently provides packages for Ubuntu only. However, executables built on different Linux distributions are compatible with each other if you provide all dependencies necessary to run the program.

The Swift executable itself is only half the story — it depends on a number of shared libraries that the operating system will load when it starts. Fortunately, the ldd command will tell us what those dependencies are.

An alternative approach would be to build the Swift compiler for Amazon Linux based on instructions for CentOS by もどかしい. However, I haven’t explored this option.

Using ldd and the following commands, you’ll receive a complete set of shared libraries used by your executable (roughly 50 shared libraries, including the Swift Core libraries, C/C++ standard libraries and all other dependencies):

$ mkdir -p .build/lambda/libraries
$ docker run \
    --rm \
    --volume "$(pwd):/app" \
    --workdir /app \
    smithmicro/swift:3.0.1 \
    /bin/bash -c "ldd .build/native/release/Lambda | grep so | sed -e '/^[^\t]/ d' | sed -e 's/\t//' | sed -e 's/.*=..//' | sed -e 's/ (0.*)//' | xargs -i% cp % .build/lambda/libraries"

To prove that the resulting package works, you can run your Swift code inside a Docker container that comes close to Lambda’s execution environment based on images provided by Amazon (unfortunately, Amazon only provides Docker images for version 2016.09 of Amazon Linux whereas Lambda uses 2015.09):

docker run \
    --rm \
    --volume "$(pwd):/app" \
    --workdir /app \
    amazonlinux:2016.09 \
    /bin/bash -c '.build/lambda/libraries/ld-linux-x86-64.so.2 --library-path .build/lambda/libraries .build/native/release/Lambda <<< "Hello Lambda"'

ld-linux is Linux’s dynamic linker that loads the shared libraries needed by a program and then run the program. It’s similar to setting LD_LIBRARY_PATH, but running the executable via ld-linux from the Linux distribution that built the executable ensures that also the glibc version matches exactly the one needed.

Providing the Node.js Shim

const spawnSync = require('child_process').spawnSync;
exports.handler = (event, context, callback) => {
    const command = 'libraries/ld-linux-x86-64.so.2';
    const childObject = spawnSync(command, ["--library-path", "libraries", "./Lambda"], {input: JSON.stringify(event)})    // The executable's raw stdout is the Lambda output
    var stdout = childObject.stdout.toString('utf8');
    callback(null, stdout);
};

The script does exactly the same as the previous Docker command and executes the Swift code via ld-linux. This also explains why it was necessary to write the Swift program to take input from stdin and write output to stdout: this how the Node.js shim communicates with the Swift executable (the data from stdin is provided as the event argument to the handler function).

Deployment

$ unzip -l lambda.zip
Archive:  lambda.zip
Length     Date   Time    Name
--------    ----   ----    ----
   10840  11-19-16 00:05   Lambda
     419  11-19-16 00:05   index.js
       0  11-18-16 21:50   libraries/
  154376  11-19-16 00:05   libraries/ld-linux-x86-64.so.2
  665904  11-19-16 00:05   libraries/libasn1.so.8[...]more libraries[...]--------                   -------
68000987                   55 files

Head over to the AWS Console and create a new Lambda function with a custom blueprint and no trigger. Give the function a name and select the Node.js execution environment. As code entry type, choose “Upload zip file” and select lambda.zip as described above. Leave the rest to the default values (128MB memory is plenty for a Lambda function using Swift).

The upload will take a while and you can test the new function straight away in the AWS Console (pick any of the test events).

Where to Go from Here

Claus Höfele

Written by

iOS at @here. Created “Stolpersteine in Berlin”, AlexaSkillsKit for Swift & CCHMapClusterController.