Introducing Apex

Serverless architecture with AWS Lambda

Apex is a small program written in Go for managing “serverless” architecture via AWS Lambda, allowing you to focus on code instead of infrastructure.

I started Apex because I’ve been working on a number of product ideas as a solo engineer, I can’t afford to spend time managing and maintaining machines if I’m going to have a successful product as a single person team. I love Amazon’s Elastic Container Service, but even managing and maintaining clusters of machines and the ECS agent is a lot of overhead if you don’t need the additional flexibility.

AWS Lambda has some limitations, as you cannot install system libraries and so on, but for many problems it can be a great solution. The main problem with Lambda and API Gateway is that the usability just isn’t there, so Apex is an attempt to fix this!

The project has already been seeing some great contributions and we’ve just released v0.4.1, so I’ll do a quick run through of Apex so far.

Project structure

Currently Apex has two concepts, a “project” and a “function”, where a project is simply a collection of functions and other resources, and a function is a AWS Lambda function.

Having project level management allows you to specify defaults and other useful attributes globally, and optionally local overrides. In fact the function.json configuration file is completely optional, as shown in the following example project structure:

Apex project directory structure

To give you an idea of what the functions look like if you’re not familiar with Lambda, your index.js file could be as simple as the following:

console.log(‘start foo’)
exports.handle = function(e, ctx) {
  console.log(‘processing event: %j’, e)
  ctx.succeed({ hello: ‘from foo’ })
}

You can read more about the configuration file properties in the Wiki.

Quick deploys

Apex supports concurrent idempotent deployments, making multi-function deploys very simple and responsive. If the code has previously been deployed it’s simply a no-op.

Configuration is also updated on deploy, so all you need to do is tweak the code or json config, run `apex deploy`, and you’re done.

Here’s what the `apex deploy` command currently looks like, deploying 3 new Node.js functions:

You can of course deploy individual functions by specifying them as arguments, such as `apex deploy foo bar`.

Rollback support

Every time you perform a deploy, a new version is created, and Apex maintains an alias named “current”, pointing to this version. To rollback you may invoke `apex rollback <name>` or optionally provide a version `apex rollback <name> <version>`:

Apex function rollback

Streaming invokes

The Apex invoke command allows you to invoke a function via JSON stdin, and supports streaming.

For example, suppose you have the following upper-casing function:

exports.handle = function(e, ctx) {
  ctx.succeed({ value: e.value.toUpperCase() })
}

And the following request.json file:

{
  “event”: {
    “value”: “Tobi the Ferret”
  }
}

To invoke it just pass the function name to `apex invoke`, and some JSON over stdin:

$ apex invoke upper < request.json
{“value”:”TOBI THE FERRET”}

So what’s this about streaming? Newline delimited JSON can be passed to stdin, so as long as stdin is open `apex invoke` will continuously invoke the command so you can use it in a pipeline. The following example uses phony to generate input:

$ echo -n ‘{ “event”: { “value”: “{{name}}” } }’ | phony | apex invoke upper
{“value”:”BERNEICE LOVE”}
{“value”:”JERRICA MENDOZA”}
{“value”:”VINCENZO DIAZ”}
{“value”:”GINO LOGAN”}
{“value”:”TEREASA RHODES”}
{“value”:”CATARINA GRIFFIN”}

Dry run

Apex supports dry runs of any command manipulating AWS resources. For example, here’s a delete:

Here’s the output when deploying a new function:

And finally here’s some configuration changes to a function:

Deleting functions

Naturally you can delete functions as well. By default we prompt the user for confirmation:

Or force deletion with -y:

Log tailing

Apex supports CloudWatchLog log tailing via `apex logs <name>`. In the future we’ll allow tailing all of your functions at the same time.

CloudWatchLogs has a slight delay of a few seconds, so that’s something to keep in mind. Kinesis would be more ideal here, however CloudWatchLogs handles cases such as uncaught exceptions or panics which manual Kinesis logging would be less ideal for.

To get the log output of a single invocation, you can just pass the -v verbose flag to `apex invoke` (the name of this flag may change):

Runtimes and Golang support

Apex supports multiple runtimes, currently Node.js, Python, and Golang. When a language is not supported by AWS, a Node.js shim is injected into the zip, which feeds the events to your program over stdin, and responses over stdout.

This is completely transparent to the developer, however you can view the zip output via:

$ apex build <name> > out.zip

The apex/apex repository ships with a Golang package which can be used to accept these events over stdin. This has a beneficial side-effect of easy local testing over stdin.

This same concept could be applied to any language which can produce a cross-compiled binary or which compiles down to one of the languages supported by Amazon. Although I can’t build these all by myself, it would be very cool if the community could come together and build the same for C, Rust, Scala, and so on.

We plan on adding hook support and moving the current runtime logic into hooks so that you could transparently compile ES6, CoffeeScript, TypeScript and others, treating them as first-class Lambda runtimes as well. If you’d like access to the Apex org to help contribute please leave a note on https://github.com/apex/apex/issues/83.

Documentation

The final command-line feature is `apex help`, which pulls from out GitHub wiki and provides nice docs in your terminal:

Future

This is definitely just the beginning, but barring some exceptions, I think Lambda and similar will drive a new computing paradigm. At the end of the day, no one really cares about machines, containers, we just care about logic and results. There are certainly limitations now, but over time this kind of platform will become more powerful.

As for the future of Apex itself, we’ll be adding features for managing API Gateway, Lambda input sources and so on to help improve the workflow. Our goal is not to replace existing tools that do a great job for this, such as Terraform, however there are cases where bootstrapping via Apex, and managing via Terraform is appropriate.

There are also several other tools that allow you to manage Lambda functions such as Serverless (which has really awesome gifs!). Hopefully over time we can get a better idea of what works best, and settle on some sort of RFC, at least at the basic project structure level for interoperability.

That’s it for now!