Building Distributed Systems With The Right Tools: Akka Looks Promising

Modern day developers are building complex applications that span multiple machines. As a result, availability, scalability, and fault tolerance are important considerations that must be addressed if we are to successfully meet the needs of the business.



As developers building distributed systems, then, being aware of concepts and tools that help in dealing with these considerations is not just important — but allows us to make a significant difference to the success of the projects we work on.



One emerging tool is Akka and it’s clustering facilities. Shortly I’ll show a few concepts to get your mind thinking about where you could apply tools like Akka, but I’ll also show a few code samples to emphasise that these benefits are very accessible.





Code sample for this post is on github.

Why Should I Care About Akka?

Let’s start with a problem… We’re building a holidays aggregration and disitribution platform. What our system does is fetch data from 200 different package providers, and distribute it to over 50 clients via ftp. This is a continuous process.



Competition in this market is fierce and clients want holidays and upto date availability in their systems as fast as possible — there’s a lot of money to be made on last-minute deals, and a lot of money to be lost in selling holiday’s that have already been sold elsewhere.



One key feature then is that the system needs to always be running — it needs high availability. Another important feature is performance — if this is to be maintained as the system grows with new providers and clients it needs to be scalable.



Just think to yourself now, how would you achieve this with the technologies you currently work with? I can’t think of too many things in the .NET world that would guide me towards highly-available, scalable applications, out of the box. There would be a lot of home-rolled infrastructure, and careful designing for scalability I suspect.

Akka Wants to Help You Solve These Problems ‘Easily’

Using Akka you don’t call methods — you send messages. This is because the programming model makes the assumption that you are building distributed, asynchronous applications. It’s just a bonus if a message gets sent and handled on the same machine.



This arises from the fact that the framework is engineered, fundamentally, to guide you into creating highly-available, scalable, fault-tolerant distributed applications…. There is no home-rolled infrastructure (you can add small bits and pieces if you need to).



Instead, with Akka you mostly focus on business logic as message flows. Check out the docs or pick up a book if you want to learn about the enabling concepts like supervision.

But How Much Effort is Required?

Let’s walk through the steps and have a nosey at some code samples that demonstrate how to solve the availability problem faced by the holiday package system.

1. Create an empty Akka application
2. Add about 20 lines of config that say “this is a clustered system, here are the addresses of the other nodes”
3. Implement the business logic using actors and messages
4. Create a supervisor that watches the business logic and makes sure it is always running on one machine — even if the machine dies, it has to restart it on another one
5. Create a distribution package
6. To scale out just add more nodes and update the configs to point to the new machine(s)

Creating an empty application is simple — Typesafe has an activator application that does it all for you.



Sample configs

On top of the default configuration, you just need the following extras/replacements to make your application clustered. Obviously you put in the correct host and port.



https://gist.github.com/NTCoding/5840489.jsNotice the seed nodes? Those are the other nodes in the cluster. In this example there are 3 nodes on my machine, each on a different port.



Business Logic

Here is the indexer actor. It sends a message to the data provider asking for a batch of data, and sends another message to the publisher telling it to publish that data.



https://gist.github.com/NTCoding/5840521.jsEach case statement is a block of code that handles a message — e.g. ‘case “start_indexing”’, means when this actor receives the message “start_indexing”, execute the following code block.



Exclamations marks are the symbols used to send messages. So ‘self ! “next_batch”’ means the actor is sending the “next_batch” message to itself.

Enabling higher availability

This next block shows the creation of the actor system. From that system, a singleton manager is created that ensures one instance of the “SingleCoreIndexer” is created and is always running — even if the node it is on blows up into a million pieces.



https://gist.github.com/NTCoding/5841418.js

Creating a distribution

Go into the sbt command prompt. Type “dist”. Have a coffee (optional).



Testing it out

And that really is all there is to it. So, did it work?



In this example I put a bug in the system that caused it to completely shutdown the JVM and take that node off-line after it has sent 2000 messages.



https://gist.github.com/NTCoding/5842033.js

Have a look at the logging output below and see if you are impressed. The comments above each block of logging tell the story of what’s happening.

Akka’s Trade-offs

These lists are not exhaustive or even close.

What’s to love?

Apart from working with a truly fantastic language, Scala, the rate of innovations around Akka are really impressive. Hence, there is a really positive community. This monitoring application looks especially useful.



I am super impressed by the testability of Akka. There are a set of high-level testing features that let you spin up multiple instances of the JVM to test interaction within a cluster. Additionally, the unit testing features allow you test message interactions effectively and easily.



Some big companies are using it in production and very happy with it.

What to be cautious of?

First you need to learn Scala — an initial investment. Plus you’ll have to train devs you hire, because Scala is a pretty rare skill.

Then you need to get used to the actor model — sending messages and not calling methods. If you call a method that doesn’t exist in a statically-typed language, the compiler laughs at you. Send a message to an actor in Scala and you have to hope the actor knows about it, otherwise you will only find out at runtime. Compiler is helpless.

With the actor model, and distributed systems a common enemy is holding state. Using the actor model usually involves passing state around all the time. It’s a negative because it’s different, and you have to learn to think differently. Also, the code will often be more complicated.



Akka’s a fairly new technology as well.