The New Stack (DAFT)

While answering a question over on Quora about what a full-stack developer should know in 2015, I ended up defining what “the stack” actually is in 2015. I thought it would be good to share my thoughts in an official post and dig into it a bit deeper.

When I hear full stack, I think of the original “stack”: LAMP — Linux, Apache web server, MySQL and PHP. This was THE defacto stack not too long ago, but then along came Nginx to take a slice out of Apache, Oracle bought MySQL which scared everyone into using other databases like PostgreSQL, and Ruby on Rails, Node.js, etc., took over PHP. Linux is the only thing still left standing, but it’s also the part of the stack you need to know the least about nowadays (more on this later).

The Old Stack

Disregarding the exact pieces of software that made up the LAMP stack, the generalized components in the stack, from the bottom up, were:

• Web App Framework/Server/Language
• Database
• Web Server
• Operating System

And you had to know every layer in the stack.

Today, an additional layer has been added, one of the layers’ purpose has changed and what a developer should focus on in the stack have shifted up a couple of notches. Let’s look at the new stack.

The New Stack — DAFT

The stack has evolved in the last few years for various reasons such as smart phones forcing us to build multiple clients (a web app isn’t good enough anymore), everyone moving to the cloud and interconnecting services (microservices), and the maintenance nightmares of the monolithic app, etc.

So I present to you, from the bottom up, the new stack.

• Tasks — Asynchronous, event-driven, background processing.
• Frontend — Web, mobile and other device clients. The synchronous user facing UIs.
• APIs
• Database

The App Framework layer which used to be where the logic lived along with generating the user interface is now an API that just contains the logic. The additional layer is the client side app which talks to the API (JavaScript, Android, iOS, etc). And the has shifted to the database layer and up since the OS and web server don’t require much thought anymore.

The Database

I won’t get into the database layer as there are a ton of great options and the options are pretty well known. This hasn’t changed much since the old stack other than some newer db options like NoSQL have come out (Mongo, Cassandra) and even some hosted ones like DynamoDB. The best fit will depend on your project.

API’s

This is where things have really changed and there were a few things that brought on this change: the awesome power of JavaScript on the front end (Gmail for example), the growth of API’s, and smart phones.

Once smart phones came out, you had to build API’s that matched the functions of your web application so that your Android and iOS apps could talk to your app. So why not do the same for your web app? There are some awesome new client side app frameworks like React (Facebook) and Polymer (Google) to help build your web apps. These frameworks are 100% client side and talk to API’s for data. So instead of building a web app in PHP, Ruby on Rails or whatever you might have used a couple years ago, then also building an API for phones to talk, you just build the API and that’s all that runs on your servers.

The best part about this is that it keeps the server side nice and simple and you can use fast, efficient technology to make the API, rather than big bloated frameworks like Rails. If you are using Ruby, you can use something simple like Sinatra that is perfect for building API’s. Or better yet, use a better performing language like Go (golang) since you don’t need all the helpers that big frameworks provide to generate views.

The new hot term of the year is Microservices and that is this layer. Turn all your data into API’s and you have your microservices architecture.

Frontend —Web, mobile and other device clients

The heavy lifting has been moved to the client. We’re back to a world of fat clients which is both good and bad. The good is that the server side is much simpler and building an API first makes it easier to build other things on top of it. The bad news is that the clients are much more complicated and to make it worse, you’ll probably have to build three of them. One for the browser, one for Android and one for iOS. Not fun, but hopefully this will change with the new client side web frameworks if they can start to feel native on the phones.

Tasks — background, batch, event-driven and scheduled tasks

As your API’s are the synchronous parts to your application to respond to requests from users/clients/devices, tasks (or jobs) are all the things that happen outside of the request/response loop. Data processing, image processing, batch processing, notifications and all the things that take more than a fraction of a second to complete.

Chris Wanstrath stated way back in 2009 that GitHub was more than 50% background work. I’d venture to say that most applications have a large percentage of their work being done in the background and if you don’t, you’re probably doing something wrong.

The basics are a message queue to queue up tasks/jobs and a task processing framework to help manage the execution of the tasks (a message on the queue is a task to be processed). There are plenty of open source tools in every language to help you here as well as some newer hosted/cloud services.

What should a full-stack developer know in 2015?

To conclude this post, here’s some thoughts on what you should know or learn in 2015.

1. At least one good database — (tip: make it easy on yourself and use a hosted one to avoid having to do the ops/management of it, that’s the hardest part of a database (Heroku Postgres, Mongolab, DynamoDB, Google Cloud Datastore)).
2. Know how to build REST API’s — accept JSON and spit out JSON — (tip: since you don’t need a full framework here, use something fast like The Go Programming Language so you won’t need to port it to something faster later).
3. Know how to build a client(s) — (tip: if you have to choose one, choose one of the new web technologies that looks good on the desktop, tablets and phones, like Polymer Project , which is a Web Components implementation. There’s nothing worse than only supporting one phone.)
4. Know how to use a task/job queue so you can run code in the background. Respond uber fast in your API and let your background jobs do the slow work.
5. Every developer needs to know HTML/JavaScript, those are the two that you simply can’t avoid, you’ll use them in some way or another at some point.