Part uh: what does it take to be a full stack developer?

A Beginners roadmap to becoming a full stack developer on the backend

It’s 2017 — the year of the orange clown — and we are likely to be annihilated for reasons beyond our control by a man who likes to make catastrophic decisions tweeting on a golden toilet.

But let’s tell ourselves that we are ready to become unicorns and ninjas, take a deep breath, and start with a solid checklist of problems that full stack developers are expected to solve on a daily basis on their projects.

A word of warning to the reckless reader about to dive in. This is not a deep dive into programming with code samples. It’s a broad walkthrough. So think of it is as a big superhero movie without the budget for special effects. Ojey? Thanks

A full stack developer is someone who knows how to work on the backend, the front-end and deploy to the server, all while speaking seven ancient dialects seeking money and attention.

This article is only about the first part about full stack requirements for the backend.

Here, at a glance, is a comprehensive 18 point checklist of things you need to know to build a full stack application on the backend:

1. Learn how to Authenticate
2. Build Roles, Permissions and Access Control
3. Learn how to CRUD
4. Learn how to REST
5. Learn to work with forms and state
6. Build an API
7. Build Notifications for Email, SMS and other realtime Webhooks
8. Build Subscriptions and Plans
9. Learn billing integration with payment gateways
10. Handle File uploads
11. Don’t be afraid to work with third party APIs, Frameworks and Packages
12. Work with, build and extend community packages
13. Create an admin interface
14. Manage Caching
15. Think in terms of components
16. Work with a modern version control system for your code
17. Work with the command line
18. Ask good questions on Stack Overflow

1. Learn how to Authenticate

Learn to let new people into your life (and your app)

Most applications need new users and you must design a way of bringing them into your business by registering them, logging them in while also giving them the ability to change or recover their password.

Two ways of doing authentication in a web-based application

There are two workflows you will have to familiarise yourself for this: sessions and tokens. You can also think of these as stateful and stateless respectively.

Let me explain a bit:

With stateful authentication, you log in the user once and then she can navigate to multiple areas within your application and access protected resources (like their bank account transactions and snapchat selfies) without resending the credentials.

With stateless authentication, your users do the same thing but you need to send in the credentials on each HTTP request. This is what you usually do with REST APIs. The gold standard currently for stateless token based authentication is JWT.

Multifactor Authentication(MFA)

One more advanced scenario that I won’t get into (for reasons my doctor told me not to) is Multifactor Authentication(MFA). This basically enhances security for your application by adding “an extra layer of protection on top of your user name and password[aws, google]”

Social Logins with OAuth

Another common scenario that you will be asked to deliver involves integrating social logins so your users can register and login with a single click using their preferred social networks. This way you get (some of) their social data into your application without asking them to manually enter it.

2. Build Roles, Permissions and Access Control

Sometimes you just gotta say no

This part is called authorization or access control (ACL). People often confuse authentication with authorization.

Authentication is how you get a user into your system. Authorization is how you control what your users can do once they are logged in.

This basically means separating your authenticated users into groups and even teams. Then defining admins and super admins for those teams and then allowing those team admins to add normal users to define user permissions.

Confused? Don’t be. Let me elaborate:

This layer is all about granting permissions, maintaining privacy, denying access to certain actions and revoking privileges where required.

You wouldn’t want some user sneaking into some other user’s account and looking at their private photos, right Anthony Weiner? Permissions can thought of simply as this:

User x can do y to z

So that translates to Sharon as an editor can edit Rajendra’s posts.

Let’s break that further down.

• Step One: Define a role for Sharon — that is editor
• Step Two: Define what she can do — her action. Here that is editing
• Step three: Define what she can do it to — here that is posts

So how does it all work? Simple: booleans. You just return true or false from your application depending on who is allowed to do what to whom(?!). So can Sharon edit Rajendra’s posts? Return true(Yes) if she is an editor. Return false(Nope!) and deny her access if she is not.

Huff huff puff. Ok let’s move on

3. Learn how to CRUD

Now you’re cooking — with data 😏

Now you can’t really create users or build a permissions without knowing how Create, Read, Update and Delete resources. This usually means learn how to deal with the persistence — most commonly the database.

But what is a resource? Well, if you’re building a book store, a book is a resource. If you are creating a team, a team itself is a resource and the members within that team are also a resource. And so is every record or account within that system they are trying to control — i.e. maybe a whitepaper, a postcard or the movie they are trying to book.

This is where structuring your data models come in and this is also usually where working with SQL and NOSQL options come in. You will be thinking about:

• How do I create new data?
• How do I edit it?
• How do I update it and delete it when I want?

So here is what CRUD looks like if you working with a framework like Ruby on Rails that gives you an Object Relational Mapping Layer (in rails, its Active Records):

# build a new student record
student = Student.new(:first_name => ‘Ann’, :last_name => ‘Smith’, :birthday => ‘1986–08–11’)
# save the student record you built above
student.save

Also these resources rarely exist in isolation. They most often have some kind of relationship or association with another resource. So let’s take a look at a scenario where you want to save information about the student’s classes, you can reach into the student resource and build a nested resource and save it like this:

student.classes.new(:name => ‘Geometry’)).save

Doesn’t that look simple and read like simple english? If you want to look at what querying, editing, deleting and updating a resource looks like, take a look at this gist

…But you can’t avoid the basics

But things aren’t always that simple and things get hairy in the real world, real fast! So you will need to learn SQL and how relational databases work if you are working with MySql, Sqlite or Postgres. Or how to structure documents and collections if you working with NoSQL .

Note that to do CRUD, you will also need to learn how to validate incoming data and check against permissions before you do anything meaningful with it. Here’s a nice quick overview of CRUD by James Hamann

CRUD

4. Learn how to REST

now back to your scheduled programming

Feeling better? Let’s move on

To setup control to the resources within your applications (like a book or an account), you will need to setup a layer on your server within your application that accepts Requests and delivers Responses. This is where you get to work with Routes and Controllers.

A route looks like this in a typical ruby app:

get '/photos/:id', to: 'photos#show'

This is how a request for a photo comes in from this route and goes to a controller with a method of show. That method then builds and delivers a response, typically in html or in JSON by accessing resources (typically from a model or another API). Your client - remember in this case the user’s browser — accepts this response and delivers the photo to your screen,

Requests can come from multiple sources (we call these sources clients). Most often in a web application, its a from displayed on a user’s browser. But if you are building a server backend for a native mobile application, the client is the app’s api layer that makes GET POST, PUT and DELETE requests from within the native app itself.

BIG DISCLAIMER: Routing and Controllers are not REST in itself. They are parts of implementing REST.

The way you design this system of answering requests and generating responses is by designing a RESTFUL API. You can read more of that here

RESTful API Best Practices and Common Pitfalls

5. Learn to work with forms and state

Forms are the most common way for your users to talk to your application. Forms are the primary gateway through which your users will send some kind of input to your application.

As a backend developer, you need to present forms to interact with resources in your backend. If a user is booking a ticket for a concert, the form is going to look like a seating grid.

This is also a form. Source: http://www.millersymphonyhall.org/ticket-events/buy-tickets/seat-chart/

Once your user starts interacting with your form, here is what you need to do next:

1. Validate user input and show error and success messages based on your application’s rules
2. Change the state of the form based on who the user is and what he is trying to do
3. Allow submission of sanitised and valid user input to your backend for processing

6. Build an API

building an api with Swagger

If you want your application to get real popular you will want to start sharing data with other applications. For instance, you’re a music company, and you want a streaming service like SoundCloud to stream your content. You also want their users to directly purchase your music from their app.

That’s where your API comes in.

API is short for Application Programming Interface. And what that term refers to is an infrastructure that allows access for others to interact with your application.

via tutsplus: https://code.tutsplus.com/tutorials/creating-an-api-centric-web-application--net-23417

APIs are also used to serve multiple clients within your own domain. Above you see an example of an api being the single source of authority for multiple mobile and desktop clients.

Here are the high level steps you take to build an API

1. Create an API server. This involves opening up protected access to the resources you want to deliver. If you have a book store, you will have an API to deliver book titles, pricing and publisher information to other websites and merchants.
2. Secure the server using APP IDs and Secrets (think of these latter as password credentials for other machines and applications).
3. Don’t forget documentation. You can build living documentation that allows other developers to view and interact with responses within

7. Build Notifications for Email, SMS and other realtime Webhooks

Your app doesn’t always have to scream for attention

When someone takes an important action in your application — like subscribing to updates, signing up for a new account or creating a new project — they need to know that their action was successful, or that it failed.

The way you communicate success or failure and help the user move forward in her workflow through your application is through notifications.

• A notification can happen with the application like a message flashing on the screen
• It can happen via email (like a verification email or a receipt delivered after a purchase)
• Or you might need to send an SMS with information for the user to login or complete a transaction
• These days you might even need to deliver notifications as realtime responses to Chatbots living in different platforms on the web like Facebook, Telegram or Slack. These are delivered as responses to their webhook requests hitting your apps in real time.
• Think also about billing receipts and subscription invoices for the things users have purchased from you. Those are also notifications that need to be delivered in a timely cycle

8. Build Subscriptions and Plans

This is where we start talking about bringing paying customers into your ecosystem. To onboard new users, you might need to offer them a free plan with a time limit and then ask them to upgrade and then charge them while doing it.

This might sound like a specialisation, but its a really common requirement. Here you will have to learn how to create tiered plans and then assign certain roles, permissions and privileges to the user subscribing to a certain plan.

It’s also a good idea to learn how to provide dynamic pricing based on configurable features within each plan. Think about examples like “buying” a new server on AWS or DigitalOcean. They let the user choose the RAM, cPU etc.

Here’s a really sweet list by Smashing Magazine ❤️ for best practices to follow while designing plans and pricing:

As a full stack developer, you will be expected to execute most of these features with some help from your designer friends :)

9. Learn billing integration with payment gateways

To actually start charging people money, you will need to process their credit card details using a form on your front-end (usually through something that looks like a shopping cart interface).

During the transaction, here is what needs to happen:

• you make sure the credit details are NOT stored or touch any part of your application and are safely sent to a payment gateway for processing.
• If the transaction is successful, you charge the customer and send a success notification. If not, then you send them a notification and help them try again.

You can also use third party services like Paypal that let you embed their UI in the shape of a form or just a button within your application. The transaction workflow is then partially managed by the provider

Don’t forget the invoices

People need records of their transactions for taxes etc.

• You will need to create an Accounts and Billing section as part of their profile.
• Then make it easy to filter and export their transaction data in PDF invoice form.
• Make sure they can email these invoices directly to their inbox.

There are libraries and third party apis available for generating PDFs in case you don’t have the necessary image libraries installed on your server.

But what does a payment gateway actually do? How does it work?

Let’s break it down:

1. Your checkout form sends details about the purchase to the payment gateway for processing.
2. The payment gateway forwards the transaction data to seller’s bank.
3. The seller’s bank forwards this transaction information to the bank that issued the buyer’s credit card to authorise the transaction.
4. The bank that has originally issued the buyer’s credit card either approves or denies the transaction and sends that information back to the seller’s bank.
5. Ka-ching! If the transaction is approved, the bank will go ahead and deposit funds into your account at a scheduled time.
6. After approval, the third party payment gateway then sends transaction details and response back to your application…and finally…
7. You send a notification to the buyer letting her know if the transaction was approved or denied.

…you know what to do

Still Confused ? Try watching this short video

10. Handle File uploads

Its highly likely that once users start using your application, they will start storing their data inside it. And I don’t mean just information stored in a database. I mean static assets like image files, PDFs and videos.

You will need to:

1. Allow only valid document formats(for audio, video, images and more)
2. Check for size limits allowable by your server and reject huge files
3. Show upload progress information to the user
4. Make sure the upload process is asynchronous and does not lock up the front-end
5. Allow users to mark their uploaded assets as private or public
6. Allow users to download or export uploaded data
7. Create a media library to filter and manage assets (optional)

11. Don’t be afraid to work with third party APIs, Frameworks and Packages

From working with the database to sending emails, your favourite developer community already has all the software available on tap to deliver a credible user experience to your customers.

Ruby, Elixir, PHP and Javascript currently have thousands of problem solving packages that can be configured, customised and attached to your application as modules and plugins.

Getting these packages into your application is as easy as using a single line command like this in your terminal:

gem install devise # brings in authentication in rails
composer install clockwork # debugging and profiling for Laravel
npm install passport # authentication for your express app

So as a full stack developer, you have to learn how to cook with the best ingredients — not grow dank weeds in your own backyard.

If you don’t like the idea of adopting third party/open source code into your ecosystem, you’re going to be working with low level problems like sessions, hashing and csrf protection to build even the most basic functionality instead of focusing on high level ideas that have to do with what is unique about your application.

All frameworks are not created equal

Also most frameworks are not as monolithic as you think. Most of them are composed of modular packages themselves that can be swapped in or out whenever you feel the need for fresh alternatives.

Remember: you can evaluate a package’s source code quality by

• checking out their build badge
• the date of last commit
• their passing test suites
• popularity(by star count)
• number of closed issues and open pull requests

…and other code quality metrics now available for most popular open source tools.

Choose your own box (wisely)

Sometimes I find it useful to classify frameworks I use as black box, white box and grey box according to their level of source code readability. This means there are certain cases where choosing a framework where you understand what is happening under the hood and that gives you a better sense of control over the inner workings of your own application.

This kind of classification, I must admit, has a subjective bent. So corresponding to my own maturity and sensibilities as a programmer, I would personally classify Ruby on Rails (Ruby) to be a black box framework, Express JS (Node JS) as white box and Laravel (PHP) as a grey box framework.

12. Work with, build and extend community packages

There are times where existing packages just don’t do what you want or exactly how you want to do it. Imagine a custom login scenario where you automatically log people into your system as guests and then further down the line, register them as signed up users. Stripe does this beautifully.

Resist the temptation here to throw away the whole authentication package. Remember, most open source packages have some kind of customisation hook built in that will allow you the specialised behaviour you seek.

So learn how to read the source code (Luke).

Some Pro tips

• Learn to read and search through the documentation before you start asking questions.
• Go to the package’s repo (the official location of the source code on Github or Bitbucket) and look in open and closed issues and you will probably end up digging out a pull request that closely resembles the functionality you are looking for. Better yet, write the extension code yourself and send the author a pull request. If it works out, you will have contributed to the open source community. Yippee 🙌
• If your pull request gets rejected, just use your forked version of the original repo.
• There are times, however, where you might just have to write your own feature from scratch. But make sure to create it with a more general purpose in mind so everyone else in the community can use it too.

13. Create an admin interface

source: https://almsaeedstudio.com/themes/AdminLTE/index.html

As soon as your application is out, your client is going to want to control it, configure it and customise it on almost a daily basis. The best way to keep your clients from destroying your (well, technically their) software is to give them controlled access through a pretty and easy to use admin panel.

Creating routes and resources for your admin panel is not as big a deal as it seems. You get to reuse most of that functionality across other parts of your app anyway.

Here are some characteristics to look out for in a great admin panel:

1. Secure and Protected against external attacks
2. Configured correctly for the right access level permissions (see no 2 above)
3. Visually comforting and easy to use. No one likes to use ugly complicated stuff
4. Focused on top most commonly used tasks
5. Can integrate external components on demand
6. Is built with replaceable components

14. Manage Caching

Beautiful and usable front-end interfaces are often made up of complex, multi-tiered and nested data. Getting that heavy dynamic backend content rendered on the front-end can be expensive and slow.

Without caching, your database will be thrashed and forced to perform complicated expensive queries to render what seems to be basically the same content.

Imagine a middle tier somewhere in your application stack that stores data in a static package and just renders it out on demand without even touching the expensive hardware like a database.

That is the cache working for you in a nutshell.

Redis, Memcached (pronunciation: mem-cash-dee) and Varnish are three technologies most frequently used to create caching solutions for your web application backend.

How does caching work in a web application?

Here I am going to shamelessly take the easy way out and just quote Zeeshan Ashraf’s answer in full on quora here (I hope he doesn’t call his lawyers):

“There are many caching mechanisms used in web applications. They can be broadly categorized in the following types.

• Content Caching.
• Fragment Caching.
• Data Caching.
• Content Caching

If the content of a particular URL does not change frequently then there is no need to generate it dynamically every time. Full HTML can be cached in memory.

An example can be an article page. An article does not change frequently. Content for each article can be generated dynamically on first request and cached. Subsequent requests can be served through cached HTML.

Fragment Caching

Sometimes whole page can not be cached. There is some content on page which is dynamic but there are parts which do not change frequently. HTML for these static parts can be generated on first request and then served from cache for subsequent requests

Data Caching

Web applications are generally backed by Database. It’s usually expensive in terms of performance to fetch data from the database each time. Frequently used data objects can be cached in application memory or in other in-memory data stores like redis and mem-cached.”

Zeeshan Ashraf

How does caching work in a web application?

15. Think in terms of components

I know “thinking in components” is more of a front-end mantra but as a full stack backend developer, you’re going to want to separate and modularise your code into nice clean components if you want to survive it beyond the first two weeks.

ceci n’est pas un component

Now that applies to both your page templates and your backend services. On the front end here is how one of my own favourite projects is organised:

{% include '_components/global/_header' %}
{% include '_components/services/_masthead' %}
{% include '_components/services/_svc-stats_blok' %}
{# include '_components/services/_client-list' #}
{% include '_components/services/_sol-portfolio' with { 'brandColor': BrCore } %}
{% include '_components/services/_customers-blok' with { 'brandColor': BrCore, 'topConnector': true } %}
{% include '_components/services/_why-aranca-c' %}
{% include '_components/services/_svc_box_grid' with {'data': entry.experienceRichtext, 'brandColor': BrCore } %}
{# include '_components/services/_featured-case-study' #}
{% if entry.tfilterRichtext|length %}
  {% include '_components/services/_table-filter' %}
{% endif %}
{% include '_components/services/_full-cta-a' %}
{# include '_components/services/_kc-content' #}
{% include '_components/global/_footer' %}

The above code comes from a twig template which I can now easily change and maintain without the entire page breaking down on me just because one of the component stops working somewhere.

Now if your authentication code is all tangled up in your controller layer, you could do far better by separating it out into a service component.

16. Working with a modern version control system for your code

All hail the Octocat

Yep. I am talking mostly about using Git with Github here. Now for beginners, using Git seems superfluous and the advantages are not always obvious.

Now think about these all too common situations you will encounter while coding:

• If you want to work with other developers on the same project, you’re going to be working with common set of files. How exactly do you plan to avoid conflicts and overwrites in your code base?
• What if you wanted to go back in your code’s history and find out how you implemented a certain feature before you rolled it back in production?
• What if you want to roll back to the last snapshot of your code before your application crashed?
• What if you want to share your source code? Will you zip it and send it by email every time you deploy a new feature?
• How will you deploy using FTP? How will you keep track of files that have changed and those which haven’t?

The only reasonable answer here is using source control.

But I won’t go into more detail as Abhishek Joshi has done a great job here:

An Intro to Git and GitHub

17. Work with the command line

There is a limit to clicking buttons on a user interface to get shit done. The deep and strong control you need over your application and its environment can sometimes only be delivered through the command line.

For working in ruby and nodejs environments, its not an optional skill-set. The command line will help you bring me software packages into your application’s universe, run your tests and even deploy to a remote server somewhere deep in the amazonian forest.

Once in the command line, you will feel like a real programmer, staring at long red characters of error spit across 17 lines of barely readable standard output while somewhere deep in your head you are actually taking down an electric grid controlled by a totalitarian corporation bent on destroying democracy.

To get started with traversing these fantasies and more, it’s hard to recommend a better book than this one by Mark Bates Conquering the Command Line:

Conquering the Command Line

18. Ask good questions on Stack Overflow

There are times as a newbie developer where you will feel frustrated, neglected or ridiculed on Stack Overflow for asking a noob question. That’s because those people are assholes :P

Did I say ask questions? I meant copy and paste 😂😂😂

But seriously — there are guidelines for asking questions and if you follow them, the results can range from pleasing to feeling gratuitously rewarded.

They are really well documented right here. Take 10 minutes to read this.

How do I ask a good question? - Help Center

So these, in my opinion, are the must have skill-sets for a developer to qualify as full-stack on the backend.

Yes! The next two parts are involve some of the ridiculous magic you need on the front-end and server side to make your full stack app rattle and hum.

Did I miss anything in this article? Does this seem too much? Tell me on twitter or talk to me in the comments. Please bookmark this article, recommend it and then immediately forget about it like everyone else is supposed to on @Medium.