Internal Minions of the Back-end World

You have an idea, say an application that can turnaround billions. You plan a basic minimalistic prototype. And as you think further, you see data and with data comes a world of Back-end, full of minion-like technologies working together and building a strong efficient structure for your back-end support. If you plan to make something more viz. a product out of your prototype, add these minions and they will fix any and all problems involved.

There are a plethora of options lying around for each such fix. You need to know them and narrow down to the one that best suits your/ your team’s comfortability and the present trend. However, also bear in mind words such as Scalability, Flexibility, Budget, Accessibility, Productivity, Licensing or Maintainability. Anyhow, knowing the kind of technologies that exist these days, why they are used and what they offer is very useful. We attempt here to do the same and go through a few such technologies in a formulated categorical manner.

Server-Side Scripting Language

A scripting language is like some magic wand. It will be the central object that creates wonders. It will make the core of the application. In plain sensible words, it will be the primary language to code any and all features of an application. It will get you started with the first draft. Perhaps, at this point, you may have to pick a framework in your language of choice. Below, is a list of few such Scripting languages:

  • PHP, As they say, “The fast, flexible and pragmatic”. Made Facebook
  • Python, Clean, high-performing, flexible and elegant. Made Instagram
  • Ruby on Rails, Same as above yet different. Made Twitter
  • JavaScript(Node.js), Newly minted, All-in-one. Made Medium
  • Java, ASP.NET, Classical yet fashionable. Made millions.
  • And watch out for the young lads. Erlang, Scala, Haskell andColdFusion! Making their way in with the next cool things.

You can merrily pick any of them. Perhaps make sure of the fact that it will somehow be quick for you to tweak in there in the days to come. But even otherwise, almost anything can be fixed some way or another.

To know in detail see — What Technologies should my startup choose?

Database and Storage

A scripting language is just a tool. Great applications are a work of the knowledge that is put behind. Data drives a web application all along and to you it’s worth many. Database management systems traditionally used wererelational. But with the introduction of NoSQL and NewSQL we have much more paths to diverge into.

  • Relational Databases, ACID compliant; always data, security first. — Oracle or SQL Server, Reliable and sophisticated. But, did you check your budget? Are you okay with Legal Issues?
    If not, go for Open Source solutions like all below! — MySQL, Lighter, popular, serves almost all purposes of a database. But if the need is for more.. — PostgreSQL, More sophisticated yet efficient solution. Open source. Comparatively complicated. But what if your concern is scalability?
  • NoSQL Databases, designed to give you flexibility and scalability. No ACID compliancy. Uses distributed strategies to provide accessibility and make development agile. — Projects from Apache’s Hadoop Cluster (HBase, Cassandra, Hive with HDFS, MapReduce). Scalable, efficient, highly available solution with reduced hardware dependencies. Very useful and bit complex. — MongoDB, memchached, Redis, Other simpler, popular choices. MongoDB is also member of the MEAN stack making it widely used. NoSQL is a popular choice among all software giants lately. But what if the need is for a comprehensive solution i.e. flexibility as well as high data priority?
  • At last, enter NewSQL Databases, an amalgamation. RDBMS + NoSQL.
    E.g. NuoDB, MemSQL etc. Yet to put their mark in the Database world.

In addition to structured data needing databases, data in the form of files can be stored either at the server itself or at dedicated storages such as AWS S3, Google Cloud Storage, Eucalyptus Walrus, OpenStack Swift, Microsoft Azure or Riak CS. The choice depends. If you are already using some web service, it will be logical to just go with what comes along. You may still want to look into what you are missing.

How do I choose between RDBMS and NoSQL? .. take an insight

Web Server and hosting

You need a server that serves your awesome application to its zillion users. A web server interacts with the scripts it has and hosts all the required content for a secure delivery to the user’s browser. The user then fiddles with it and sends you back signals via this very server. A server is rightly a high performance machine, but there is also a server application that runs in there and provides you with these features. That application is what you need to choose wisely.

  1. Apache — With LAMP as a popular choice of web stack, Apache holds the highest market share among web servers globally.
  2. Microsoft’s IIS — Another dominant player. For the Windows lovers.
  3. Nginx, Lighttpd — Young and trending. Does high performance and low memory usage.
  4. Tomcat, Netty, Unicorn, Passenger — Upcoming competitors.

Subsequently, your web host (the server OS) is usually dependent on the technologies you choose. If it is majorly all Microsoft’s, Windows should be your choice of OS. However, in other cases you are likely to go with Linux systems (Ubuntu, Red Hat, Debian) due to their developer friendly make.

Top Enterprises that use Linux

Webhosting : Linux vs Windows …

Content Delivery Networks and Caching — The theatres

We can compare this to a TV channel. Like a TV channel broadcasts the content to your TV set at home via their vast networks, you would want to put your application’s content on at least one such channel. Of course, not a TV channel but a server channel. Once you are done with building your platform, you should start looking for such channels — called Content Delivery Networks — which effectively increase the rate at which the content reaches the end-user.

Leveraging CDN(s) is a really clever move. By caching your files on their networks, a user gets his request responded fastly by the nearest resourceful server of the network. In general, caching is required at every level of your entire architecture for smooth delivery of content to the end-user, hence, reducing load on the original server. The available options for CDN are majorly commercial and include Amazon CloudFront, Akamai, EdgeCastand the mentioned before, Fastly.

Load Balancers and ADC

Load balancing concerns with your scalability model. Load balancers were designed to provide an interface over your internal network(cluster) of servers to efficiently manage the load and distribute it across different machines, say server instances efficiently. Once the balancer receives a request, it does some algorithmic math to choose an instance for the task, sends it the task, waits for the processing, verifies the result and pushes it back to the client. If you have a server cluster, you can write a balancer yourself using any known scheduling technique and as per the “scale out” and/or “scale up” model you prefer. For example, Facebook uses its own tech called AutoScale for this purpose. For your specific usage, you can explore following softwares that provide load balancing, namely,

  1. LVS, — Easy to use, based on Virtual Servers. Used by Wikipedia
  2. HA-Proxy, — “Fast and reliable”, Used by Reddit, Tumblr, Twitter
  3. Nginx — Not primarily a balancer, but anyways highly useful
  4. Varnish — Primarily as web accelerator, used by Facebook, Vimeo

Application Delivery Controllers are composed of load balancers and additional features like Firewall, Caching, Authentication, Authorization and Accounting. They can be explored too.

For more — Explore this dedicated list of Load Balancers

Cloud hosting and Virtual Private Servers

Virtual Private Servers are efficient servers that provide you with instances that run OSs virtually using a utility called Hypervisor. These instances are then used as servers for your application. Cloud hosting is providing cloud based web hosting using such virtual as well as other actual server clusters. This will be useful when you consider scalability and performance. At some point in time you would like to switch to them if you are wary of the infrastructural intricacies like everybody else. There are a lot of services available for such hosting. These can optionally be called Cloud Computing IaaS solutions. Examples are Amazon Web Services (EC2), Google Cloud Services (GCE), Microsoft Azure (VM), Rackspace Open Cloud, IBM Smart Cloud etc. Additionally, these services provide you with lots of other cloud solutions such as load balancers or databases, useful for your web application.

Know the tech stacks of your favourite websites.

Logging, Monitoring and Alerting

You require logging at each level too. This gives an insight into each part of your web architecture and is useful in drawing statistics and subsequently, estimating the vulnerabilities and implementing proper architectural support. This will be handy when variety of technologies are jointly working and you require the structure to be robust altogether. We have targeted solutions available for this part too. Examples : Facebook Scribe, Apache Flume.

This is not it. This was just a basic view to the web back-end architecture. Unfortunately, there goes a lot more in there; a lot more technologies to support and a lot more complexities to handle for real. Things like Firewalls, Source Code Management (Git, Capistrano, Jenkins), Analytics and Front-end stack are still missing. Anyhow, I have only so much to say for now. With these whole lot of technologies one would just wish to have a single utility that takes care of everything across everywhere. Well, it is simply unrealisable.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.