Threads and Connections in Backend Applications

Building a Backend application requires a communication protocol, a port to bind to, and process to serve requests and produce responses. When using TCP in particular as the transport protocol, a stateful connection is created for the client who wishes to connect to the backend. Before this can happen the backend application has to actively accept a connection, otherwise connections will remain in the operating system backlog buffer which will eventually fill up. Reading the TCP stream from a connection is also the job of the backend application, this moves raw stream bytes from the OS buffer up to the backend application.

In this post I explore five architecture patterns on how threads and connections are managed. Before I do that, let us clear up some definitions so we are on the same page.

Listener — When the backend applications listens on a particular IP and port it creates a socket. A socket is not a connection but a place to connect to. Just like a wall socket, where you can plug in connections. Many connections can be connected to the socket. The listener is the process where the socket lives.

Acceptor — With a socket in hand, the backend can call the OS function accept passing this socket to accept any available connections on this socket. The accept function returns a file descriptor representing the connection. Connections have to be actively accepted by the application in order to serve clients. Otherwise the connections will remain in the OS accept queue unused. The…