How Web Apps Remember Who You Are
An overview of “state”, HTTP protocol, web cookies, and session identifiers
“I did it again!”
Here I am logging in to my Google account for the third time today. My affinity for a tidy desktop (or for procrastination?) led me to close my browser window where my email inbox had been open. Now I have get an authentication code from my phone, and spend an extra 15 seconds logging in again. If this experience is relatable to you, it might be because you are used to having “stateful” interactions with applications on the internet.
Laying the Groundwork
As we engage with a website, we expect the website to keep track of our interactions to some extent. This allows us to make progress in the task we are trying to accomplish on the site. For instance, before I can read the contents of my inbox, I have to log in to my Google account. After doing so, I can browse my inbox as long as I please. Google's servers remember that I am logged in the entire time. Without this sense of continuity, it would be challenging to do any complex task on the internet.
A series of stateful interactions like this is called a “session”. When the user logs in, they begin a session. From then on, the website remembers that the user is logged in. It does not require them to continuously re-authenticate when they navigate around the site or begin writing a new email. The session ends when the user closes their browser or logs out. This behavior is typical on the web. What is interesting however, is that programmers have to work hard to create this functionality in their apps. The sense of continuity that a session provides is not an accurate representation of how the internet actually works under the hood.
Sessions are a feature that is carefully and intentionally designed on top of the standard set of rules that control internet communication. These sets of rules are called “protocols”. Multiple protocols, each with its own limited responsibilities, are layered on top of one another to create the internet infrastructure. The top-most protocol is HTTP, and it establishes the rules for how web applications pass information back and forth between each other. HTTP requires that client applications (like your web browser) make requests to servers (computers that “serve” or host the site) in order to retrieve or modify data on the site. Servers interpret these requests, and then formulate a response which they send back to the client.
While not directly related to the topic of sessions, a little extra background on HTTP will be helpful later on. HTTP requests and responses are human-readable, plain-text messages. The HTTP protocol prescribes a consistent structure for these messages, so that applications can parse them and extract whatever information they need. Requests and responses can contain three sections:
- A request line (for requests) or a status line (for responses)
- A set of headers containing meta-data in the form of key-value pairs
- The main body of the message
Sections 2. and 3. may be omitted in certain contexts.
Why HTTP is stateless
Let’s talk a little more about this concept of “state”. State refers to the context surrounding an individual interaction with the web. Returning to the email example we talked about before, once the user is logged in, all subsequent interactions with the email server happen in the context of being logged in. Being logged in is part of the current state of the user’s interaction with the site. From the user’s perspective, it appears that the server hosting the site remembers that they are logged in throughout the rest of the session. But that is not actually how it works. HTTP is a “stateless” protocol. By stateless, I mean that each and every request/response pair is treated in complete isolation from its predecessors. The server does not “remember” any information about the user’s progress on a site between requests. The entire context surrounding each request must be reproducible by that single request. While this presents a clear challenge to creating sessions in web applications, the fact that HTTP is stateless provides numerous advantages as well.
Consider for a moment how much extra work the server would have to do to actively keep track of session information between request/response cycles. When a server is handling a request, system resources are taken up to process the request, execute application logic, and formulate a response. For most requests, the server can accomplish all of this in a very short amount of time. If the server held on to the current application state for a given client, the server would spend most of its time idly waiting for the client to make another request. This is not a good use of resources of you are a web server (I get annoyed when it takes 15 seconds to log in…try 15 billion clock cycles!). By relieving the server of any responsibility for tracking session data, it is free to handle a new request as soon as it finishes sending the latest response.
A closely related advantage is that the server does not have any extra work to do when a breakdown in communication occurs. Let’s say that a client stops sending requests, or a request is lost on the way to server. The server does not have to recognize this, and allocate resources to clean up lingering session information. After each request/response cycle, the server “forgets” everything related to that interaction and moves on with the next request. If a client stops sending requests in the middle of a session, the server is none the wiser. It only cares about handling each individual request that it receives.
Probably the most important benefit of the stateless model is that it makes web applications more scalable. Since each request/response pair is handled independently, it usually does not matter whether multiple requests from one client are handled by the same physical server. Any server which is hosting the application will do. As a web app grows in popularity, more servers can be added, and a load balancer can be used to distribute requests evenly between all of these servers. This means that throughout the duration of a session, more than one server may handle the requests from a single client (sticky sessions are an exception to this, but will not be discussed here).
These advantages come with trade-offs. In order to have sessions, we need to work around the statelessness of HTTP. There must be a way to persist the application’s state across multiple request/response cycles. Although there are many solutions to this problem being used today, we will drill down on one in particular — cookies and session identifiers.
Cookies
As mentioned above, since the server does not keep track of session data between requests, each request from a client must be able to inform the server of the current state. A single request may arrive at any physical server, and that server should be able to formulate a response that accounts for the entire context of the conversation. Returning to our Google account example, let us suppose that I have already logged in to my account. My browser should be able to make requests to the server, and the server should behave like it remembers that I am logged in. Here is how this works: When the client makes a request that affects the state of the conversation, the server sends some information back to the client that is related to that state. This information is stored by the client. When the client sends the next request, it includes this session information in the request. The purpose is to inform the server of the current state. The persistent information that is passed back and forth is called an HTTP cookie.
A cookie is simply a small piece of data that the server sends to be stored on the client-side. This data is structured as a key-value pair, where the key is the name of the cookie, and the value is the data that the cookie contains. A cookie can also have additional information appended to it in the form of attributes. These attributes provide meta-data about the cookie. Here is what a cookie may look like:
lang=en-US; Expires=Mon, 03 Jun 2022 13:38:23 GMT
Lets break it down: lang is the name of the cookie, and en-US is the primary information this cookie is communicating. The data that comes after the semicolon is one attribute. This particular attribute specifies that the cookie is only valid until the given timestamp. There are other attributes for other purposes; a noteworthy example is the Secure attribute, which prevents the cookie from being sent in an HTTP request unless the request is encrypted.
Not all cookies are used to managing session data. The cookie shown above is an example of a persistent cookie, because it has an Expires attribute attached to it. When persistent cookies are send to the client, the client writes them to long-term storage where they remain until they expire, or are deleted for some other reason. Session cookies do not have an Expires attribute, and are only stored in memory. This causes them to be deleted when the session ends or the client application is terminated. That is why I am forced to login to my Google account again after I accidentally close my browser window.
How exactly do the client and server pass cookies back in forth to each other? Recall that HTTP messages have three main sections, one of which is a list of header fields. When a server wants to send a cookie to a client, it adds it to the HTTP response as a Set-Cookie: header. If the server needs to send more than one cookie, it uses an additional Set-Cookie: header for each cookie. When the response arrives at the client, the client recognizes the Set-Cookie: header and stores the cookie locally. When the client sends the next request to the server, it uses a Cookie: header to send the cookie. If more than one cookie needs to be sent, a single Cookie: header is used and the cookies are separated by semicolons.
Passing session information back and forth using cookies allows the session state to be persisted across multiple requests/response cycles. This strategy does have some vulnerabilities, however. Cookies can only be used to store small amounts of data. Passing large amounts of session data back and forth during every request/response cycle would significantly increase traffic on the network. This seems unnecessary because some of this information will remain unchanged throughout the duration of the session. It would be inefficient to repeatedly pass the same information back and forth. Another issue with passing the session data in a cookie is that the data may be vulnerable if a third party intercepts the HTTP message. Many servers work around these problems by using something called a session identifier.
Session Identifiers
A session identifier (SID) is a unique, randomly generated string of characters which the server ties to a particular session. Instead of sending all of the session data to the client in a cookie, the server can store the session data in a database, and use the SID as a key to retrieve that session data at a later time. The server puts the SID in a cookie and sends it to the client. Every time the server receives a new request, it checks it for an SID. If there is one, the server uses it to fetch the session data, and formulate an HTTP response based on that. Here is an example of a cookie with an SID: SID=34Dsrt546DFcc6
Using an SID is more efficient in terms of data transmission, because the session data is stored on the server-side in a database instead of being sent to the client over the network. But how about security? If a third party has access to an HTTP message, can’t they still use the SID to access sensitive information? Web apps have multiple strategies to combat this. One is setting up sessions to expire after an interval of time. When the SID expires, the user is prompted to enter their credentials again, and the session data is assigned to a new SID. This security measure limits the amount of time that an attacker may have access to a stolen session. Another countermeasure is to require the user to re-authenticate when they are doing certain actions, such as editing account information or viewing payment details.
Cookies and session identifiers are a clever way to work around the statelessness of HTTP. They allow the server to focus solely on handling individual requests, while providing a way for session information to be persisted across request/response cycles. All of these technologies work together to make web applications secure and efficient, while at the same time promoting a seamless user experience. There are many other, more advanced methods for simulating state in a web app, but cookies and SID’s remain a foundational strategy for this purpose.
I wrote this article as a means to “tie up” a few concepts that I have learned over the past several months as part of my studies at Launch School. It is representative of my current understanding — my mental model — of these concepts. If you, the reader, notice any flawed information in this article, please reach out to me. I would love the opportunity to correct and refine my understanding of these topics, as well as this article.
