A Background on Debug Headers

In a previous post, I discussed how to read the basic HTTP headers that may return with your HTTP response and how these impact caching. These are just the first building blocks needed in order to better understand where and how your objects are being cached within the disparate world of Content Delivery Networks. The true key to understand what is happening within these CDNs to your content is with Debug Headers.

These handy tools are in place in some way, shape or form on each Content Delivery Network that you may leverage. While some of these debug headers are easily published on every request, certain CDNs will only append these debug headers with the presence of a specific request header. For even more security, CDNs sometimes even require additional request headers and custom keys to be passed with a request in order to receive the appropriate debug headers.

While ultimately receiving these debug headers are relatively straightforward, the actual interpretation of the information return in the response can be quite challenging. In some instances, you’ll need to refer to detailed CDN guides that break down what each and every header means. In others, single characters may provide a wealth of data that can only be interpreted with internet specifications.

In this post, I’m going to explore the general benefits of these debug headers, some of the information they provide and how to best interpret this data. While we’re going to focus on Apache Traffic Server’s Debug Headers in this post, we’ll also provide links to some CDNs discussing their own debug headers in order to provide a well rounded guide for all things CDN debug related.

Debug Header Samples

By presenting “X-Debug” in the request for an object, the Apache Traffic Server — when setup with the appropriate package — will respond with a whole host of debug based options. Traditionally these options include a few critical elements of information that aid in debugging requests. These variables include: Cache State & Cache Location, Transaction ID, Cache Key

Cache State & Location — The Cache State and Location is typically revealed by the response contained within the “X-Cache” header. Here, you’ll receive information pertaining to the cache status of an object, as well as where that object is (for multi-tiered content delivery networks). The value of the cache status often includes:

• Miss — Object Not In Cache. This typically means the object was served by going all the way back to origin to retrieve the asset.
• Hit-Stale — Object in Cache, but is in need of freshness check. This typically precedes a 304 request to verify object freshness.
• Hit-Fresh — Object in Cache and good to serve from cache.
• Skipped — Cache Lookup Was Skipped. Typically done if the configuration defines cache bypass.

Additional information present in the “X-Cache” response is the name of the server responding with the cache information above. By appending multiple responses within the same response header, CDNs are typically able to present the entire request flow for an object.

For example, the below response would indicate an object that is not in cache at the first location, is stale in cache at the second location and is fresh in cache at the third location.

X-Cache: Miss server01.ats.lga.foo.com; Hit-Stale server03.ats.lga.foo.com; Hit-Fresh server01.ats.yyz.foo.com

As a result of this debug header, you can see the flow for this object through the CDN, and understand how things are behaving for your individual request, eventually being handled from a parent tier cache in a Toronto(YYZ) point of presence.

Transaction ID — Designated by “X-Transaction-ID” this value typically provides a unique string that is useful for identifying a transaction within logs. This enables retroactive analysis of a request to chase down exactly what happened to a specific request for a single asset. This feature is incredibly useful when trying to figure out how a specific object behaves within the entire CDN workflow and to diagnose any problems that may arise for a configuration.

Cache Key — Cache Key is perhaps the most useful debug header that you can evaluate. By providing the exact way that the CDN is interpreting and caching your object, you’re able to see whether things are appropriately configured on a specific request.

For example, let’s say you are removing query strings from a cacheable object in order to improve cache efficiency. While the request for the object may be http://www.bar.com/page/uri.jpg?123456, stripping of query parameters from the cache key should result in a different response from the x-cache-key header. In this instance, the cache key should be:

X-Cache-Key: origin.bar.com/page/uri.jpg

X-Cache-Key may also be revelatory if you have a more complex setup. Say that you’re using a different string match to replace a path within an object. By reviewing the cache key, you can ensure that this path replacement is occurring as expected, ensuring a smooth request flow for any asset requested against this specific setup.

One example of this scenario is if you’re configuration calls for /page/ in the URI above to be stripped on requests back to origin and replaced by /home/. In this scenario, the X-Cache-Key response should look like:

X-Cache-Key: origin.bar.com/home/uri.jpg

Add in complexity around regular expression configurations and advanced options and the X-Cache-Key feature becomes a crucial factor in being able to ensures CDNs are handling your objects appropriately.

More on Debug Headers

The usefulness and variety of CDN debug headers are numerous. Each CDN’s unique take makes memorizing these options and interpretations difficult.

While there is work underway to better standardize these requests and options as part of CDNi work in the Internet Engineering Task Force, the maturity of these headers across multiple different platforms still leaves a disjointed landscape of options for any content delivery specialist to navigate.

In order to better navigate these response header values, below are some examples from many of the popular commercial Content Delivery Networks. It is also recommended to always receive an education from your CDN of choice on their Debug Headers and all the different permutations when on boarding with that particular provider.

More to come from the CDN Edge in the future.

Useful Links

Akamai Pragma Headers:

Akamai Community

Fastly Debug Headers:

Deciphering Fastly-Debug header

Limelight Debug Headers:

Troubleshooting using HTTP Headers - ppt video online download

Additional Debug Header samples from a Catchpoint blog post:

Debugging CDNs: A Need for Change | CDN Performance

The CDN Edge — Debug Headers Can Be Your Best Friend was originally published in disney-streaming on Medium, where people are continuing the conversation by highlighting and responding to this story.

Content Delivery Networks (CDNs) have become a vital layer of the internet ecosystem. Once a term used to discuss companies focused on distributed networks of proxy servers, the solutions now provided in this space are far more encompassing.

CDNs are no longer just focused on distributed datacenters designed to scale to deliver static content. Now they are inclusive of edge computing, video streaming, mobile content acceleration, load balancing and DDoS Security, amongst other offerings. CDNs are central to serving a large portion of the internet content today.

And yet, for all of their features, functions and form, they still remain relatively obscure. Difficult to penetrate networks with internal architectures that are not easily understood. You may find out only a little bit about where your content is deployed at any given time, but to truly gain deep insight into where a specific individual request may be satisfied and to debug how your content is being delivered, you need specific ways to gain this insight.

In a continuing series, we hope to focus on best practices for engaging with cached content, the latest features on CDNs and some of the upcoming advancements in the industry.

This first entry in the series covers the specific aspects of the headers you need to be aware of to analyze your content cache health. A crucial aspect to debugging your CDN experience, Headers are the building blocks to understanding each and every request.

Know Your Headers

A simple spot to start in debugging any CDN request is to know your HTTP headers. These headers are crucial to being able to evaluate any request that comes in. Some key headers to keep an eye on include:

• Max-Age: Typically classified as TTL, or Time to Live, max-age (as well as s-maxage) is actually the HTTP header that is tracking this value. The Max-Age will provide insight into how long the object has remaining in cache. S-MaxAge which is more intermediate cache focused. In the case of a presence of both values S-MaxAge will take precedent.
• Expires: Another crucial aspect to determining the TTL of an object, Expires provides the exact date and time where an object will no longer be considered valid, resulting in a content refresh to ensure that it is still a valid cacheable object.
• Last-Modified: Want to know how old the file is that you are requesting? The answer lies in the Last-Modified header. While this does not necessarily tell you how long the object has been in the CDN cache, this header does inform you when the object you were requesting was last updated. With content that has a low TTL or is frequently updated, this is typically a good indication that cache is working as it should
• E-Tag: The E-Tag is a unique identifier per piece of content. E-Tags are typically a good indication of the uniqueness of an object in cache. If the object changes, the e-tag will change. With this simple header, you can verify that the object you’re serving and the object in cache are exactly the same. It also allows for an easy way for CDNs to verify that content is fresh — when it’s leveraged.
• Status Codes: We all know that Status Codes such as 200, 206, 403, 404, 503, 504 are crucial to any response indicating the health of the object, but did you know that certain CDNs have specific ways they handle these status codes and respond? 404s, while traditionally indicative of an object not being found, can also be the result of a CDN temporarily caching an error code. Make sure to note the difference based on the Max-Age of the error response. What about a 503 — Service Unavailable — or 504 — Gateway Timeout? These could indicate CDN issues or content origin issues. To dive further you need to be very cognizant of the status code you receive back and take the next step accordingly.

These are just some of headers that we look at when evaluate requests but are definitely the first few that help give an indication into how content is behaving and the role cache is playing in the requests.

A Simple cURL Can Be Your Friend

Performing a simple GET request on your content via Curl, or through a proxy tool such as Charles, is a key way to understanding CDN behavior. All CDNs provider debug headers that, when passed along with your request, will give you details back on what is happening with your content.

These debug headers, easily available by request from your CDN’s technical team or via an internet search, provide back a wealth of information that gives you an almost instant way to analyze the behavior of your content inside the Content Delivery Networks.

In the case of some of these debug headers, you’ll find information such as the TTL of the object — both as you set it, as well as how the CDN is caching it. You’ll also discover where your content is served from and whether or not it was being served by the CDN’s cache or being fetched from your origin (Hit/Miss).

Just The First

In a subsequent post, we’ll go into the above debug headers in detail and how to use them to analyze your CDN experience. We’ll also explore some alternative ways to test your content in cache, dive into ways to leverage proxy tools and more to further evaluate and explore the way your content is cached on the edge.

The CDN Edge — Best Practices For Managing Your Content in Cache was originally published in disney-streaming on Medium, where people are continuing the conversation by highlighting and responding to this story.