Stripping `server: Cowboy` from response headers in Phoenix
Or: Cowboy, CowboyPlug, onresponse, stream_handlers, and Phoenix.Endpoint
The Frame.io suite of applications (web, iOS, editor integrations and more) constitute an extremely rich and interactive end-user application. Video professionals of all stripes use the application to improve their video workflows, tying together many different elements of the editing and review process and many different resources and individuals. All of this adds up to an active back-and-forth between the clients and the API server.
Among other things, HTTP/2 represents a reduction of transport overhead from HTTP/1.1. For an app such as ours, the overall latency improvements from improving our transport layer could be significant.
Best of all, it’s extremely simple to upgrade to HTTP/2 from Phoenix 1.4. Phoenix 1.4 supports (but does not require) PlugCowboy 2.0; once you upgrade to PlugCowboy 2.0, Phoenix will be able to serve HTTP/2 with no further configuration. You might have to ensure that your endpoints are configured to serve over
After upgrading PlugCowboy 2, and setting up local self-signed certificates, everything Just Worked and we were serving https with no problem. However, one security-related regression showed up. That’s when things got dark.
By default, Cowboy includes a
server header in its responses. To some, this can pose a security risk—though that's a position that's not shared by everybody, least of all the maintainer of Cowboy. At Frame.io, we take security very, very seriously. If there is a way to harden our servers and reduce the amount of information we're leaking, we'll do it. So we strip the
server header out of our responses.
In Cowboy 1, this is accomplished through the use of hooks.
cowboy:start_http takes an optional
onresponse callback, which accepts the rendered response and does whatever it likes with it before passing it on. So your
onresponse callback can pull out the response headers and return a version without the
Under Phoenix, because Cowboy sits almost all the way at the bottom of its web stack , the way to pass the
onresponse callback is a little less direct.
If your callback is named
on_response/4 (as ours was), and you're exposing your web endpoint over HTTP, then your config entry will look like this:
This is what will be returned from
Phoenix.Endpoint.init/2, and then used to initialize the Cowboy handlers.
Phoenix depends on Cowboy, but it also uses the Plug architecture to handle web requests. So it also needs the PlugCowboy library to bridge the two. The config keyword list returned from
Endpoint.init/2 is therefore structured for consumption by PlugCowboy. Reading the PlugCowboy docs we see that the contents of
protocol_options are the 'remaining protocol options'. That feels a bit vague to me, but it links to the
cowboy_protocol docs where
onresponse is mentioned.
Under Cowboy/PlugCowboy 2, a couple things have changed.
onresponse is out; instead Cowboy 2 employs the more powerful, if more involved, concept of stream handlers.
In Cowboy 2, HTTP request/responses (both 1.1 and 2) are generalized into streams. A stream handler is a module which is registered for handling every stream, and receives a call for every event that could pertain to it. In other words, instead of providing a hook for a specific event in the request/response cycle, as before, a stream handler will be called for every event and should be implemented as a noop for any event it doesn’t want to do anything with.
Because it was surprisingly difficult to find a template for a noop stream handler, here is the entirety of the handler we built to remove the
server header from our HTTP responses:
info/3 callback here is the only one we want to implement a behaviour in. The equivalent of a Cowboy 1
onresponse hook in Cowboy 2 is an
info call with a
:response tuple. It's there, in our first function head, that we manipulate the HTTP headers for our response.
The rest of the callbacks are noops, so we simply call
cowboy_stream:<callback> with the same arguments we were given. Remember that we are implementing a behaviour defined in pure Erlang, so
__using__ macros and
defoverridable are not available. So if you don't implement all the callbacks, Cowboy will error out when it tries to call the function.
That said, where to put it? Here’s a massive warning for any reader: we were not able to implement this according to the PlugCowboy 2 docs. In the docs, there’s the same entry in the Options section:
:protocol_options — Specifies remaining protocol options, see Cowboy docs.
Under PlugCowboy 2.0.0, if we replaced our
onresponse keyword with an entry for
stream_handlers (which is the equivalent option found in the Cowboy 2 protocol options), it had no effect.
After some experimentation and reading the code, we found that we had to put our
stream_handlers entry in the root level of the
http config. So our config now looks like this:
There are a couple final things to note about setting up all these moving parts. The first is that if you were specifying
compress: true in your
http options, and you want to add a custom stream handler, you have to remove the
compress entry from the options and instead ensure that the
:cowboy_compress_h handler is specified in the list of handler modules.
The second is that the order of the handlers matters, and it might not be what you’d intuitively expect. In our first stab at putting everything together, we specified the three handlers but saw that the responses our client was receiving still had the header in them. That’s because we had put our custom handler at the end of the list. In fact, any changes you make with a stream handler to an HTTP response must happen before the stream is passed to
:cowboy_stream_h, for the simple fact that
:cowboy_stream_h is responsible for actually writing the response to the socket. So in our first configuration we were removing the header after we had already sent our response to the client.
After understanding the new stream handling logic in Cowboy 2, figuring out how to implement a stream handler, and understanding where and how to specify custom stream handlers in Phoenix, we were able to process our HTTP responses correctly and strip out those pesky response headers. Hopefully if you are trying to do something similar, some of this information will be useful to you.
¹As before, dependency management might be an obstacle to a simple upgrade. In our case, we were using a SAML library for Elixir that depends on the venerable esaml, which hasn’t been touched in some time and has a hard dependency on Cowboy 1. We ended up forking esaml and removing the parts that rely on cowboy, as we only rely on the part of the library that speaks the protocol.
Like what you’ve read? We’re hiring!
At Frame.io, we’re powering the future of creative collaboration. Over 500,000 video professionals use Frame.io to seamlessly share media and gather timestamped feedback from team members and clients. Simply put, we help companies create better video, together.
Across the stack we’re big users of AWS Lambda, Elixir, Swift, Go, and React. We’re a small, polyglot team that thinks big and works collaboratively to solve the biggest challenges for our customers that include Vice, BuzzFeed, Turner and NASA.