HTTP Headers and Serverless — Highlights from DevTalks Nov 2018

We were thrilled to host the third round of DevTalks this month. The idea behind DevTalks is to bring together software developers from the Helsinki area to listen to interesting speakers and to connect and learn from each other. We were delighted to see an almost full house with 140 software developers.

In previous DevTalks, we have focused on tackling Complexities in Software Development and dived into the world of Microservices and Advanced SQL. This time, we delved deeper into the realm of backend services, with talks on advanced HTTP headers, and microservices and serverless by our two amazing speakers: Andrew Betts and Sam Newman.

• Andrew is a technical product manager at Fastly and served as a member of the W3C Technical Architecture Group (TAG) for two years.
• Sam is an independent consultant, author, and speaker working mostly with the cloud and microservices. Our engineering team had a great half-day workshop with him before DevTalks.

We kicked it off with some dinner and drinks, and then we dove into the world of HTTP Headers.

Andrew Betts — HTTP Headers for Hackers

Working for a company that provides a Content-Delivery Network and being part of the committee that oversees web architecture have provided Andrew with a broader view of HTTP headers than most people.

Since the HTTP was invented by Tim Berners-Lee, we’ve been adding all kinds of new things into it, mostly in the form of different headers. However, the thing that Andrew made very clear is that many of us (perhaps, most) are using these headers when we shouldn’t, i.e they’ve been deprecated. Others are using them incorrectly or not using headers that we really should be as these features would provide a lot of value and security if adopted more widely.

Which headers should we use or not use?

There’s a bunch of headers that we should actually stop using for various reasons. Some of them are actually not even used nowadays (P3P header sent by 10% of domains) while others have been superseded by newer or better-defined alternatives. Examples of those are the Expires header, where Cache-Control should be used instead, and X-Frame-Options, where the equivalent Content-Security-Policy is more appropriate.

One big group of headers we should use less are headers that are meaningless to the browsers and often debug information for developers. Examples of the useless headers for browsers are Pragma, which is actually a request, not a response header. Via in many cases, as well as X-Cache and X-Request-Id and all of their variants and close relatives are also considered redundant. These provide no value in 99% of cases and could be enabled conditionally for developer use for example by using a debugging cookie.

On the flipside, there are a second group of heads that should be adopted more frequently. A few handy tools for checking how your own site is doing are securityheaders.com and observatory.mozilla.org, which audit for your site and inform you which headers you should and should not use.

Start with these headers:

• Referrer-Policy: Ensures that your sensitive URLs are not leaked in referer headers
• Content-Security-Policy: Controls and mitigates XSS vulnerabilities and data injection attacks
• Strict-Transport-Security: Ensures that your site is always accessed through HTTPS
• Access-Control-*: Whitelists cross-origin requests

The future of HTTP headers

Some useful and interesting proposals that have been made to the HTTP specification that we should keep an eye on include:

• Use Link (rel=preload) header to start preloading resources as early as possible
• The proposed HTTP 103 status code, which could allow sending headers even before the web server has decided which status code and other headers need to be sent
• Accept-CH: Informs the browser to send information (Client-Hints) such as screen pixel ratio or if the client wants to save on data transfer, in subsequent requests
• Server-Timing: Provides server runtime debugging information to the client in a format understood by the Chrome debugger
• Feature-Policy: Turns off complete features of the web platform on the site, i.e. media autoplay or document.write

Many of the HTTP headers relate more to the entire site than to the resource itself being fetched, which is why there’s a proposal to combine these under a single site-wide manifest that is advertised with the Sec-Origin-Policy header. This would clean up the headers and save some bandwidth as well as reduce mental overhead when dealing with the headers.

Sam Newman — Microservices and Serverless

In the workshop with Sam, he shared his deep knowledge and experience with Domain Driven Design to inform microservice design decisions. He led us through a practical design exercise called Event Storming. Event storming is a team process useful for defining bounded contexts in your problem domain (a Domain Driven Design concept). These bounded contexts can then be used to define microservice boundaries.

Later on in Sam’s presentation, he offered suggestions on ways teams can use serverless architecture (the concept, not necessarily the branded framework), to implement microservices.

The World of Microservices

One definition of microservices is that they are “Independently deployable services that work together, modeled around a business domain”. As products grow larger and more complex, and you the teams working on them are larger, the more issues with synchronization, dependencies and too many communication lines between individuals there will be. There’s also a correlation between a team’s organizational structure and product architecture, but it also works in reverse — if we model our teams a lot differently than our product architecture, for example by having lots of teams working on a huge monolithic codebase; things can get lost in translation.

According to Sam, microservices are an architecture optimized for autonomy. The emphasis is on the independently deployable part as microservices allow for distributing responsibility for different parts of the system, while reducing coordination needed to make changes within the whole system. Ideally, teams should own one or more aggregates or domains within the whole product that are modeled as different microservices, and provide well-defined interfaces through APIs for other teams to access. A real antipattern couples different services through backdoors to the internal data models, like accessing the underlying database from another service.

This creates a tight coupling between those two services and teams responsible for them. Sam shared an anecdote of a situation like this, where the underlying database was tightly coupled to 10+ independent services, many of which were unknown. The only way the team could find out what those services were and who owned them, was to turn off the database and wait for people to call to ask why their services stopped working!

The Cloud

As microservices are an architecture optimized for autonomy, it also works very well in the cloud environment. After all, the cloud means autonomy through self-service and higher order abstractions over infrastructure. Operating software in the cloud removes dependency on big operations teams that take care of physical servers and data centers and allows developers to easily deploy and use resources. We’re also transitioning from Infrastructure-as-a-service (or IAAS) platforms, where we rent virtual servers on-demand, to more flexible solutions that allow us to package our software as containers and deploy them on multiple different platforms (such as Kubernetes).

Serverless

The latest development after containers has been the move to Serverless world. Serverless does not only mean Functions-as-a-service (FAAS) platforms but rather all kinds of services where it is not actually necessary to think about the physical or virtual servers anymore, such as Database-as-a-service platforms, cloud storage, messaging solutions, and others. Many of these services build on top of other cloud solutions, and some even operate on top of container systems as an implementation detail.

However, the higher the level of abstraction, the less infrastructure overhead there is, but there’s also less control over the implementation. Ultimately, Sam’s thesis is that even if containerization is a better form of virtualization, usually serverless is a better form of abstraction when it comes to software development.

Serverless microservices

What do serverless microservices look like? A microservice is usually modeled around a single aggregate, like “invoices”, and it owns all of the related data. Access to the data happens through well-defined APIs, and in the case of serverless, the aggregates state transitions can be modeled as FAAS functions. A service thus consists of cloud functions, that are all independently deployable but coupled to the same database.

How do we then keep the separation of different aggregates’ data intact? What if your invoicing service needs to access some user data? We can still think of the logical collection of the cloud functions within the aggregate as an isolated service, and all data access needs to be funnelled through one of its APIs. If the invoicing functions need to access any user data, we need to call the user functions, and not go through a backdoor into the user database.

Many FAAS platforms also provide some form of persistence layer in the form of a database service or similar. Unfortunately, none of these platforms offer very good tools for having a well-defined isolation between the cloud functions of different aggregates. In practice, this requires strict adherence to the aggregate and service boundaries on the database level.

Summary

For some time now, the hype has been building not just for microservices but also the modern infrastructure techniques with which we can provide them (i.e Serverless). What really stood out from Sam’s workshops and talks was that good conceptual design of the problem domain is crucial to architect robust and maintainable microservices.

Domain Driven Design provides many design tools and techniques that aid in this process, in particular, the bounded context. The bounded context is created in close collaboration with those that own the problem domain (organizations that we build software for, and their customers), and can be formed using techniques such as Event Storming. It’s these bounded contexts that help us design and implement microservices that any number of business applications can use to drive business value.

One often understated, but very important concept is that the success of a microservice is not only measured by its ability to provide applications with an API into the business domain. A successful microservice is also measured by how easy and enjoyable it is for developers to use and build upon.

This is the goal of Domain Driven Design — to design and build software that fulfils business goals whilst making that software enjoyable to use and develop upon, over its entire lifecycle.

DevTalks will be back next spring

We had a very exciting night. I would like to thank Andew and Sam for the inspiring talks, and all the participants who joined us for coming and engaging in intriguing conversations after the talks!

DevTalks will be back next spring. Join our Meetup group to stay tuned for updates.

Originally published at www.smartly.io.