Delivering a better WAF — Faster

Andrea Swaney
Signal Sciences Labs
5 min readDec 4, 2018

At Signal Sciences, we’re in the business of reimagining the WAF and our purpose is to redeem the goal it set out to solve: automated protection of applications in production against known and unknown threats. We strive to do it with a model that fits with today’s fast pace of business.

As far as WAF perceptions go, we find that prospective customers fall into two camps: those who are dead set against using a WAF because of past heartburn with long setup times coupled with constant care and feeding, which require teams of resources and rule sets. The other camp has a WAF, but faces barriers when looking at scaling to other apps and architectures like hybrid and multi-cloud, which all require complicated setup and rule tuning.

We lived this experience firsthand, as our founders were practitioners running security, engineering, and product teams at Etsy. We set out to address the challenge in defending both modern and legacy applications by developing a technology that didn’t suffer from the pitfalls of existing WAF approaches:

  • Months-long efforts to install and operationalize a product
  • Broken legitimate web transactions in the form of false positives
  • Buried data analytics and a lack of insights that failed to inform operations and engineering on issues

In order to overcome the disillusionment with WAF, our primary goal was to get installed quickly, and show immediate visibility and value with real-time malicious website activity.

In order to do that, we knew it was critical to involve the development and operations teams who were part of deploying and consuming data from the product. Our technology team has created product principles that align with these goals, which help orient our product roadmap today:

  • Ease of implementation and use
  • Clear insights
  • Visibility that enables fast decision-making

Here’s what happened when we brought our next-gen WAF to market.

Starting fast — and then, scaling

To get installed fast, we knew we needed to integrate with all of the packaging and automation technology greatness to distribute our software.

We help prospective customers automate installation and updates using packages, public repositories, and configuration management tools like Chef, Puppet, Ansible, and Salt. Customer install time is an important KPI for us, and it’s become something of a sport at Signal Sciences to see how quickly new prospects bring their first agents online. The current record is under a minute; the average is just over an hour.

Installing on production traffic is the real deal

While testing in sandboxes can be a quick and efficient way to kick the tires on a new technology, they don’t mimic production traffic accurately, and therefore are a poor indicator of how effective the WAF will work on production websites. When you’re talking about legacy appliance-based WAF solutions, the sandbox doesn’t alert you to the complexities ahead of SSL certs, DNS changes, and rule tuning for false positives.

We wanted customers to see how we performed on their live websites. That way, they could quickly see how effective our next-gen WAF shows attacks and abuse against their actual live apps and APIs, and so, we conducted our proofs-of-concept in real customer production environments. That’s a pretty big ask, especially when we started five years ago, but we were pleased to discover how willing our first customers were to take that chance.

Doing a POC on production traffic paid off for both us and our customers. From our perspective, it made it possible to show just how quickly our next-gen WAF could provide real, effective protection. In one early bake-off against one of the industry big boys, we successfully completed our POC with a technical win before our competitor even finished installation. We won the bake-off based on results, not just speed, but that first impression didn’t hurt.

Getting Operations and DevOps teams on board early

In order to gain that buy-in to install in production, we had to anticipate the questions operations teams would ask, and we developed metrics to show how our software impacted their systems in terms of CPU and memory usage. For the DevOps and site reliability engineers (SREs), we showed average response times and sizes, as well as what impact our service had on latency (2–3 milliseconds on average).

Insights are more valuable than events

Fast installation is only valuable if you get meaningful insights right away (I mean, you can install anything quickly without it doing anything). Most of our customers with prior WAF experience complain about two things about the quality of WAF data: (1) learning mode and (2) unknown reasons for blocking decisions. Learning mode prevents teams from getting data right away, as the WAF tries to predict traffic patterns and the teams are required to tune out false positives. (This process repeats every time a change is made to the app). The lack of information around blocking decisions creates distrust among the operators of the tool with the teams triaging the root cause of an error without having any explanation from the tool that surfaced it.

The key to getting immediate insights is transparency. By sharing request payloads that went into any given decision to flag an IP as malicious via the tools they already use (Slack, Jira, Datadog, PagerDuty, Splunk, etc.), we provide self-service security data to all teams. Accessing the data themselves gives them confidence to move to blocking mode. (See our Customers page to read about this from our customers’ perspectives.) Instead of blocking a request immediately if we detect a potential attack, we developed a smart Cloud Engine that uses big data analytics to determine default thresholds that indicate an actual attacker, versus just a weird request.

On time, under budget, and ahead of schedule: a project manager’s dream

From the customer’s perspective, a live POC can turbocharge the whole procurement process. Lance Horner, head of security for Day & Zimmermann, a construction and engineering services company, came to us during his preliminary WAF evaluation with the intention of allocating budget later in the year. As it happened, the POC went so quickly and easily that his team was live in production in under five minutes, gaining immediate visibility and blocking attacks with the kind of accuracy that a legacy WAF would take months to learn. With nothing left to prove, Lance was able to get immediate approval for the purchase, and could refocus his time and budget for the remaining months of the year on other strategic projects.

For overstretched security teams, a fast POC with easy installation and short time to value is a big win. Customer environments become more secure on day one — not months in the future — with active blocking, clear insights, and actionable intelligence. With no need for continual tweaking and tuning, teams can turn their attention to other critical areas. Because if there’s one thing we know for sure in security — there is never a shortage of things to protect!

Reach out to us if you’re interested in learning more — and perhaps you’ll be the next to lower the record on installation time!

Photo by Marc Sendra martorell on Unsplash

Originally published at labs.signalsciences.com.

--

--