How Much is Enough?

You can’t be too rich or too thin. Apparently, Wallis Simpson did not coin that phrase, but wvr. It’s also kinda disgusting and bourgeois. That’s Mister Snowflake to you, muthafucka.

In this article, I argue with myself and come out on top. Disagree loudly with the appropriate parts. I won’t be offended because I contradict myself. Easy win. Bada bing bada boom.

So riddle me this: How much quality is enough? Devs stereotypically want all the quality. Especially younger devs, who have invested their egos in their code and can’t bear to live with defects. Orgs that have failed to do any sort of risk analysis, formal or informal, don’t have a rubric for deciding what level of “quality” is the Goldilocks level, so they might blindly demand that quality should be pushed to its limit.

As I have alluded to before, insurance makes a great analogy. How much insurance is enough? Answer: it depends.

You know where I’m going with this, right: how much security is enough?

Unfortunately we’re still in the prepubescent phase of AppSec, so many orgs can’t figure out what questions to ask, so they demand all the security. Fix every vuln! Patch like the wiiiiind. Buy every piece of kit you see on the RSA Conf floor. Sounds dumb, right? Well, I doubt that anyone is actually doing all those dumb things (at the same time) but you get my point.

It’s all about risk.

Let’s digress briefly and look at DevOps and agile (lowercase a because I am not talking about Scum — uh, I mean Scrum.) I started out back in the day as an XPer. That movement made a very conscious decision to go all out after quality. Quality was paramount but this is the key: its measure was based on Customer acceptance. Are we delivering what the Customer wants? Quality was not just the absence of defects.

Even back then, “zero bugs” was a thing, where bugs would be fixed upon discovery. We didn’t take the time to assess risk and nor did we do much triage. We just fixed the little fuckers. Delivering what made the Customer happy was just enough quality. We tempered that drive for quality with an insistence on doing “the simplest thing that could possibly work.” That helped to keep defects down a bit maybe. But we sure as hell didn’t maintain any long-ass “backlog.”

I’m a huge fan of “zero bugs” but isn’t that exactly the kind of thinking that I disparaged above as immature? How am I going to dig my way out of this one?

Lean Software came along and told us that defects led to 無駄 (muda — usually translated as waste) due to rework. In order to optimize flow of work products (that’s code to you and me) through the systems, defects must be found and fixed as close to the source as possible. Indeed, the andon cord must be pulled as often as possible, and a decrease in the number of times the andon cord is pulled signifies a reduction in learning opportunities.

Well sheeeeeit, sounds like “zero bugs” to me, the same thing that I called egotistical and prepubescent. I’m digging myself deeper.

Anywaaaaay, Patrick started DevOpsDays and Jason did it and then Jez wrote about CD and Eric coined Lean Startup (and brought “pivot” and “MVP” into the corporate lexicon. Thanks, Eric, thanks a lot.) These were natural evolutions. They were transformative, and I believe they’re table stakes. Security was this thing you (maybe) did before shipping, when it was already too late. Fast-forward a few years and now we’re evolving to a place where Sec is getting jammed in between Dev and Ops (and yes, it’s still called DevOps, not that other thing.)

And that’s put AppSec under the microspcope. We’re having to ask questions like How much security is enough. Hitherto, it didn’t matter how much was enough because no one paid any attention anyway. Now, if security is part of the SDLC, then a security bug can (must?) cause the andon cord to be pulled. Stashing a vuln into some sort of backlog is not clever.

Have I just completely changed my mind? I started this article with a point to make, namely that all defects should be assessed based on risk. Now, after looking at agile and Lean and DevOps, I see the incredible value of “zero defects”, of reducing muda.

Ok, one more wafer-thin analogy. Maybe this will settle my argument with myself. (Sorry you had to see this.)

So Gary McGraw said a few times on his (awesome) Silver Bullet podcast that the “shift left” that intends to incorporate AppSec into DevOps is all fine and good, but agile practices don’t address security “architecture”. (Sorry if I’m mangling your message, Gary.) I mumbled and muttered a lot when I heard that and declared that he just doesn’t know what he’s talking about.

Upon reflection, however, I began to understand. (Gary is a very smart dude, jusayin.) Security architecture refers to cross-cutting non-functional requirements in the security realm. Architecture in this sense influences what code gets written but it is beyond the code. If all you’re trying to do is implement Feature A, you are not necessarily going to think about architectural-level things.

Here’s an example. You have a system that sends commands over RF to a spacecraft. The commands are encrypted. Maybe you use PKI to negotiate the keys. That’s all fairly straightforward stuff, easy enough to express as Stories and implement in code. Now here’s the security architecture part: Protect against replay arracks. Oh, oops. Now simply encrypting the commands is not enough. You need to come up with a mechanism to do that, but more important, you need to know that it’s necessary.

Here’s another example. You’re using PKI as above. Who issues your certs? GoDaddy? No. You do some deep threat modeling and decide that you’ll run your own CA. That’s kinda a BFD because now you have to deal with physical security of the CA boxes on a much higher plane. You have a whole shittonne of dilligence in setting up and using the CA. These are the crown jewels so you need some heavy protections. Again, you need to be aware of the need for this.

You need a sec expert to see these sorts of architectural issues.

Ok, now for the analogy I promised. UX is to UI as Security Architecture is to AppSec. UX is a cross-cutting thoughtspace that transcends the nuts and bolts of UI. You can build a perfectly functional UI and still manage to fuck up UX. And UX is something that you need a UX expert for. You need someone who knows the ins and outs of UX who can assess what’s needed.

I’ve been privileged to learn some ingenious techniques for doing just-in-time UX research and incremental threat modeling. These are the tools that hold the promise of shifting “security architecture” and UX left and into the SDLC. The implicit goal of these lightweight, iterative approaches is to increase throughput and decrease defects.

Is it wise to try shifting those left? Is Gary right? Should we continue pushing on these (which I find absolutely fascinating, btw) or should we insist on measuring the risk of all the things?