Seven Hard Realities of Application Security For Non-Developers — Part 1

Application security is finally getting the visibility the discipline deserves. Multiple blog posts and new books are bringing the craft of application security to the infosec masses. That’s a good thing.

What might not be clear to aspiring application security practitioners or to CISOs and other senior cyber security practitioners wanting to grow an application security program are the very real challenges to being successful at both the individual contributor level and at the program level.

Looking into the developer community from the outside might give the false sense that securing the software development lifecycle is the same as securing any other IT asset. In the loosest technical sense, it is. That said, as this post will demonstrate, the process, focus areas, and issues can be very different.

For context, this two part series of posts comes from a place of love. I love application security and have been standing up successful application security programs since early 2001 (when the discipline was loosely termed as “software security”, if even referenced at all). I’ve been writing code since 10th grade, led large cross-functional commercial software development teams, and am currently a CISO with an increasingly robust app security program that supports a fast-moving entrepreneurial digital transformation effort. This post is not to scare people or put them off to diving into application security; I’m just sharing many of the things that actual app security practitioners know well, but only infrequently share.

The seven hard realities presented in this series are not focused on whether the app security practitioner or team knows how to identify application vulnerabilities or has learned to use their tools. The seven realities frame the very real issue of the difficulty of actually getting to the program to the point of being able to apply appsec skillsets and deciphering the results in a meaningful way.

For those that want to cut to the chase, the seven hard realities of application security are as follows:

#1 — App Security Success Is In The Ability To Adapt To Dev Culture, Not The Other Way Around

#2 — App Security Is More Than Scans And OWASP Top 10

#3 –The App Sec Team’s Dev Skills Largely Determine The Division of Work Needed With Devs

#4 — App Security Tools Are Less Mature And Comprehensive Than Expected

#5 — Not All AppSec Defects Are The Same Or Even Are AppSec Defects

#6 — DevSecOps Is A Very Limiting Misnomer

#7 — Measurable Results Are What Matter At The App Security Program Level

The seven make for an ambitious list so we’ll only cover the first three realities in this blog post. The second post in the series will cover the rest.

Let’s jump right in…

App Security Success Is In The Ability To Adapt To Dev Culture, Not The Other Way Around

So, lets say that you are the initial application security cadre at a company with a fairly mature development organization. Perhaps, you are this company’s CISO armed with at least one headcount and some budget for services and tools. In either case, you are full of vim and vigor. You should be.

Where do you start?

Unfortunately, the very first issue to deal with isn’t how to set up your tools and get started, but how to navigate, deal with, and adapt to your organization’s developer culture in a meaningful way. You have a strategy, why should you worry about the culture? Peter Drucker said it best: “culture eats strategy for breakfast.”

Dev organizations without a strong culture are few and far between. While I have seen dev teams staffed with largely junior developers, you should assume that these are the exception. You aren’t generally going to find that strong of a skew towards junior devs in mature or well-funded organizations.

The reality is that any team with mid-level or especially senior level developers are bound to have strong opinions about, well, everything. That may be daunting to some. To me, it signals “opportunity”.

I welcome those strong opinions. If you believe that the opposite of love is apathy, not hate, then strong opinions come from people that love their product and genuinely want to make improvements to that product. They may disagree with your well intentioned approach and that’s ok at this point. I fully understand that to someone without significant time interacting with application developers and devops professionals at scale, it may be somewhat of a surprise the first time that they encounter a surly dev who flat out says, “nope, not doing that.” Be prepared for it. It’s coming.

Aspiring app security professionals and CISOs alike need to understand and account for the following:

Developers tend to have a completely different culture than other IT professionals. Actually, in many organizations, they don’t regard themselves as IT. In their view, the software developer represents royalty and every one else is there to support them to some degree.

Think about it this way. The dev community has formalized entire frameworks so that they could eliminate or significantly reduce the role of project management (“Agile”) and the role of testers (“Test Driven Development”). They were viewed as constraints by the devs to get to the best solution. So, please tell me again, how application security will be different won’t on the surface be viewed as a constraint.

Your application security program will only be as strong as the willingness of the extended dev /devops organizations to support you. Even the strongest and most talented app security function will still have strong dependencies and non-trivial requirements that have impacts in terms of time and bandwidth on the development process. This means that you’ll need to find how to best engage with the devs and devops within their own processes, rather than try to do the heavy lifting to establish some new process. Trust me, a new process won’t work.

So, your first planning step will be to understand the existing methodology and frequency that the dev team plans, resources and assigns work.

So, you might be thinking, “oh, well ,we are the app security team. This will be easy since everyone knows that we are the highest priority”. This enforcer-driven thought process may work in highly regulated industries or in other parts of IT, but the hard reality is that it’s likely to fall on deaf ears in the dev community. Even if forced to comply, the enforcer approach may get the app sec priorities quickly put into the “slow roll” bucket.

“Enforcers” transitioning from the traditional infosec approach of demanding everything be done NOW! will have a hard time of finding success in their transition to app security. They will be continuously swimming upstream of two very strong currents: the dev culture and the process within which the dev plans and resources work. Eventually, they’ll tire and the currently will carry them downstream against the rocks.

You’ll have to learn to use the current to your advantage.

Instead of being viewed as the enforcer, you are going to have find a balance of enablement and enforcement. Enablers can frame the app security program as helping the dev community produce a better, higher quality product with less rework. These are things that can more easily connect with devs that love the product. You’ll need to save the enforcement silver bullets for when they really are needed…which should be rarely.

App Security Is More Than Scans And OWASP Top 10

Most of the historical thought leadership around application security has been driven by the tool vendors. This means that there has long been a perception that application security is just a set of activities around scanning code. I’ve been a strong advocate and writing for years that the discipline of application encompasses far more than just scanning; app security is about governing, securing, and protecting the entire software development lifecycle for code, applications, and APIs: design, build, pre-production, production, and, the normally overlooked, “end of life.”

The reality is that a robust application security program is far more expansive and complex that simply scanning applications. My personal opinion, after having stood up numerous successful app security programs is that the app sec discipline is too broad for one person.

This post won’t deal with the details of the scope but I have several past blog posts on the topic as well as adding business value to application security programs. Tanya Janca has also written a well reviewed and easily understood book that covers the full breadth of what’s needed to secure software development lifecycles.

Static scanning for OWASP Top 10 vulnerabilities are a great focused start. I’m personally embarrassed for our industry that the level of foundational application security is so ineffective that 7 of the top 10 OWASP vulnerabilities haven’t changed in ten years. An initial focus on measuring and fixing those obvious vulnerabilities make for an easy story for both the extended development organization and for executives. If you are going to breached via your code, at least let the exploit be something interesting and not because the devs weren’t performing input validation on a test form.

That said, OWASP Top 10 can’t be the only focus. It marks a very low bar for overall program success. I think of the top 10 more like meeting initial table stakes than some sort of long term program goal. There are tons of other types of app sec vulnerabilities not in the top 10 that also need remediation and controls.

Given the broad scope, it’s unlikely that an infosec engineer will be able to handle to application security responsibilities on a part time basis or, at the program level, with part time resources. Invest in someone that can become a peer with the devs and remain focused on their delivery. Even if your organization only writes internal applications, application security needs to be resourced like a program, not a set of part time activities.

App Security Requires A Division of Work Determined By The App Sec Team’s Abilities

The division of work within application security is also an area that is unknown to many aspiring application security practitioners and CISOs planning to govern application development.

There are four truisms in terms of properly structuring and resourcing application security:

1) Organizations aren’t successful over time governing themselves.

2) A fixed set of application security requirements unique to each organization determines the full set of program activities to govern and secure an organization software development lifecycle.

3) Of this fixed set, some requirements can only be implemented or performed by application developers, app dev testers, or devops and, for which, they need to be resourced.

4) The rest can have a division of work (or even more interestingly as pointed out in conversations with J Wolfgang Goerlich, a division of passion) determined largely by the coding ability of the app sec team.

Some organizations want to roll everything under the devops umbrella or have the devs govern themselves according to a set of policies. Unfortunately, there are few successful examples of self-governance over an extended amount of time. To me, the app security team must remain a separate and independent team in order to fully carry out their governance responsibilities.

Next, we have to assume that there are a fixed set of things that need to be governed within application development. This set of things varies by organization depending on the dynamics of the type/amount of internal code written and the organization’s structure: devops, test/QA, source management, etc.

An app sec team can’t work in a vacuum and expect to find success. Some things including activities that seem very application security focused, the app security team can’t do. For instance, the app sec team obviously aren’t the ones writing secure code or implementing the build pipeline. That said, the more the appsec team can understand and write production code, the more requirements that appsec can take on without significant dependencies on devs or impacts to their time.

Some of the areas that come to mind for the potential division of work (or passion)and should be planned for in a purposeful way include the following:

• Who maintains any existing application controls that are already in place
• Who creates, owns, and maintains the secure coding standards? What’s the relationship with other coding standards?
• Who owns integrating application security tools and security controls into the build environment and how?
• Who reviews scan results for false positives?
• Who makes fixes to code? What’s the process? How are security defects prioritized? Is there an SLA for fixing these defects?

I’ll do a more complete analysis of these opportunities to divide work between the application security team and the extended dev team in a future forum. However, this list will at least get you started.

Note that many of these items require the ability to read and interpret code.

I’m a fan of hiring and training mid-level developers that are tired of the grind of production dev work and want to move into application security. Former devs also have the language, the understanding of dev culture, and secret handshakes required to be successful faster. It’s simply easier to train a dev to become an effective security practitioner than the other way around. I also want an app sec team that independently assumes more of the review governance role and can identify pertinent anomalies and control issues at first glance. It is only one of N possible approaches that can address the application security issue.

Don’t mistake what I am saying here. I am not saying that app sec practitioners have to be able to write production code. There are reputable vendors that offer these services as a managed services option. I am also aware of application security teams that largely are unable to code yet handle their responsibilities internally and, from what I can see from the outside, they appear to be successful. Some orgs use interns or junior infosec analysts to save money. Others believe that their only role is to provide application security to push the “start scan” button on a vendor provided solution. I don’t know how successful these last examples are.

The message (and accompanying hard reality) is that application security teams without an internal base of development skill will have significant dependencies on the time and largess of the development organizations that they support or on a significant expense of vendors and service providers. Adding to the complexity, application security teams without deep experience in the intricacies of the software development process will not be able to provide expert judgment of the coverage, quality, or efficacy of work that is delivered from outside of the app sec team. As a CISO, I view this inability as an additional cost of inexperience.

Even the integration of application security toolsets into the build process require some coding ability. Someone, somewhere (whether on the app security or not, internal or contracted) has to be sufficiently resourced so that they can own that work.

SUMMARY

There is a lot to absorb about application security that has little to do with the techniques and methods that comprise the art and science of application security. You can do this. Awareness is the first step.

In part 2, we’ll cover the last four of the seven hard realities of application security for non-developers.

• app security tool maturity
• app security defects
• devsecops
• measurable program level results

The beautiful thing about application security is that it can truly be as full of potential as the uncarved block.

Read Part 2.

For more insights into how cyber leaders can best enable the business and build rock solid cyber programs, please follow me on Twitter at @opinionatedsec1

You can also find more of my previous content at the “CISO & Cyber Leaders” publication on Medium: https://medium.com/ciso-cyber-leaders