Yes. Cyber Leaders Can Reduce Distracting Work

Opinionated Security
CISO & Cyber Leaders
10 min readJun 25, 2020

Distracting work. Disruptive work. Unplanned work. Call it what you want.

We all know that cyber teams are most effective when security operations are normalized, yet, many teams and executives continue to regard distracting work as the norm. We, as an industry learned to accept the constant churn, frequent whiplash, and “death marches” as part of the job. But, does unplanned chaos and distraction really have to be the way that we operate?

This may surprise you. but I don’t think that distracting and disruptive work should be the norm.

There has to be a better way….

….which brings us back to distracting work.

What if cyber leaders went all in and viewed any unplanned work as “distracting work”? That’d be a big goal. But, could that be achievable? Obviously, we couldn’t reduce that work to zero but how could we get the distractions to a point to where we could focus on making improvements rather than always responding to incidents?

In my view, distracting work should be the bane of every cyber security leader. We want a seat at the executive table but seem to be unable to engineer consistency into our own organizations. Distracting work impacts our ability to retain great employees and keep them on a long term path to securing the organization.

In metaphorical terms, it’s the difference between fixing the holes in a sinking boat and just bailing the water. Distracting work is just bailing. As you make progress fixing the holes in the boat, the boat is in less danger and there is a less pressing requirement to spend time bailing the water. If all we do is bail, we’d be bailing forever without time to do much else.

I think of four primary paths to reduce distracting cyber work.

  • Enable process controls to engineer the risk of human error out of business processes
  • Build secure capabilities that reduces the surface area for risk root causes to exist
  • Manage upwards to align the cyber plan with executive expectations
  • Manage incidents as they occur by responding and finding incident root causes

The irony is how many cyber teams only really focus on #4.But, to continue the sinking boat metaphor, #4 is really only bailing the boat.

So, let’s look at each of the paths in detail for how they can each contribute to a reduction in distracting work.

(1) Enable process controls to engineer the risk of human error out of business processes

This is first on my list for a reason: human error accounts for anywhere from 52% of security incidents (CompTIA) to 90% of security incidents (both Kaspersky and Kroll).

Is there a better place to focus than the prevalence of human error’s contribution in terms of managing distracting work? Probably not.

That said, the irony is that this focus area probably receives the least amount of attention by cyber teams. The most common anecdotal reaction to human error that I see is a shrug of the shoulders and a fatalistic acknowledgement that it’s human error. Perhap we also feel that it is some other team’s repsonsibility to manage.

o, how ca you address human error as a cyber security team? Here is a somewhat controversial thought that works for us: don’t accept human error. Period.

Wait. What?!?

The key is to view the human error process purely from a controls perspective. Using this perspective, we might be able to better engneer the process in a ways that reduces the occurance of human errors.

For example:

  • A control was missing that allowed the human error to occur.

Example: There is no documented procedure or training for a particular activity. The employee that does it has done so for years without error or incident. That employee is sick, retires, or otherwise has someone else perform the activity and that person misses a step which results in a breach.

  • A control was present but not followed and there was no governance for the control to ensue that it was followed.

Example: There is a documented procedure for a particular activity. An employee doesn’t follow the procedure and misses a step which results in a breach.

  • A control was present and the governance for that control was not effective.

Example: There is a documented procedure for a particular activity as well as checklist that is be signed off on by a supervisor. An employee doesn’t follow the procedure and misses a step. The employee completes the checklist by habit and the supervisor igns off without checking. The result is a breach.

As demonstrated in the examples, any of the above situations will allow “human error” to occur. All were preventable with either a control or governance of a control. If every occurance were analyzed from a controls perspective, we could continue to mature the underlying process so that the chance for error continues to be minimized in accordance with the level of risk.

I’ve used this approach and it works. That said, it also takes dedicated effort as well as communications plan but the payoff is a reduction in distracting work.

Although human error is only a part of the issue, it represents a large amount of potential surface area fro generating distracting work.

(2) Build secure capabilities that reduce the surface area for risk root causes to exist

Rather than put band-aids on insecure capabilities, I’m a fan of building out of distracting work. This building might be done by the cyber team or the team that owns the business process.

Capability building is generally focused on one of the following three outcomes:

  • Growing (or helping other teams to grow) new capabilities that either make organzations more secure or are secure by default
  • Maturing (or helping other teams to mature) existing capabilities that result in more secure approaches, have increased scope, or are more comprehensive in coverage
  • Strengthening security governance around capabilities

This focus goes beyond the deployment of new tools within the security portfolio of projects. It includes the ability to threat model whle in design to foresee potential issues, development of supporting processes that reduce opportunities for introducing risk, deepening the scope and comprehensiveness of coverage, and improving the measurements both at the operational and program level.

It is improtant to note that these improvements can be made incrementally and continue over time. The incremental steps can be defined by maturity goals and the requirements associated with each level.

That said, you’ll have to understand your current capabilities and business objectives in order to know how to prioritize what needs to be built. You’ll need to drive your services vendors to align to your plan. Blind reliance on your vendors to perform this prioritization might end poorly. I’ve had a case in which our big name services vendor was really tightly focused on IAM and CASBs at a time when our most pressing threats and risks were elsewhere. If we had not known our business or performed our own internal analysis, we might have had an elevated possibility of breach as a result.

Effective capability building also means that the cyber security team will have work closely and play well with other teams. You won’t be able to stand in the corner and point to your compliance frameworks to be effective at capabllity building. Communications and negotiations will become key skillsets.

Some examples:

  • The cyber team will have to help other tam understand that “industry best practice” and “security frameworks” are not end states, but only table stakes;
  • The cyber should be threat modeling in the design phase to predict what risks may exist and need to be managed before deployment or coding.
  • The cyber team needs to work thoough testing of functionality in pre-production
  • The cyber team will have to capture identified risks, document them, and ensure that the decision about how those risks are managed is done at the appropriate level.

So, clearly your cyber team will need to engage with other teams to build and govern capabilities. The idea is to spend your team’s time engaging in activities designed to reduce the opportunities for distracting work rather than being constantly whip-lashed by the distracting work once the capability is in production.

You can do this.

(3) Manage upwards to align the cyber plan with executive expectations

When you think about distracting work, how much of that workcomes from leaders above you? If that’s an issue, perhaps it could be an indicator that you aren’t managing upwards as well as you could be.

Please note that the intent here isn’t that you shouldn’t ever accept unplanned work from your leadership. That would be a potentially career limiting move. The key is to transform distracting work into planned work by fitting it into your plan rather than have to deal with turn your team on a dime right now.

Perhaps your leadership isn’t clear that you have a plan or they don’t understand your plan/priorities.

To me, a lack of a plan is like a vacuum and someone’s plan is going to fill that vacuum. Leaders are often closest to “feeling” that vacuum and if they don’t think that you have a plan, they are often the ones that will fill it with their plan. The result? Unplanned or distracting work.

How do we mitigate unplanned work that comes from above.

Cyber leaders have to manage upwards. The plan is probably clear in your mind but what about your leadership? Your leader should understand that you have a plan and have enough confidence in that plan to not feel the need to put their own plan in place. This is a constant communication upwards to keep your leaders informed.

But, the messaging around that plan to leaders can’t just come from you. Your leaders talk to others as well. Ensure that you are clearly socializing the plan with all stakeholders. To effectively manage upwards, you’ll need to rely on your peers and direct reports to be able to communicate your plan effectively so that leaders hear a clear and consistent story.

I like to use quarterly security commits to keep everyone aligned. Commits are tightly defined, meaty capability building work that each pushes the cyber program forward in some tangible way. They also are communicated before the quarter and have a pre-negotiated priority with my leaders. The only thing with more priority than commits are actual incidents. We deliver on the commits consistently because of the pre-planned priority and, because our leaders know that we are executing against a plan, they are more wlling to fold unplanned work into the larger current plan.

Distracting work can also originate from within your own team. Your direct reports (or their directs) can get off plan for any number of reasons. It’s your job to keep them focused and on plan.

One of the most common distractions from inside the team is out of band requests. Example: you handle vendor security reviews on thursdays. Everyone begins demanding theirs out of band and so every request being being handled as an out-of-band exception. In this case, the key is to establish criteria for actual emergency exceptions and drive all other requests into the normal process. This way, out of band distracting work becomes planned work.

Communications and negotiations are powerful allies in reducing disruptive work. .

(4) Manage incidents as they occur by responding and finding incident root causes

All cyber teams know that responding to incidents is a key part of their responsibility. That said, after the team has handled an incident, now what? How can the incident (and the associated response) help to reduce distracting work?

My underlying assumption here is that cyber security organizations at every level of maturity can learn from every single incident. Large and small. Forever.

You just have to look hard enough and be very critical of your own performance.

This isn’t a negative approach. Your team can always do something better or faster. You can capture criteria for improved controls that you need and later find controls that meet those criteria (as opposed to just buying tools with the most popular name).

Another way to think about this is by comparing what is done after a pen test and the priority of post-pen test remediations to the findings and priority of remediaion of findings after an incident. Do they carry the same priority and executive attention? Normally, executives mandate remediations of the pen test findings with a high level of priority. Why not do the same with remdiations noted from actual incidents?

If your cyber team isn’t generating their own findings from actual incidents, they should be. Real incidents can be your best pen test as there are no limitations for bad actors.

A simple guide for gleaning value from incidents might be as follows:

  • What controls are missing that would have detected, deterred, or blocked this incident?
  • What controls weren’t effective as they should have been in this incident and why?
  • What was the team able to do during this incident but needs to be able to do faster in future incidents?

My recommendation is to also not wait the few days to a week for the incident post mortem to raise and begin to answer these questions. Begin to capture them immediately and communicate them to your executives. Resolve these issues as your budget allows. Track the ones that remain unresolved. Budget for those that require budget.

Answering these questions may also cause you to rethink the value of even the biggest names among your security tools or help find other tools that simply are a better fit for the requirements you have identified during incidents. We went through this process and now have better, more comprehensive coverage as a result of our findings.

Over time, you should begin to eliminate certain types of incidents or at least cut down the time required to manage the most frequent types of incidents. That will mean less distracting work.

Hopefully these suggestions help your team think differently about distracting work and the levers to pull to reduce distractions.

Making sense of a disruptive cyber is doable. It may not be easy, but nothing worthwhile is.

Working your way out of disruption and chaos is a decision that is yours to make.

--

--

Opinionated Security
CISO & Cyber Leaders

Tony Grey * CISO for an insurance company * grew team from 3 to 22 * led large software teams at Microsoft * blogs about cyber leadership & program development