How to Deal when High Estimates Delay Security Remediations
There’s no way a fix is that hard, right?
One frustration I frequently hear when I talk to others in the industry is that when InfoSec teams bring large, complex security concerns to app teams, the developers estimate the project timeline to be so large that remediations aren’t prioritized until months later. Even worse, the estimates seem unfathomably large, even taking into consideration the research, coding, testing, and deployment. Distrust festers within the InfoSec team.
What gives?
A peek into the estimation process
Software development teams provide dozens of estimates a year, most of which I bet are inaccurate. Some attribute this to the overly optimistic personalities of engineers who are prone to underestimate the required effort, but the truth lies more in the estimation process than the calculation abilities of the engineers.
Based upon decades of research on software development, agile thought leader Steve McConnell created “The Cone of Uncertainty.” The Cone of Uncertainty is a graph that shows the variability of an estimate given the time in the project lifecycle when the estimate is delivered.
For example, based on his datasets, McConnell found that if an estimate was given at the time of initial concept, the project’s true timeline could vary by as much as 4 times. To put that in perspective, if a team provided a project estimate of 6 months, the real timeline could vary anywhere between 6 weeks or 2 years.
However, as teams removed uncertainty by solidifying their requirements, UX designs, and technical designs, they could estimate more accurately.
The problem is that if you are an Information Security professional bringing a large concern to the application team (more than just static code analysis or pentest results), the effort is still in the initial concept phase. There is a problem statement, but little else. Then we request a remediation date, leaving the team to estimate in a wide degree of uncertainty.
Who Cares? It’s just an estimate.
The accuracy of estimates poses a significant challenge to businesses, as many organizations conflate estimates and commitments. When low-grade estimates become timeline commitments, late software delivery negatively impacts product launches, client implementations, revenue recognition, sales quotas, contractual obligations, or customer satisfaction.
While app teams could ideally reserve estimate delivery until later in the development lifecycle, when given little choice, many respond by producing high-confidence estimates, which looks like this:
Using our initial example, the app team’s logical estimate may be 6 months, with a 50% degree of confidence. However, if they fear the consequences of being inaccurate, they may provide an estimate ranging from 12 to 24 months to minimize their risk of being wrong.
One wrong estimate? Big deal.
This estimation problem compounds when you consider an entire product roadmap. To illustrate, this development team was projected to deliver 3 features by the end of Q4.
However, if Feature 1 takes considerably longer than expected, the entire roadmap is impacted.
Now this team will be lucky if they deliver 2 features this year, which is only possible if Feature 2 doesn’t take longer than anticipated. In this case, Feature 4 was deprioritized altogether.
At first, this may simply seem to be the nature of business and software development, but when revenue or customer satisfaction is at risk, the stakes become too high to underestimate. To make it worse, the stakes rise significantly when they’re estimating a security initiative.
Why is underestimation worse for security remediations?
To our chagrin, many organizations place a low ROI on security efforts. They don’t produce revenue, and at best, they achieve cost avoidance. Development teams recognize that it is extremely unpalatable to their leaders if long-running security projects delay roadmap features and result in bottom-line impacts.
That means that not only are development teams already conditioned to estimate projects high, but they’re trained to add in even more buffer to achieve a high-confidence projection. Whereas a 75% confidence estimate may be par for the course for a revenue-generating feature, the developers may require 90% confidence for security or other technical initiatives.
So now what?
Now you may understand an app team’s proclivity to estimate conservatively, but you still don’t want to wait months for security remediations to be implemented.
So how can security teams and app teams work together to minimize the uncertainty that delays security remediations? Let’s take a look.
Example: Insufficient Logging and Monitoring
For our discussion, let’s assume the technical gap is insufficient logging and monitoring across an entire application. As a former dev manager, here is how I might have estimated this project if I lacked understanding of the issue and was put on the spot without time for my team to experiment (yes, I know it’s bad!):
As a dev manager, I would realize that this estimate seems high, but there are a lot of unknowns. This seems like a reasonably justified, well-calculated estimate that would be unlikely to extend beyond that timeline and delay the development of other roadmap features.
With an estimate this large, however, my product owner might not be inclined to prioritize logging and monitoring ahead of other planned roadmap features. Therefore, maybe the project start date will be forecasted on the roadmap for 8 months out, and using two developers, would be targeting completion 16 months from now. That’s a long time to wait.
However, there are three strategies you can employ as an InfoSec professional that may help you avoid this situation.
Strategy #1: Make the scope as small as possible
Instead of attempting to prioritize the entire logging and monitoring behemoth, one viable approach is to prioritize logging and monitoring in just one simple but high-priority area. For example, perhaps first you would target the login process.
The estimate is less than 10% of the original, albeit more work will follow afterwards. This is huge! There are three major benefits to this strategy:
- With the scope much more defined, the value of this effort is more clearly articulated. Now the team can have visibility into attacks like brute force attempts, password sprays, or credential stuffing, whereas they did not have that before. This becomes a much easier sell to a product manager because it’s clearer than “we’ll be able to know some stuff,” which was essentially the original message.
- The small 2–4 week size allows it to be squeezed into a product roadmap much earlier than a multi-month endeavor. Instead of waiting for months to see any logging and monitoring, you will likely see progress much faster.
- When it comes time to estimate the next logging and monitoring initiative, the app team’s estimates are likely to be much more accurate and include far fewer buffers. This is because they’ve removed many of the unknowns, and they’ve progressed through the Cone of Uncertainty. Perhaps they find that they only need 1 hour/file on average. That has a tremendous impact on the overall estimate.
Strategy #2: Reduce the Uncertainty
If strategy #1 isn’t an option but you’d still like the benefit of removing the uncertainty that is inflating the estimate, then see if the app team can give up 1 developer for 1 day to create a proof of concept. After 1 day, they should know how far they have gotten, what they inadvertently broke and need to account for, and have a better sense of how long the project will take. Perhaps after this one day using our example, they realize just as before that they only need 1 hour/file on average. This significantly reduces the initial estimate:
Whereas the app team may not be able to give up several weeks without impacting existing projects, they may be able to relinquish one person for a single day to reduce the uncertainty in their estimate.
Strategy 3: Plan for a Plan
Finally, if those options don’t work, one last option is to plan for a plan. You may agree that in one week, you will reconvene with the app team to discuss the timeline. This gives the team some breathing room to investigate and learn more about the project. This not only reduces conversational friction, but it has the same benefit of moving them through the Cone of Uncertainty to give them more confidence in their estimate. Sometimes waiting one week, while it feels like forever, will yield a much more favorable estimate that is more likely to get prioritized.
But know there is no silver-bullet for reducing estimates
One caveat is that while an app team may progress through the cone of uncertainty, there is no guarantee they will produce a lower estimate than they did originally. Perhaps if they run a proof of concept, they break critical processes and identify complexities that increase the project estimate. Your goal is not to browbeat the development team into producing a lower estimate, but to ensure that the estimate provided is as realistic as possible and isn’t needlessly (but understandably) inflated.
If a project truly does require a lot of time, it’s better for everyone if the project schedule allows for it. At least then, you’ll be more confident that the long wait is warranted.
