Cycle Patterns in Cybersecurity Management

Find the stuff that need fixing and fix it

Photo by Joel Fulgencio on Unsplash

Doing valuable work always generates some deliverable, it can be document, an email, a ticket, a chat message, a report, changes in a database, etc.

In practice, most day in day out valuable cybersecurity work follows the same pattern:

• Find stuff that may have security flaws to fix
• Fix the security flaws

Both the finding and the fixing is documented in deliverables.

Security flaws are called weaknesses, weaknesses that can be found in Applications, Networks and Systems. An incomplete list of stuff that may have weaknesses, divided among Applications, Networks and Systems follows:

• Applications: Source Code, Domains, URLs, Libraries, Containers, Packet Managers, Databases, Frontend Applications, Native Apps, Backend Applications, APIs, LAMP/SMACK stack
• Networks: IP Stack including DNS, internal and external IP ranges, Network Stack (e.g. Ethernet), VPN, WiFi, Routers, Switches, APs, FirewallsI, DS/IPS, WAF
• Systems: Operating Systems, Hypervisors, Containers, Orchestration, Cloud Infrastructure Instances, Cloud Storage Services, Directories, Password Managers, Email, Chat, PKIs and Digital Certificates, AntiMalware solutions, Personal computers and mobile devices, Servers, SIEM, Encryption and Tokenisation, Licenses, Proxy, NTP and Synchronization Services

Note: I will not go into details now, but risks, incidents, standard/regulation compliance and third parties follow the same management pattern as well.

Now, all this finding and fixing, documented in deliverables suffers of the characteristic of becoming stale fast. As soon as the finding and the fixing is completed, reality moves on and there is more finding and fixing to do.

That is why the patterns are cycles, long or short cycles, but cycles nevertheless because the value of the deliverables just does not last. Pentest report of last week? Probably still good. Three months report? Would not bet yesterday’s doughnut on it. Every cycle is a feedback loop where we examine the results of decision and take corrective or improvement action when necessary.

Risk assessment and Compliance feedback loops are normally long, with a period of a year, unlike Continuos improvement feedback loops that are one month long and sometimes event shorter. In some literature you will find this referenced as “Fail fast”. If what we are doing is not working, we need to find out as soon as possible, and that is good. “Fail fast” creates many opportunities to understand if we are getting the expected results and introduce changes when necessary.

Before describing the cybersecurity cycles, two observations about deliverables:

• Having deliverables makes crystal clear what is the distribution of responsibilities. I create the deliverable, you receive it. My jobs ends when I send it, your job starts when you get it.
• Counting deliverables is the most effective cybersecurity metric, because value is created at the point in time when the deliverable is completed and sent to the recipient. (Note: A metric is a quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements.)

A single overall cycle looks pretty much like this,

During a cycle:

1. Work, performed according to policies and procedures is documented in tickets.
2. Metrics are collated, archived and represented in Reports
3. Reports provide an interpretation of the meaning of the metrics, giving situational awareness of the situation for managers (aka, are there any problems?)
4. Review of the results of past decisions, if they had the expected effects or not (aka validate hypothesis), found in the previous meeting minutes
5. Decisions to introduce new fixes or improvements, and their expected effects (aka formulate hypothesis), recorded in the meeting minutes
6. Fixes or improvements will normally lead to changes in policies and procedures, that document how work is carried out.
7. And the cycle starts again.

The X in the diagram above can be any of the following cycles

1. Discovery
2. Prioritization
3. Verification
4. Remediation
5. Knowledge Management
6. Compliance
7. Activity
8. Quality
9. Resources

Whereas there are nine cycles, most organizations with low maturity only implement some of them. Let’s examine every cycle in some detail

Discovery Cycle

During this cycle we find and record all assets (Applications, Systems, Networks) that the organization owns or controls. We can’t protect what we don’t know we have, so this cycle is critical for the success of the rest of the cybersecurity management cycles. How frequently this cycle runs depends on how automated it is and the size of the organization. The list of assets is normally recorded in a database or list depending on the size of the organization.

Prioritization Cycle

During this cycle we classify the discovered assets depending on the risk they represent for the security of the organizations. Other cycles will perform more work on high priority assets, optimizing the use of resources. The following sample report has the following breakdown:

1. Priority 1: Financial application visible from Internet
2. Priority 2: Applications visible from Internet
3. Priority 3: Financial applications that are used internally only.
4. Priority 4: Applications that are used internally only.

Prioritization is best used for assets, not weaknesses.

Verification Cycle

During this cycle we verify/check/pentest the assets and list the weaknesses found. There are many taxonomies (OWASP, CVE’s, CIS, etc) that are relevant to this cycle. This cycle frequency is normally in lock step with remediation, as checking for weaknesses faster than they are fixed does not bring a great return of investment. The following sample report reflects how much verification work has been performed on different types of assets:

Remediation Cycle

During this cycle we fix the weaknesses found. This cycle frequency is normally in lock step with verification, and every remediation instance needs to be re-tested to make sure it is effectively remediated. The following reports reflects how much remediation work has been performed on different types of assets, with a twist, as the metric used is not weaknesses but weaknesses multiplied by days, making the metric worse (higher) as long a weakness is not fixed (Why this sample report calls weaknesses “riesgos de seguridad” is a looong story, better told some other time):

break down by team

break down by type of weakness

Knowledge Management Cycle

During this cycle we implement necessary changes in policies and procedures. This cycle frequency is in lock step with how frequently reports are generated. Updated in policies and procedures will implement changes and improvements, from different sources like meeting minutes where other cycle reports are reviewed, lessons learnt from incidents, or fixing known non-compliances. The following report reflects how much document work has been performed bringing documents along their lifecycle:

Compliance Management Cycle

During this cycle we implement necessary changes in policies and procedures in order to meet requirements of standards and other regulations. We also collaborate with internal and external auditors providing evidence of compliance. Non-compliances follow the same lifecycle pattern weaknesses do. This cycle frequency is in lock step with how frequently audits are performed.

Activity Management Cycle

During this cycle we keep track of all other cycles frequency and assignment of tasks. This cycle frequency is in lock step with how frequently reports are generated. Planning work and forecasting performance is the main output of this cycle. The following sample report reflects if the verification cycle is performed according to agreed targets for assets of different priority:

Quality Management Cycle

During this cycle we keep track of how fit for purpose is the verification cycle, and run tests for the rest of the cycles. Some obvious metrics are false positives, false negatives and number of weakness found per asset during each verification cycle, as in the reports below. Customer satisfaction is another metric that is relevant for this cycle, but only organizations with a really high maturity use it.

Resources Management Cycle

During this cycle we keep track the use of money and man/hours. This cycle frequency is in lock step with budgets, so yearly is probably the norm. Using this cycle we can inform the prioritization cycle and save resources while keeping performance, or we can use it to compare return of investment of different approaches when verifying and remediating weaknesses. The following sample report reflects the yearly evolution of cost per unit of work.

Symptoms

These are the most common symptoms of not using these cycles:

1. When certain people go on leave or get sick, performance is affected.
2. Audits are painful and it takes a significant effort to pass successfully.
3. Changes in the ways things are done are difficult and slow to implement.
4. The same errors are made over and over again.
5. More than 20% of the time of the team is used trying to determine what to do or how to do it.
6. It is no infrequent to enter discussions with other teams about who is responsible for what.
7. The reports used do not reflect the performance of the team or the level of security.
8. Magic bullets are tried by management on a monthly basis and forgotten shortly after.
9. New software, was supposed to solve all management issues. Instead, it has introduced issues of its own.

Conclusion

Cybersecurity management is quite complex, but there are patterns we can follow to simplify it.

There are some common symptoms of theses cycles not being used.

If you don’t have reports that reflect the activity of cybersecurity management cycles, or you don’t formulate and validate hypothesis about your decisions, you have work to do to implement continuous improvement.

To learn more

This article is part of a series that starts here: Principles of Evidence Based Cybersecurity Management