A SOC-MSSP guide (2 of 4)

12 min readNov 19, 2023

From reporting malicious activity to catching it as it happens

Part 1: https://medium.com/@7rm1ef8/a-soc-mssp-guide-1-of-4-3f5450638a98
Part 3: https://medium.com/@7rm1ef8/a-soc-mssp-guide-3-of-4-658e0cf99745
Part 4: https://medium.com/@7rm1ef8/a-soc-mssp-guide-4-of-4-a78779d830dd

Summary
3. Financial aspects
3.1. Balance
3.2. Hidden costs and optimizations
3.3. Conclusion

3. Financial aspects

This chapter covers what may well be the hardest part of managing a SOC: making good decisions on where to spend the money. Both SOC and MSSP have limited budgets, in one case it is limited by whatever has been allocated to it and in the other case by its sales revenue and the margin objectives that have been set.

The hard part isn’t to manage to stay within the budget — although it can be challenging at times — the hard part is to carefully plan the spending so that the SOC is actually relevant in its missions long term, mainly when it comes to detecting attacks early to prevent damage. This is hard because it takes a combination of skills and knowledge that are rarely held by one individual:

A strong understanding of the needs and expectations of the executives and/or customers and good communication to explain to them the choices made.
A detailed knowledge of how the SOC in its current state operates, both internally and with other entities — especially its strengths and weaknesses.
A clear vision of what skills and levels each job needs and the actual skills and levels the people holding these jobs have.
A precise knowledge of what profile each member of the SOC has, what they can and can’t do, what they could or couldn’t do and what they would or wouldn’t do.
A deep technical understanding of the mid/long term impact of any decision made, whether that decision regards people, tools, training, or interaction with other entities

In practice, most of the time, the decisions are made by executives with the information at their disposal, which is mostly hearsay about what others do and/or what the customers want. Oftentimes the decisions end up being about buying/using a new tool or adding some junior staff (or firing senior staff in hard times) and that seems logical, because when you spend your time handling money and projecting costs and revenues, the solutions you come up with are money related.
However, these decisions rarely solve anything and often add to the issues: even a very expensive tool will never be used to its full potential/worth its cost with its default configuration, and very good junior staff still misses the experience of senior staff, because experience isn’t something you can learn in books, it’s something you gain with practice.

As seen in the previous chapter, a SOC or MSSP is a complex entity because of its highly technical environment and the human relations it requires to operate properly. Therefore, its actual worth resides within its crew and only by exploiting this wealth — and/or working to grow it — can it be and stay relevant long term. One way to do just that is through industrialization by focusing the knowledge and experience of the senior staff into normalization, standardization and automatization of the processes and tools used by junior staff.

3.1. Balance

In order to establish the best course of action to meet the short term objectives while building for the long term, one can start by thoroughly examining the balance of expenses vs revenue for the SOC or MSSP.
For this exercise to be meaningful and yield actionable results, the different items must be meticulously detailed, especially when it comes to time spent on activities and missions because the goal is to determine what could and should be industrialized and with what priority.

3.1.1. Expenses

The tables below show an example of expenses that are often overlooked in an MSSP environment, but end up costing more the bigger the MSSP is. The list of items gives an idea of what to look for, but is not exhaustive in any way, and although it has been made with an MSSP environment in mind, many items are still relevant for a SOC.

To populate the “average cost of one unit” column, it is preferable to include the workforce cost, the cost of the infrastructure needed to perform the item and the cost of any tools and other licenses needed. The goal here is to get a representation of the actual cost of production for one unit of that item.

It could be obvious, but in case it isn’t: it is good to have approximations for each column of each item, but it is better to have actual metrics and it is best to get those metrics per month to be able to monitor and project evolutions.

Keep in mind that this is not a theoretical exercise and that it has to be done carefully as having a precise view of expenses leads to pinpointing the wastes and minimizing the wastes is a crucial step in a long term plan.

The first table represents immediate and recurrent expenses depending directly on how the SOC/MSSP is structured, the choices of tools and technologies made, the workforce and the contracts with its customers.

Examples of SOC/MSSP immediate and recurrent expenses

The second table lists delayed or indirect costs that should never happen in a perfect world. These are harder to measure in terms of actual cost, but they tend to be way higher when they happen.
This is especially true long term with items like repercussions on the SOC (and company) reputation due to customer dissatisfaction or even more so due to error, incompetence or negligence.

Examples of SOC/MSSP delayed or indirect costs

The tables listing expenses shown above are not exhaustive and are merely pointers to how one should list expenses of one’s SOC or MSSP. Ideally, every task performed by every person in the SOC should have its own entry and be measured as it is the best way to lead to questions like “Why are we doing this?”, “Why is this task taking so much time?” but also “Why are we not doing that?”.
These questions and their corresponding answers should by themselves go a long way towards prioritizing the resources allocated to internal projects and give a clear and precise view of what and where the issues are.

3.1.2. Revenue

The list of revenues for a SOC is often limited to the budget it is allocated by the company.
However, the list for an MSSP can be a bit longer, although its customers’ service subscriptions should make up the bulk of it. Let’s explore a few ideas that could add to the revenue.

The most obvious one depends on the type of subscription or contract the MSSP has with its customers. One-time projects such as adding a new sensor to monitor, creating new detection rules / use cases or upgrading the already existing detection rules and use cases could be sold on top of the subscription. These could also lead to higher subscription revenue because of the extended perimeter monitored. Again, depending on the contracts, things like a complete response (and not just the investigation part) or advanced investigation (e.g. with forensics) could be in the service catalog.

Other ideas that could expand revenue is leasing a dedicated resource to a customer for an agreed upon amount of time, performing external (to the SOC/MSSP or even the company) training or any other creative way to directly sell the skills of the SOC personnel and bettering the SOC or company reputation.

Since most of the revenue for a SOC or MSSP is not controlled by it, in the sense that the SOC isn’t the entity that negotiates the prices with the customers, it is very hard to optimize anything in terms of revenue beyond suggesting new kinds of services like those mentioned above.
However, expenses optimization is a real option to consider in order to create breathing room in the budget.

3.2. Hidden costs and optimizations

Like in most entities, there are costs in a SOC or MSSP that are somewhat hidden and there are some ways to optimize these expenses.
The following points are not specific to a SOC and can be applied to other entities. They do not constitute an exhaustive list either and such items should be carefully considered by establishing a strategy for each, because they have a long term impact on production costs and quality.

Evaluate precisely the cost of training a new recruit for each position in the SOC. This way when someone is asking for a training, a raise or wants another position, it will be very easy to balance the potential loss of this person — individual skills, performances, mindset, internal working knowledge on top of which the cost of recruiting someone new and training him or her — versus the cost of meeting the demands.
In many cases, it is actually far cheaper to negotiate and lean towards the demands than it is to take the loss and hire someone else. This is work, so it boils down to the math of the position’s worth, both for the company and the employee and there shouldn’t be any emotions, hurt feelings or letting things get personal here.
This circles back to the previous chapter about knowing the people working in the SOC and their profiles, with an added cost/benefit approach that could very well be explained to the person at risk: “From what you have been doing over the past N months, we calculated that your work is worth X to us, therefore we can go up to that point, but not over to meet the Y for which you asked”.
Analyze the costs of security solutions and their actual added benefit as-is — i.e. with the current skills and tasks assignments of the SOC analysts — and compare that with what could be achieved towards the SOC priorities if the budget for the overpriced/underused solutions were redirected on other (internal) investments. Of course, the Return On Investment (ROI) should be taken into account for this comparison, both for the security solution and the other (internal) project.
The goal here is to stop paying for things that are barely used or with little added value, if that budget item could be better allocated to some other project that would enhance the SOC production, either by increasing its quantity or quality or by reducing wasted expenses.
Compare the cost of having teams (mainly those in charge of response and MRO) 24/7 available in shifts versus that of having some members of these teams doing on-call duty. There are a lot of variables to take into account here and not just flat costs: the employees may be more productive, happier and on a better mindset if they are working, even in rotations, than if they get called regularly at night during their on-call duty and then have to work their regular shift the next day.
A department that is available 24/7 would also be able to answer in case a customer has made an on-call duty subscription — and even faster than with regular on-call duty — and they also could work on any alert or incident that happened outside of regular working hours, therefore lightening the workload for the regular day shift. Of course in that case, if the customer hasn’t subscribed to a 24/7 monitoring, the notifications would be (automatically) postponed to the start of the day shift.
Build a table that lists every task of every job in the SOC, the number of units to produce every week (or day or month), the average time it takes to complete, the total time needed, the cost per hour and total cost all depending on experience (junior, full or senior) as per the example below:

Examples of SOC/MSSP tasks, time taken and cost depending on employee’s experience

This is the baseline on which to build simulations of staff needed by level of experience. Some tasks will be limited to full and senior or senior only and some others would rather have junior or full handle them as the cost would be lower. Therefore, it will help simulate how many people with what level of experience would be the best for a particular team/job in the SOC to optimize cost efficiency (adding other columns like FTE, revenue generated by task, etc would come in handy for that matter).
However make no mistake, these are actual people behind the numbers and those people have preferences in jobs and tasks, wishes in terms of career evolution, life goals, etc.Therefore, the ratios (FTEs per experience level) simulated will always be unattainable goals and should be treated as such: a direction rather than an objective. This circles back to a previous point comparing the cost of losing a person and recruiting a new one versus that of meeting the demands.

All of these tables and graphs take a lot of work to build, especially if the data doesn’t exist at first and has to be created. Also that work may be tedious, but it is well worth it.
Firstly, it paints an accurate picture of the state of the SOC and gives a lot of vision, if not control, to the SOC manager about the health of his or her SOC.
Secondly, it translates technical and operational feelings of what works well and what does not work in the SOC into actual, factual numbers and costs that are very well understood as-is by executives. Instead of expecting executives to spare the time to learn and understand the technical and operational aspects of the SOC, it is up to the SOC manager to speak their language in order to be heard and understood. This is one way to do just that.

Since a SOC has limited options to generate more revenues, most of the effort regarding financial optimization should be on reducing expenses. Therefore, the first step is to understand where the highest expenditures for the SOC or MSSP are, and this is precisely where all the suggested tables and graphs come in.
Usually, the two top items are the salaries of the personnel and the tools (licenses) used by the SOC.
The licenses for the tools could be negotiated with the vendors, and a regular review of tools’ usage should take place to ascertain that they are the correct tools to answer the needs, and that there isn’t another (cheaper) tool that could be more suited. There can be some gain there, but ultimately, people need proper tools to do a proper job.
On the other hand, from a purely economic point of view, it would be great to have the experience and know-how of senior staff at the price of a junior one. Obviously, this isn’t happening, but what can happen is using the time of senior staff more wisely to enable junior staff to approach the work they do in terms of quality and quantity. In other words, senior staff can be tasked to create documentation, processes and procedures, all kinds of task automatization, and finally monitor the work done by other less experienced personnel. This would set standards and enable everyone to achieve them by using the knowledge and methods produced by experienced people. This way it requires fewer people with less experience to achieve the same quality and quantity of work. Therefore, depending on the situation — SOC or MSSP, expected growth or not — fewer people are needed for the same perimeter to monitor or the same people could monitor a larger perimeter.

3.3. Conclusion

The best way for a SOC and especially an MSSP to improve production stability, resiliency, quality and efficiency is to invest in industrialization. This mainly means identifying the key people who are able to do that job and making sure to keep them until the goal in level of industrialization is reached.
The tools are important, but having the right people on the right jobs with the right level of skills matters more: in case of attacks or other cybersecurity crises, skilled analysts with lesser tools will manage way better than poorly trained and inexperienced analysts with top of the shelf tools.

All in all, the best decisions in terms of spending are most of the time about enabling the SOC to accomplish its missions better, faster and more reliably by investing in its production capabilities rather than acquiring an overpriced underused tool or adding new members to the overflowing pool of juniors.