Chronicle Road to Detection: Context — Part 1 of 3
Note: Yes, this is written while wearing my vendor hat. But do keep in mind that I only work on things I believe in! So, don’t knock that hat off my head :-) You can also find a cleaned-up full whitepaper version of this series at https://hubs.ly/H0pvD_30
Today, security operations and detection/response is largely about visibility and data — security telemetry data. Such data allows us to know what is going on in our ever-expanding environments and ultimately enables the much-sought-after situational awareness.
While many assume that this means security is about having a lot of data or even big data, it is really not. It is about data that can be used for detection, alert triage, response, threat hunting and other tasks.
In essence, this means it’s not just the quantity of data, but also the quality — consistency, structure, fidelity — of data that creates a clear signal about the attacker’s present and past behaviors. In fact, if more data translates into more noise, it is of no help. If more data translates into the user needing to run more searches, it is again of no help. If more data means that you have to pay dramatically more without a clear increase of security insight, it is the opposite of help again...
The other critical component, and the one least under a security vendor’s control, is visibility. Visibility is composed of the various windows of insight which generate telemetry from within an enterprise network. These may be traditional log sources, network and endpoint sensors and other data.
Typically, aspects of visibility are from discrete points of view and therefore offer only a partial picture of activity occurring within an enterprise environment (for example, only firewall logs will give you a skewed picture most of the time). This “pre-selection” is typically a side effect of rate-based or quota-based pricing, making the ability to store all the necessary data logs unaffordable.
But storage is only part of the problem. Accessibility of security data is the focus point upon which a success in enterprise defense program hinges on. Without a mechanism to access log data, no amount of storage is going to make an organization safer if said organization cannot slice and dice or otherwise get at the stored data.
So how does Chronicle solves the critical challenges of data and visibility:
- It offers defenders the opportunity to maximize the visibility data they can access by not requiring pre-selection of data sources.
- Our predictable pricing model (per employee per year) allows defenders to not have to sacrifice aspects of their visibility due to cost of storage or access.
- Under the hood, Chronicle offers huge amounts of compute power allocated to data manipulation operations and queries.
Indeed, successful detection and investigation tools are first about being able to collect and retain visibility data, without incurring an inordinate cost. Building a platform that can scale to petabytes is not that difficult in this public cloud age. However, creating such a platform that would not cost millions for nearly every organization is dramatically more difficult.
However, the visibility is not the same as threat detection — you need to actively surface the malicious and suspicious activity and present them to the security team.
This brings us to the next phase for Chronicle. The time has come for the platform to evolve to detection — to uncover new threats, both in real time and historically.
How do we detect? Wait for Part 2 tomorrow.
(written together with Brandon Levene)
