Chronicle Road to Detection: Context — Part 1 of 3

Anton Chuvakin
Feb 26 · 3 min read

Note: Yes, this is written while wearing my vendor hat. But do keep in mind that I only work on things I believe in! So, don’t knock that hat off my head :-)

Today, security operations and detection/response is largely about visibility and data — security telemetry data. Such data allows us to know what is going on in our ever-expanding environments and ultimately enables the much-sought-after situational awareness.

While many assume that this means security is about having a lot of data or even big data, it is really not. It is about data that can be used for detection, alert triage, response, threat hunting and other tasks.

In essence, this means it’s not just the quantity of data, but also the quality — consistency, structure, fidelity — of data that creates a clear signal about the attacker’s present and past behaviors. In fact, if more data translates into more noise, it is of no help. If more data translates into the user needing to run more searches, it is again of no help. If more data means that you have to pay dramatically more without a clear increase of security insight, it is the opposite of help again...

The other critical component, and the one least under a security vendor’s control, is visibility. Visibility is composed of the various windows of insight which generate telemetry from within an enterprise network. These may be traditional log sources, network and endpoint sensors and other data.

Typically, aspects of visibility are from discrete points of view and therefore offer only a partial picture of activity occurring within an enterprise environment (for example, only firewall logs will give you a skewed picture most of the time). This “pre-selection” is typically a side effect of rate-based or quota-based pricing, making the ability to store all the necessary data logs unaffordable.

But storage is only part of the problem. Accessibility of security data is the focus point upon which a success in enterprise defense program hinges on. Without a mechanism to access log data, no amount of storage is going to make an organization safer if said organization cannot slice and dice or otherwise get at the stored data.

So how does Chronicle solves the critical challenges of data and visibility:

Indeed, successful detection and investigation tools are first about being able to collect and retain visibility data, without incurring an inordinate cost. Building a platform that can scale to petabytes is not that difficult in this public cloud age. However, creating such a platform that would not cost millions for nearly every organization is dramatically more difficult.

However, the visibility is not the same as threat detection — you need to actively surface the malicious and suspicious activity and present them to the security team.

This brings us to the next phase for Chronicle. The time has come for the platform to evolve to detection — to uncover new threats, both in real time and historically.

How do we detect? Wait for Part 2 tomorrow.

(written together with Brandon Levene)

Anton on Security

A new start for my security blog

Anton Chuvakin

Written by

See www.chuvakin.org

Anton on Security

A new start for my security blog

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade