What would be a disruptive loss in DFIR?

I attended the DFIR Summit in Austin, TX this week and had the privilege of meeting Andy Rosen and hearing him speak about the history of digital forensics. One of the questions he was asked to address in his talk is what he sees as potential upcoming changes/challenges in the field. After briefly discussing the impact of disruptive technologies, he raised an insightful point about losing relied-upon technology (i.e., having the proverbial rug pulled out from underneath you).

Five frontiers of competition

His discussion of disruptive technology reminded me of an older (2005) article by mengwong entitled, Strategic Advice for Spammers. The author explains that “as an industry matures, competition moves along five frontiers:

  • “functionality (can we get the damn thing to work at all?)
  • reliability (will the damn thing please stop crashing?)
  • convenience (let’s shrink it so i can take it with me.)
  • price (if it’s a commodity, give me the cheapest)
  • fashion (indigo or graphite? hey, maybe key lime.)”

According to mengwong, “Only after one frontier is crossed does a market focus on dimensions relevant to the next.”

Convenient DFIR automation

In the DFIR community, the current focus (arms race?) seems to be automation. While some are further along than others in this arena, I see the overall current competitive frontier as convenience. While some tool developers believe they already have this figured out and are focusing on price and fashion, new research and open source tools are daily exposing their shortcomings and showing “better ways of doing it”, continually increasing functionality and reliability.

Convenience is a touchy subject in this field because of the disdained impulse for an “easy button”. In the past, attempts to automate analysis have been derided with slurs such as “Nintendo forensics” and building a “Find Evil button”. And yet, the need for scaling DFIR is imminent. This can lead to cognitive dissonance for how practitioners and researchers view some of the automated approaches.

On the one hand, we need to scale our response/analysis efforts, which requires automation. On the other, we don’t want lazy/ignorant forensic examiners and incident responders/handlers who just do “push-button forensics.”

Advice for tool developers

I think there are a handful of ways that tool developers can address these concerns while creating helpful automation utilities:

  1. Don’t promise more than you can deliver (keep the marketing hype in check)
  2. Educate examiners about how your tool works (open source, whitepapers, quality documentation, etc.)
  3. Don’t ignore data you don’t understand (alert the examiner when data is not parsed, an error occurs, or results are ambiguous)

Advice for examiners

The onus ultimately falls on the examiner. While tools may over-promise and under-deliver, obfuscate findings or omissions, or mystify the process of parsing an artifact, it is ultimately the examiner’s job to collect, process, analyze, and present the data/findings. Examiners will do well to keep in mind:

  1. Validate all findings (“trust but verify”)
  2. Learn what the tool is doing and how it works (the $MFT is not parsed by a pure flippin’ magic subroutine)
  3. Test the tool and compare results to others to learn its capabilities, quirks, and shortcomings (and find others who have done this such as NIST)
  4. Expect potholes in the road (real-world data is messy, your machine needs a reboot, and your weekend of processing was just interrupted by a Windows 10 upgrade notification)

Back to the disruptive loss

So what does any of this have to do with disruptive losses in the DFIR community? Consider this question:

What if <insert backbone technology here> fell by the wayside or ceased to be free/cost-effective? (e.g. popular code libraries/APIs, Elasticsearch/Kibana)

Think of software companies that exclusively wrote code for the Motorola PowerPC platform who perished as the Intel chip eventually conquered the market. The Gartner Hype Cycle shows hundreds of technologies that have seen better days.

Or think of the recent example where a disgruntled developer deleted a small project (only 11 lines of code!) from the node.js package manager (npm) and broke hundreds of apps. The DFIR software development community relies on hundreds of open source Python packages being available in PyPi (“pip install crucial-lib”), aptitude package manager (APT), etc. Should one of these go missing it could potentially impact numerous tools. But more dangerously, should one of these libraries be altered in a subtle way to where it impacts important tool functionality, this could mean missing evidence (this is why good development practices are vital such as using virtual environments and building a requirements file with versions specified for dependencies in Python development, for example).

Many of the tools in DFIR are built on top of a myriad of other libraries, tools, and technologies, and often by novice developers (myself included!) who often poorly document their code and dependencies. While this is not entirely a bad thing (more code and community participation is awesome!), it increases the need for continual tool validation and testing and potentially creates large blind spots where a disruptive loss may go unnoticed. Such losses would cause setbacks by causing tools to regress to the functionality and reliability frontiers.

What do you think? Am I making any sense? What do you think would be “disruptive losses” to the DFIR community?