Simple Security
Published in

Simple Security

The FBI called and said I’d been hacked …

It seemed like a pretty typical day until Agent Smith* from the FBI called to tell me that our server had been hacked. *Agent Smith’s name has been changed to protect his identity, but his real name was just as unbelievable. He waited a moment to let me digest the news, and then added “your site is a target. It has either been compromised or very soon will be”.

I was skeptical.

Our site was running a small web server for an embedded software company Embedthis Software. We’d been pretty careful. It was running a customized embedded web server, and I knew that normal web-server exploits would not work. Furthermore, the server was fully patched, and running a minimal recent Linux distribution. It should be safe.

However, I was also aware that this server was hosting and distributing software that is used in hundreds of millions of embedded devices. Certainly, it would offer an attractive target in which to insert a few malicious lines of code.

But, come on! The guy said his name was Agent Smith. This sounded totally bogus! It had to be a social engineering ploy.

Still, I had no choice but to hear him out. Agent Smith repeated that he wanted to know if we’d been hacked recently. I asked if he could give me more details, but he did not want to disclose anything more over the phone. Agent Smith was being careful. I was being careful too. We had a standoff.

To resolve the situation, Agent Smith suggested I go to, call the listed phone number at his regional office, and ask to be put through to him. I did. To my surprise, Agent Smith answered the phone. I was gobsmacked.

It was clear this was a threat that we needed to take seriously. It was time for a full investigation of our site.

The O/S looked fine, none of the core utilities had been modified, log files were intact, and there were no suspicious root kits. Our software and the web server had not been touched. There were no hidden files, no recently modified files, or permissions, and no mysterious programs. In short, we could not detect a single speck of evidence of an attack. We had no indication that the server had been hacked, and yet we no longer had confidence in its integrity.

Out of an abundance of caution, we prepared a new server and decommissioned the old one. I went home and slept well, confident the threat had been erased.

Or had it?

We never found out for sure, and long after that implausible but very real call from the FBI, I would wonder, “had the server really been hacked or not, and is it safe now”?

Ignorance isn’t bliss

The problem is that it is immensely difficult to prove a negative. The server hasn’t been hacked, but can you know for sure?

You can do all the right things:

  • Patch quickly and regularly
  • Run with minimum privilege
  • Lock down ports, firewall and use a minimal distribution
  • Capture and actively monitor logs
  • Install IDS tools
  • etc, etc.

But can you know for sure?

This question would nag at me for years. I always felt I was playing catch-up. We did not have a dedicated security team, so I would patch, but only after zero-days were already active in the wild. I would update IDS signatures, but only after security vendors had seen the exploit and had taken their time to prepare signatures.

I knew there was no universal panacea. Security is an endless, vigilant race against threats. But I wanted to answer three simple questions:

  1. Have I been hacked?
  2. Am I being attacked?
  3. Where am I vulnerable?

I want to end the endless manual checking. I want 24/7 peace of mind, and most of all, I want no more calls from Agent Smith.

Later I would learn that an excellent solution would be to use Immutable Infrastructure so that my server cannot be modified as easily in the first place.

Learn More

Setup PowerDown’s transparent spot management in just five clicks.

Go to to learn more.




Recommended from Medium

Protecting E-commerce Firms from Credential Stuffing and Credential Cracking

Data rights is a human right

The Great Hack movie

Can big data and machine learning predict the next cyber attack ?

Free Wallpaper Downloads For Samsung Phones

{UPDATE} Tuk Auto Auto Riksja Hack Free Resources Generator

lrn2hack4real: why virtual cyber ranges are lame

WildPressure APT Equips with New Malware to Targets Both Windows and MacOS. — CyberWorkx

Closing The Gender Gap May Depend On Male Players Within Cyber Diversity Initiatives

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Michael O'Brien

Michael O'Brien

Founder and CEO, SenseDeep.

More from Medium

OverTheWire:~$ Bandit Level 25 → 26 → 27

Compliance checks come to windows

Tails: The Most Secure Operation System

What is Cybersecurity? Can you protect yourself?