Report: Critical infrastructure vulnerabilities at record levels

It’s the classic good-news / bad-news story about internet connectivity. The convenience is seductive. So are the cost savings. Increased efficiency makes good business sense. But the risks of being online can undermine all those advantages, especially if an organization fails to prepare for them.

And that’s the story of industrial control systems (ICSs), which run much of the nation’s critical infrastructure — electricity and other energy sources, water, sewer, transportation, and more.

Claroty, an industrial cybersecurity company, in its most recent biannual ICS risk and vulnerability report, wrote that “assets are exposed online in record numbers, and along with them, all their blemishes: unpatched vulnerabilities, unsecured credentials, weak configurations, and the use of outdated industrial protocols.”

All of which amount to the digital version of an organization leaving the door, the vault, and all the file cabinets wide open.

Claroty reported that the number of vulnerabilities discovered in ICSs during the first half of 2021 was up 41% from the previous six months, to 637, affecting 76 vendors. More ominously, a majority of those vulnerabilities were ranked as high or critical severity, required low attack complexity, were remotely exploitable, didn’t require access privileges, and could cause a total loss of ICS availability.

It’s not just researchers who notice such things, of course. Cyber criminals do too. The report doesn’t have statistics on how many attacks attempted to exploit those vulnerabilities, but did cite several that got international attention this year, including ransomware attacks against Colonial Pipeline and JBS Foods, and an attempt to poison the drinking water of Oldsmar, Florida.

But even if attacks are not yet as rampant as vulnerabilities, the impact of those on Colonial and JBS was significant — major fuel and food supply disruptions that led to price spikes and millions in payments.

And they have been serious enough to generate high-profile attention from the federal government.

• The Department of Homeland Security’s Transportation Security Administration (TSA) issued a security directive in July “that requires owners and operators of TSA-designated critical pipelines that transport hazardous liquids and natural gas to implement a number of urgently needed protections against cyber intrusions.” That directive followed one in May that required operators of critical pipelines to “(1) report confirmed and potential cybersecurity incidents to CISA; (2) designate a Cybersecurity Coordinator to be available 24 hours a day, seven days a week; (3) review current practices; and, (4) identify any gaps and related remediation measures to address cyber-related risks.”
• In June, the federal Cybersecurity & Infrastructure Security Agency (CISA) published “Rising Ransomware Threat to Operational Technology Assets,” a fact sheet detailing the threat to “operational technology assets and control systems.”
• President Biden issued a National Security Memorandum on July 28, establishing an Industrial Control Systems Cybersecurity Initiative, described as “a voluntary, collaborative effort between the federal government and the critical infrastructure community to significantly improve the cybersecurity of these critical systems.” The goal, he said, is to encourage the use of “technologies and systems that provide threat visibility, indications, detection, and warnings, and that facilitate response capabilities for cybersecurity in essential control system and operational technology networks.”
• Just this past week, at a White House meeting with Biden, the heads of some of the biggest tech companies in the world pledged to spend billions on cybersecurity training and to improve security technology. The White House said Biden called the meeting to discuss the efforts of private-sector critical infrastructure entities in the banking, energy, and water utility industries to improve cybersecurity and collaborations with the government. Apple’s Tim Cook, Microsoft’s Satya Nadella, Amazon’s Andy Jassy, Alphabet’s (Google parent company) Sundar Pichai, and IBM’s Arvind Krishna were among those present.

After the meeting, the White House said the National Institute of Standards and Technology would work with the private sector on new guidelines for building secure technology and assessing the security of technology, including open source software.

But then, it was more than eight years ago that President Obama issued an Executive Order on improving critical infrastructure cybersecurity. And the problem has only become worse since then.

Joel Brenner, a former senior counsel and inspector general at the National Security Agency, in a March 2017 report titled “Keeping America Safe: Toward More Secure Networks for Critical Sectors” wrote, “The digital systems that control critical infrastructure in the United States and most other countries are easily penetrated and architecturally weak, and we have known it for a long time.”

In a later blog post, he declared, “The White House has been issuing ineffective directives addressing critical networks like clockwork since the ’90s.”

A heavy legacy lift

So why haven’t there already been massive improvements in ICS security? Because it’s a difficult and complicated problem to solve.

For starters, much of the nation’s critical infrastructure is what security experts euphemistically label “legacy.” That’s not a compliment. It means that while it was designed to function safely and for a very long time — decades — it wasn’t originally designed to be connected to the internet.

In the past, a low-tech but effective way to deal with that was the air gap — keeping the operational part of ICSs off the internet and also disconnected from any other computers that are on the internet, like information technology (IT) networks.

While an air gap doesn’t guarantee security, it makes it a lot harder for malware to jump from one system to another.

As Security Week put it nearly three years ago, the use of the air gap meant that “factories and shipyards were more or less immune to cyber attack.”

But even then, the air gap was eroding or disappearing entirely, thanks to increasingly intertwined operational technology (OT) and IT. And with the air gap gone, ICS vulnerabilities are more easily exploited remotely by hackers, as the Colonial Pipeline and JBS attacks demonstrate.

So why not just update and patch vulnerabilities in ICSs? That’s complicated too — vastly more complicated and expensive than downloading a free patch for an app on your phone or laptop. It usually means getting the vendor of the system to install the patch and retest the system to make sure it works. Many will charge for it.

It can also be a scheduling nightmare — preplanning as much as six months ahead to take down a system in a very narrow window of time. Indeed, Claroty researcher Chen Fradkin acknowledged in a blog post that “patching and product updates require downtime that’s intolerable in many arenas.”

Deep technical debt

Finally, sometimes patches can be incompatible with an older operating system.

Sammy Migues, principal scientist at the Synopsys Software Integrity Group, said ICS security is a hard problem in large part due to a massive amount of “technical debt” — the money it would take to bring ICSs up to date.

“We could spend $10 billion of President Biden’s infrastructure bill just on this and maybe still not get ICS security to match current threat models,” he said, “probably because a lot of the current software is still not secure and there aren’t enough chips to remanufacture every ICS device in the U.S. And who would reinstall them all?”

That doesn’t mean ICS owners and operators can’t improve their security. They can, and should definitely make the effort, “in a well-thought-out and prioritized way, Migues said.

“See ISA/IEC 62443, for example,” he added, which is “the world’s only consensus-based series of automation cybersecurity standards, and a key component of government cybersecurity plans. This program covers the complete lifecycle of industrial automation and control system assessment, design, implementation, operations, and maintenance,” according to the ISA website.

The Claroty report also offers a list of mitigations that will help even if patching isn’t possible, starting with network segmentation — not a complete disconnection like an air gap but dividing a network into component parts so administrators can limit access to OT systems.

They should also limit employee access to what workers need to do their jobs — the principle of least privilege — and require secure remote access. That can help detect and prevent ransomware, phishing, and spam.

Migues said it’s important to be aware that ICS security requires focus on both digital and physical components of systems. While better software security is critically important, the Oldsmar, Florida incident demonstrates that an attacker shouldn’t be able to change the treatment of a water supply using remote access.

He said he’s not blaming what was likely a small team on Oldsmar doing the best they could with limited resources. “But maybe turning the drinking water lethal is something you need to drive in to work, badge into the building, and log in to do,” he said.

“If we want more security in OT that actually monitors and/or controls kinetic events, we need to invest more in making that stuff modern, workable, and manageable,” he added. “We should absolutely not be replacing that physics and chemistry with software. We do need to invest in software between the IT and OT that double-checks what the physics and chemistry are saying, but not unilaterally override it when the software thinks it’s smarter than a gurgling sump pump.”

Not magic, but nuanced

Overall, experts say ICS operators need to follow recommendations that have been available for two decades.

“There’s no true black magic to managing security in ICS environments,” said Michael Fabian, principal security consultant with the Synopsys Software Integrity Group. “As far as what people should be doing, it’s similar to how you would approach other systems, but there’s an additional layer of nuance due to the unique characteristics of each system or vertical.”

He said the security methodologies, controls, and requirements for ICS should be based on ISA/IEC 62443,“as it’s the most mature and well-rounded of the industry standards.”

But Fabian acknowledges that can be difficult. “The security objectives and technologies are fairly well understood overall, or should be,” he said. “But the application of them in a given environment requires understanding the technology, how it does what it does, the needs of its users, operations and maintenance factors, and others, to develop how to apply those objectives to that environment. This is where people fall down, notwithstanding internal organizational factors like funding or strategic variance.”

Creating a checklist and “slapping some tech in there to ‘fix’ it just doesn’t work,” he said, since organizations must analyze, understand, and plan what their security objectives are and how to apply them in their environments.

“Organizations just struggle with that overall — it’s not much different than IT cybersecurity challenges” he said.