Exploiting OPC-UA in OT Environments
In a talk at Black Hat USA 2023, Sharon Brizinov and Noam Moshe from Claroty Team82, disclosed a significant vulnerability in the Open Platform Communications Universal Architecture or OPC-UA, a univsersal protocol used to synchronize different OT devices. In this episode they also discuss a new open source OPC exploit framework designed to help OT vendors check their devices in development.
[Heads Up: This transcription was autogenerated, so there may be errors.]
VAMOSI: Communication is important. It allows me to convey my thoughts so when you hear them it makes sense. Or least I hope it makes sense. So the fact that I’m speaking in English and you can understand English allows us to communicate.
What happened when you different OT devices, built by different vendors, and you want to link their together?
This might be an apples to oranges comparison, but bear with me. Within layer 2 of the TCP/IP stack, there’s the Address Resolution Protocol or ARP. What this protocol does is make it possible for two or more devices that don’t otherwise have a convenient way to communicate to join a network. ARP does this by resolving their Machine Address Control or MAC addresses. and associating it with an IP address so your non-internet connected printer can join the network.
There are a lot of assumptions here. For one thing, ARP does not provide methods for authenticating. Usually you’re connecting devices on an internal network. Attackers ought not be inside that.
Nonetheless, there are examples of ARP poisoning, where someone creates a MAC address with the intent of intercepting the messages. So think again about the office printer. That 20-page legal document you just sent to the printer might now pass through a bad actor’s machine before it reaches the printer. This is a classic example of a man in the middle attack.
Again, how might this relate to an OT device in the field? Because of the differences in devices, in RTOS, in languages, there’s a protocol that allows most OT devices to communicate with each other. Thus you can set the temperature across multiple devices from various vendors. And you can receive data from these devices on your HMI.
This is the story of OPC-UA, a machine to machine protocol, and how a vulnerability in its implementation could have left machines worldwide open to attack.
I’m Robert Vamosi
This is Error Code.
{Music]
BRIZINOV: So I’ll start. My name is Sharon Brizinov, director of security research with Claroty.
MOSHE: Yeah. So my name is Noam Moshe. I’m gonna build the researcher with Claroty Team82.
BRIZINOV: So clarity is cyber in cybersecurity defense company. We’re trying to protect industrial network health care networks. So anything around OT, IoT, and medical devices.
VAMOSI: So, before we get too deep, let’s start with the basic definition of OT.
BRIZINOV: Yeah, so the operational technology is a certain type of network computing network that mainly consists of devices that have something to do with the physical world. So for example, if you think about machines that produce something manufacturing line, you have the physical actuators, and you have the sensors, for example, a temperature sensor, but you need some kind of an orchestrator that will control all the inputs and outputs. And this is the PLC, programmable logic controller, some kind of computer that you can remotely program. It’s logic to control the operation and physical process. So ot network is the entire network where all of these devices sit together and work together in order to create something physical.
VAMOSI: Aside from the manufacturing example, OT could also be more mundane examples such as elevators.
BRIZINOV: So elevator is a part of modular networks in a specific network called BMS building management system. But a tiny percent is also considered part of an OT network. we have the physical aspect of like, either like manufacturing something, or like controlling some kind of process or in the case of elevators, of course, it’s like the physical aspect in the real world effects of moving people from one floor to another right
VAMOSI: What Sharon and Noam presented at Black Hat is a vulnerability in a common protocol, the Open Platform Communications Universal Architecture or OPC-UA protocol, that is used today across almost all OT devices.
BRIZINOV: Yeah, sure. So in the past couple of years we’ve been researching in research team eighty two. OPC UA is a standardized protocol. Very common in OT networks. Its main goal is to synchronize different devices and different systems on some values. So for example, if you want to have one device producing some kind of a value that is correlated to something in the physical world, for example, it keeps track on the current temperature, and you want to synchronize other devices or other systems with this value, you would use OPC UA to sync all the devices and systems with this value. So it lets you transfer and communicate on certain types of data, specifically what we call tags, or points in our physical dimension.
VAMOSI: One of my problems with IoT protocols like MQTT is that they are old and they have been shoe-horned into a modern use for which they were not originally designed. So I’m wondering if this OPC-UA protocol has been around for a long time, or is it a relatively new protocol?
BRIZINOV: So OPC-UA is the new version of an older protocol called OPC. Or in its more complete name, OPC DA so OPC was the first protocol in the OPC category that was supposed to unify this kind of information transfer between devices and systems in the OT network, but it lacked some features and it was pretty limited due to some constraints. So OPC UA was developed to cover this and to progress and make it better. So OPC UA has been around for, I think, more than 10 years, which is not so not so much in terms of industrial activity, but it’s definitely one of the most popular and most common protocols, you know, networks these days
VAMOSI: so given the number of vulnerabilities, what made Sharon and Noam start researching this one? What were they looking for at the time?
BRIZINOV: Sure. So as I mentioned, OPC UA is a standardized protocol. So it’s very common and many vendors agreed on supporting it, because they wanted a protocol that will be used by all the vendors regardless of the specific equipment and specific protocols, proprietary protocols being used by the vendors. So this fact made OPC UA a unified universal protocol that everyone can communicate with because all the vendors are created to support it. It made OPC UA a very popular protocol in the industrial world. And so, obviously it has a huge impact if some flaws are found within specific hardware or software equipment in industrial, industrial vendors or companies.
VAMOSI: So that’s good. They were looking for a universal protocol that might have vulnerabilities that could affect a large number of industries. Better that good guys find it than the bad actors.
BRIZINOV: So obviously, since clarity has a lot of involvement in the security of these networks, we want to make sure that current modern implementations are secure. So we started to look at different servers, appliances, and products that support OPC UA, and then we started to find some bugs. In addition to that, CMC, it’s a very important protocol. It’s a sub company of Trend Micro. They also wanted to push the industry forward. So through their Pwn2Own hacking competitions.
CHILDS: Welcome to Pwn2Own in Miami 2023 I am Dustin Childs head of thread awareness here at the zero day initiative and I just wanted to take some time to show you around first of all you know you see the room you know you’re here because we got the great sign out front including txOnd networks which is our sponsor for this year’s event so happy to have them as a co-sponsor and uh very glad to see their involvement let’s come in the room and check things out
BRIZINOV: They offered prize money to any researcher or security researchers that will find and report bugs or actually vulnerabilities in this protocol, so it gave our team a lot of it incentivized us to accelerate the research on this.
VAMOSI: So in Pwn2Own, the competition is a bit different than the usual BugBounty. Teams are given a list of targets and have weeks to prepare, then they have a certain amount on time to present there vulnerability. If they fail, they fail. If they success they gets points and potentially money. And these Pwn2Owns are focused. For example there’s Pwn2Own Japan which looks at Automotive; there’s Pwn2Own Vancouver, Pwn2Own Toronto, and then there’s the ISC one at Pwn2OWn Miami.
MOSHE: Yeah, so particularly Pwn2Own Miami, is the OT brand of Pwn2Own, they have a lot of sub brands, like in Toronto, you have the IoT competition. And in Miami, you have the OT competition. And in the last three years, we’ve participated three times and we actually won the master of Pwn awards this year.
Specifically, it was mostly on OPC UA and different OPC UA implementation leads, servers, clients and even gateways. So yeah, we’ve participated a lot in form two on Miami, and we currently hold the title of Master of pone con two on my own,
VAMOSI: If you could unpack that a little bit, I know the rules of Pwn2Own are a little bit different than your typical hackathon. So the the master category is quite a feat. You get so many points and so forth to become that.
MOSHE: Yeah, so basically at the start of the competition, CDI is a zero day initiative that issues a list of targets. These targets are different devices, different servers, clients, gateways, different kinds of software or hardware kids and they are the targets for the competition, and on each target unwanted outcome is decided, for example, remote code execution and unauthentication. Remote code execution is the outcome and a specific set of points is issued to whoever manages to find a vulnerability that exploits it in the specific way and during the competition. Everyone registers for their desired target list that they’ve uncovered going to buildings for and all the issues are presented on stage. And whenever their condition ends, the amount of points it’s like, basically, issued and whoever has the most points gets the master phone award. And during February this year, we actually won efficiently meaning we had the most points we exploited targets Yeah, I think either 10 nine or 11 targets and Gani remote code execution and a whole lot of them. Some of them were a denial of service attack. So either at now service which is actually pretty critical when we’re talking about physical and physical programs like if you manage to somehow DDoS or dos device it could have physical effects, which could be even more severe than just gaining remote code execution, because we are talking about disturbing the process instead of Yeah. So which could be incredible. So yeah, we either 29 or 11 targets again the master of Pwn award,
VAMOSI: And so what we’re going to talk about, is it a flaw in the protocol itself? Where is it in the common implementation of that protocol?
BRIZINOV: Yeah, true. So it’s a very good question, because for OPC UA, there is a very, since it’s a standardized protocol that has been developed through a very long period of time. The specification for this protocol, or very firm, so the specification are very thorough and complete. So it’s very difficult to find flaws in the specification. Usually the case is when developers are trying to implement the right to their computer code that correlates to the specification and then some implementation flaws arise.
VAMOSI: Over how much time?
MOSHE: so we were given two months, or three two to three months in advance with the target list. And basically you need to do your research and find vulnerabilities on all the different targets. And then you do it during three days in Miami during the s four event where you basically schedule people and on stage you should exploit each target and it’s a binary output. You either succeed, we have like 30 minutes, timespan and retries. You should have your exploit ready and you just send it there you start to exploit and hopefully succeed.
VAMOSI: I’m wondering if we could talk a little bit about some of the industries that were affected by the research that you did?
MOSHE: So basically, certainly understand that almost every industry uses OPC UA because it basically standardize the communication method before OPC UA, if, for example, you have a PLC of Siemens, only HMI and stuff it support their proprietary protocol, which they do not share could communicate with it. So instead of buying all of this proprietary equipment, which did not work with other equipment from different manufacturers and different vendors, OPC UA was created in order for everyone to communicate using OPC UA. It basically acts as a gateway between different protocols. And that way I can compete. I can connect my let’s say, list controller to my HMI or to my screen or cetera, or of course my PLC is to HMI and my actuators, etc.
VAMOSI: HMI is a Human Machine Interface. It’s the dashboard that reports out what the OT devices are doing.
MOSHE: So it basically acts like a common gateway. And in OPC, we have three different devices. It could be an OPC server, which basically every different kind of device connects to it and it acts like the main server holding that acquisition and doing it. We have a gateway that communicates one protocol to another one. And lastly, we have clients. Think about it like if you know an HMI screen, which means to present like an A level of war tank for example. And during our research we actually attacked and vulnerabilities in all three different entities both in cells, gateways and clients, and I can present to you for example, the client exploitation technique, which we found two different popular client software, we found a similar vulnerability. Basically, the client connects to an OPC UA K salvo and tries to read tags from it in order to present something visually. So we thought to ourselves sense a lot of clients are built using a web browser. For example, they present my HMI using a web browser that presents My agent my HMI screen, what would happen if we could somehow inject malicious JavaScript code during the tag reading? So our attack surface is pretty simple approach. We basically look for a client to connect into our OPC UA server and whenever they read the tag, or for example, the metadata about our server, we inject malicious X asset payload, and we have control on the browser, we can execute JavaScript code on the browser. And then we use this privilege chain together with a few other vulnerabilities to actually exploit the OPC UA client and execute code on it.
VAMOSI: it’s not just manipulating the Human Machine Interface or HMI. By chaining together vulnerabilities you can manipulate the OT object itself.
MOSHE: Exactly. Because we spotted this both on inductive automation ignition, which is a very, very popular salvo that basically does that acquisition and presents to you a UI and GUI of an HMI of your screen. So first of all, we connect we learned the client that you connected to our malicious OPC UA server, we send a malicious tag or see, just like John said, you know, we see the main data, the data business logic, is reading and writing tags. For example, what is the water level in your tag, and we weaponized one tag that whenever it is read in excess payload will be executed. And then we actually exploited the integration server. And so we got using that project upload procedure and executed code in some logic during the process, upload procedure and we did it from the user’s browser, meaning once we executed JavaScript code using deficits vulnerability, we found in the OPC UA client, we use this browser in order to do a project upload on the server and invoke that change another vulnerability in order to achieve remote code execution on the inductive automation syllable. So it’s like a chain of vulnerabilities.
VAMOSI: So the target that you had was like a water utility? What type of industry was it that you were looking at?
MOSHE:: Because obviously, OPC-UA, in general because it basically needs to support every protocol. And that way you can connect your PLC your water tank, like you’ve just said, and all it could be like physical manufacturing food and beverages, oil and gas. A lot of industries like almost all modern industries use OPC UA to act as a centralized protocol. In their manufacturing or in their building etc. It is mostly an OD protocol, meaning it’s less of a TMS like it said, If you believe but more manufacturing food and beverages, gas and oil, stuff like that. So it’s a motive, of course.
VAMOSI: Remember OPC-UA is applicable across a lot of different industries.
MOSHE:: Exactly. And so let me talk about the presentation we’ll be performing in Blackhat. So the lecture said we’ve conducted our OPC research over the last three or four years where we’ve looked at different products that support OPC, different SDKs, and different software development kits that are very, very popular in different fields in the industry. And during these last four years, we actually try to look at OPC UA as at the protocol level, meaning we try to look for vulnerabilities that could affect not only one vendor of one specific code implementation, but more of like an idea of our ability that a lot of different vendors and a lot of different products could miss implemented where something is not clear enough, something’s not specific enough and look for like, basically a logic vulnerability and try to look for how it could actually be implemented in July. So we basically amassed lots and lots of different vulnerabilities and tons of different products, which I’m sure we’ll be happy to share with you and try to look for protocol level vulnerabilities. During that time, of course, we develop different kinds of vulnerabilities and these for example, if we’re talking about an ad where basically OPC where you communicate using channels, and you open a channel and then you can communicate with the server. And our ad was what if we would send the message which would take the server a long time to process and we could be talking about a few seconds all the way to a few minutes. And then during this message while they are still processing it, for example, we would close the channel. Then whenever the Silvo would try to act and use our session object, it would actually be freed already. If we’re talking about OCP code and the null pointer reference vulnerability will occur. This will go to we found a few servers that implement the idea of long requests, freeing up the session and we find different servers that were affected by this idea for vulnerabilities. And another one is, for example, chunk flooding, where in OPC, you can send one message that is separated into different chunks. And that way just the client tells the server Hey, this is the last message they have a flag. What would happen if we would just keep sending different chunks and never say that the last the last chance flag, the server would allocate more and more memory etc. Or for example, the idea of something very complicated message where each of the object the parameters in the message is an object of objects, and so on and so forth and wish we would perform basically start exhaustion, vulnerability on the syllable where they would need to allocate and call another function of powers my object and once again, it is a list of objects that every one of them is also a list of objects and so on, so forth. So these kinds of abilities are like protocol ID vulnerabilities, and where we try to look at many different products that could be affected by them.
VAMOSI: So given that it’s a protocol who do you turn to to mitigate this? Is it all the different vendors who are implementing it or is it the actual administrators of the protocol?
BRIZINOV: I’d say both. So first of all, the vendors should fix it so whenever we find a vulnerability, we’re recording this to the vendor, and we’re working with the vendors to fix the vulnerability. Once the vulnerability is fixed, the vendors are issuing patches and fixed versions, which asset owners have the responsibility to upgrade and patch their software within the factory or within the LTE network. So it’s a shared responsibility of researchers to find and report vulnerabilities vendors to fixed vulnerabilities and issue fixed releases and asset owners to upgrade their software in the network.
VAMOSI: Okay, and so when you present this, most of the vendors have already pushed out a patch. All of the vendors have pushed out a patch.
MOSHE: Yeah, I think almost all of the vendors issued a patch
VAMOSI: But how do you get that word out? I mean, obviously, speaking at BlackHat is one way to do that. But it seems like you found something that’s fairly common and not all the asset owners might be in the audience at Black Hat..
BRIZINOV: Yeah, that’s why we’ve been working with the vendors for a very long time. And we made sure patches and fixed versions are available. And we also communicated this to the to the community so asset owners will be aware of this and hopefully will upgrade their software in the network.
VAMOSI: is there a form of search for vulnerabilities like this? Some way that you ca
MOSHE: ICS cert. This is the organization led by CISA, the cybersecurity infrastructure in United States, and there are definitely the leaders in the ICS Cert organization.
VAMOSI: And so they could begin by spreading word of this vulnerability and the need to upgrade immediately, et cetera, et cetera, they would be the ones issuing that
MOSHE: True. They will be part of the chain, they will also so usually we’re also using their assistance to coordinate with the vendors. So Sisa or specifically ICS cert are part of this chain. By working both with researchers like ourselves, and also the vendors. And they’re pushing both sides towards an agreement and towards very specific timeframes for fixing vulnerabilities and also notifying the general public by issuing advisory security advisories.
VAMOSI: Do you find that there’s a maturity in the OTC space that when the vendors are approached with these sort of things that they have the capability to deal with it or are they still somewhat immature and they they don’t? really know how to react to a security vulnerability?
MOSHE: Sure. So okay, so it very depends how mature the vendor is. So since we’ve been researching the most, most common software and hardware and equipment, so these companies are quite mature and well with their customer base, so they have huge customers, and therefore they have the responsibility to make sure their products are secure, but they’re also very mature in how they receive a vulnerability reports handle them internally. And providing fixes in a short time. relatively short time. So we’re talking about a couple of months. But yes, most of the vendors we’ve been working with are very mature, and they actually were very happy to receive these reports from us, instead of from someone whose intention is to do some harm.
VAMOSI: it seems like for a number of years OT vendors coasted along and there weren’t that many researchers looking at it. There weren’t that many vulnerabilities. That sounds like you’re right. If they are common, then they’re going to have that maturity level where they can handle their intake and they can produce the updates.
MOSHE: Yeah, true in recent years, due to multiple factors actually, vendor ICS vendors became much more aware of security, and much more. Acceptance of security researchers conducting penetration testing on their products. And producing vulnerability reports to them. So I would say that the current situation is that ICS vendors are very knowledgeable and accepting towards receiving vulnerability information and working with the vendors to fix those.
VAMOSI: And a lot of that was obscurity through the fact that you know, PLCs were various and diverse and whatnot. But now you’ve got this common protocol that’s linking everything together. There’s more opportunities for the bad guys to get in. Shall we say?
MOSHE: Yeah, I think the number one reason for why you know why the security domain has developed so much in ICS areas is due to convergence. The convergence between it and OT so in the past, OT networks were completely disconnected. And even air gapped from the IT network. And so researchers or even IT security analysts or administrators did not have any type of access to these devices. So they had no chance to research or to interact with these devices. So once we started kind of the convergence process between it and OT, modern OT devices are fully Ethernet based. So it’s much easier, the barrier to interact with an IoT device is much lower. And so researchers are starting to discover more and more vulnerabilities and this is exactly the change we’ve seen in the past few years. So more vulnerabilities started to be discovered. So it’s not like there were no vulnerabilities in the past. It’s just they were hidden behind bureaucracy and behind accessibility to all the equipment
VAMOSI: And when you say lower the bar, this also lowers it from say the resources of a nation state down to a criminal organization.
MOSHE: Represented Yeah, and in recent couple of months, we’ve seen ransomware gangs targeting old equipment. Yeah, you’re definitely right.
VAMOSI: And do see them going after different interests within OT, like oil and gas for example, or critical infrastructure, water utilities, electrical
MOSHE: I think so. So if we’re talking about, for example, ransomware groups, they will try to address or attack anything that will potentially be a financial gain for them. So if we’re talking about shutting down in a hospital, just to gain some financial reward. Yeah, they’ll definitely go for it.
VAMOSI: Sharon and Noam didn’t just talk about a vulnerability they found. At Black Hat USA 2023 they also announced a framework to help OT vendors check their products.
BRIZINOV: We can definitely talk about the OPC exploitation framework. So maybe I’ll start with a short background story for this. So as Nam said, in the past few years, we’ve been researching OPC UA products. This includes the SDK, so the very source code that you can implement in your software to gain OPC UA capabilities but also OPC UA fully ready software and applications, servers, clients, etc. So we’ve been doing a lot of research and we found a lot of different vulnerabilities and in the past, so we’ve been doing this research for a very long time. And from almost the very beginning of our research, we implemented a kind of OPC UA exploitation framework very similar to meta sploit where we’re adding all of our exploit payloads into the framework. So with time we started to collect more and more exploit payloads implemented in our framework. So now we have a very powerful framework. And we’ve been using this to research all of the products. So for example, we found a vulnerability in one product. We implemented the payload to trigger this vulnerability and now we were able to check all the other products and maybe to find more bugs of the same category using our framework. So now we have this great framework we’ve been using to research all of these OPC UA products and we’ve been thinking to ourselves, so maybe we can share this with the vendors so they can check the same payloads on their products. Maybe we’ve missed a couple of products where we were not thorough enough in our tests. So we actually created a small coalition of many OPC UA vendors, and we shared this framework with them. So they could do some homework on their own products, and use our framework to test the security of their products. So it was very, very beneficial. They used this framework for a few weeks. I’m not sure yet about the results, but I do think they found some more vulnerabilities that were not covered before. And we’re also going to present this framework in blackcat.
VAMOSI: Will you be making that open source for everybody to use?
BRIZINOV: We will make it available as open source. But it was very important for us to contact as many OPC UA vendors as we can and share this with them a few weeks or even months before the actual release.
VAMOSI: I’d like to thank Sharon and Noam for talking about their Black Hat USA presentation. The OT space hasn’t gotten enough love in terms of vulnerability research. And as more and more attacks are focused on critical infrastructure, it’s way over due for serious security discussion in my opinion.
