IoT — The Insecurity of Things (Part-2)
IoT on the Road
In the first part of this series, we looked at IoT security at home. In this sequel, let us head out and hit the road to explore security in IoT-based scenarios on the road. In the previous case studies/examples, we delved into the specifics of how exploits were accomplished only briefly. However, for one of our key examples in this article (the connected car), we will go into the attack mechanisms and exploit path in depth. This is because this case study serves as a fabulous example of how attacks can be made progressively sophisticated by putting bits and pieces of observations together and building upon smaller flaws across multiple areas to accomplish a substantial system compromise.
OK, so seat belts on and let’s go!
Security of ‘connected’ cars
Automobile manufacturers have long been in a race to showcase new, cutting edge capabilities in their latest models that combine internet connectivity, speech/image/video recognition, GPS systems, Bluetooth connectivity with consumer devices, etc. in various intriguing features. Overall, the promise is to enhance convenience, safety and provide a seamless connected experience to the driver and passengers in the vehicle.
How do these systems fare when subject to malicious attacks? Nothing highlights the risks better that this thesis by researchers at University of Washington, using the Jeep Grand Cherokee as the target! Not only does this highlight several opportunities for improving security in connected cars, it also serves as an excellent case study of how a determined attacker can put together small errors in different areas of a system into scary end-to-end attacks. It also offers several insights into secure design at systems level.
CAN Network and ECUs
All modern vehicles have internal networks that are used for communication between different internal systems (for example when cruise control is engaged, the cruise control unit communicates with the engine controller over a network). Most of the in-vehicle systems are themselves micro-controller driven and are called electronic control units (ECUs).
The most commonly used communication network for this is called the Controller Area Network (CAN) and was introduced in the 80s by Bosch. The original motivation behind the CAN network was to save on the cost of copper wire that was earlier run between each pair of individual devices. The CAN specification defines a simple but reliable protocol for sharing a common CAN bus for communication between ECUs. There is a strictly ordered system of transmission priorities and every ECU on the CAN bus listens to and can respond to any message that is broadcast by another ECU.
There are usually at least two such networks — one network that hosts critical ECUs (e.g., engine controller, anti-lock brakes system, etc.) that is segmented from another network hosting other non-critical, comfort-related ECUs (e.g., HVAC, windshield controls, etc.). Usually gateways are positioned between different networks to help convey any messages across the networks after performing any validations.
Various modern ‘connected car’ use cases (e.g., assisted/automatic-parking) have introduced the need for more networks and interconnections and new message types (e.g., automatic steering control messages) that were non-existent in the past. Also, for various media/infotainment scenarios, a whole different network called the MOST network has also been developed over the years. (There are even more other network types which we won’t go into here.)
The radio still rocks!
The good old radio on most dashboards also serves as a hub for enabling most of the integration and user interactions in these systems. The radio is also termed as the ‘head unit’ (HU) for the role it plays. External interaction via some sensor or device typically results in a message to the radio/HU and the radio, in turn, interacts with the target ECU for accomplishing the desired cyber-physical outcome. So, as an example, the radio may receive a message about low tire pressure and, in turn, may communicate to the dashboard to display a warning to the driver.
The CAN network was designed at a time when the internet was in its infancy (i.e., when internet was used mostly in universities for facilitating academic research). As a result, CAN is not ‘internet aware’. That is, neither is the protocol designed to consider malicious actors nor are the ECUs hardened against foul play on the network. Naturally, if an attacker is able to get arbitrary messages onto the CAN bus, it can cause all sorts of havoc.
Also, as depicted below, the litany of ‘cool’ features that connected vehicles offer have completely changed the network diagram of the in-vehicle networks. This gives attackers a vast attack surface and many pathways to probe for possible vulnerabilities in design, in implementation and in system integration.
Let’s hack!
With that background, let us see how the UW researchers progressed through different stages of their attack. Note that in the Jeep Grand Cherokee, the UConnect system on the dashboard serves the role of the radio/head-unit (HU) and we will use the term radio/HU/UConnect interchangeably below. The UConnect system had a primary processor (we will call it ‘consumer’ CPU or CCPU) for most of its consumer/infotainment functionality and another ancillary processor (we will call it ‘vehicle’ CPU or VCPU) for interfacing with the other ECUs in the vehicle (for all CAN bus communications). The CCPU had a serial line interface with the VCPU for the scenarios that required messages to be passed between the CCPU and the VCPU.
Stage-1: Reverse Engineering, Local Code Execution
The researchers did much of the initial exploration using the UConnect system in an actual vehicle they had at their disposition. They also downloaded and explored the ISO images of the system available online from the manufacturer to facilitate system updates at dealerships.
Using reverse engineering tools, they were able to create a detailed map of core elements of the system functioning from these ISOs. They also worked out the different components of the hardware, the many processes and services that made up the OS (UConnect ran the QNX OS — a variant of embedded Linux), different apps and their internal communications and the inter-connectivity with the other vehicle systems.
Notable findings and accomplishments in this stage were as under:
- The system was found to make extensive use of a service named D-Bus service that facilitates inter-process communications (IPC) in QNX. Although, the service was meant for internal communication between apps/processes in the system, it was also found listening on various network interfaces.
- The system could be ‘jail-broken’ using the built-in software update mechanism. The system update workflow involved signature validation before an update was applied. However, the researchers found that it was possible to trick the system into accepting an arbitrary update by pulling the USB out post signature validation and replacing with another USB containing a rogue image while the system was rebooting into update mode just after validating the signature of the update. (Interestingly, the code used to validate the integrity of the update is itself read from the update ISO image giving another lever to attackers.) Lastly, on one particular model, they also found some code in the image that skipped integrity check if a particular byte in the image had the value “S”!
- They also found that the portion of the flash where a multi-media card (MMC) would be mapped was writable and they could provide shell scripts that could be run upon reboot or during system updates by writing files to the MMC area. (This was leveraged in their end to end attack later.)
- Also, while they had been excited in early exploration to find a command injection vulnerability in one of the system services, to their elation, they found another method supported by the same service that provided for arbitrary command execution ‘as a feature’. Basically, perhaps as an oversight, the method just accepted an arbitrary string of shell commands from a file and happily executed them. After this finding, it wasn’t even necessary to ‘jailbreak’ a device to run arbitrary code.
Armed with these (abilities to locally execute arbitrary commands, update the OS), they went on to explore what was feasible without physical access to the vehicle.
Stage-2: Remote Code Execution using WiFi Hotspot
Although the local explorations and exploits were notable accomplishments, that is not what most attackers seek to do. So, the next stage was to see what could be done over the remote access interfaces — WiFi being the first target.
The UConnect system sported a WiFi hotspot that allowed the device to set up a network for the occupants to connect their mobile devices. Often the default WiFi settings in such a setup are poor in that they either use weak encryption options (or no encryption) and/or fixed default passwords. However, UConnect supported ‘WPA2’ and it was the default WiFi encryption mode (even though the device did offer WEP and no encryption as other options). Also, at first glance, the default WPA2 password didn’t appear trivial either — a difficult to guess 8-character password. But with more exploration, interesting flaws showed and gave the attackers their first remote breakthrough:
- Reverse engineering the WiFi service code indicated that the default WiFi password was based purely on epoch time (number of clock ticks) after first boot of the UConnect system. This implied that — if an attacker could guess the approx month and year when the vehicle was manufactured— it was possible to determine the default WiFi password of any vehicle within a few million brute force attempts (which would take just a few hours on a computer).
- However, there was another more serious flaw in the WiFi password generation! When the system first booted up, it looked for a clock signal to set the initial system time. If the clock signal was not available yet, the system assumed a hard-coded current time of 01 Jan 2013. Turned out that during the first boot — looking to set the default password — the WiFi service queried the system time very early in the boot-up stage and the default WiFi password always ended up being around 30 seconds after 01 Jan 2013. This meant that brute-forcing the default password needed not a few million but only a few dozen tries!
- As noted earlier, D-Bus service (supposed to be only for IPC) was bound on the WiFi network interface as well. Therefore — so long as an attacker was within a range of about 30 feet — it was possible to connect to a UConnect device and send arbitrary commands to run on the device using the flaws discovered during stage-1. They could not only do basic AV/media-related attacks (e.g., raise the radio volume, provide fake GPS coordinates to the device, disable various knobs, etc.), it was also possible to do various HVAC/body-related annoying actions such as turn the wipers on/off, move windshields up/down, flash headlights, etc.
The attacks accomplished over WiFi were disturbing in themselves. However, they required the attacker to be in the vicinity of the vehicle (or tailgating it). This wasn’t ideal. So, the researchers started exploring what is possible over the other remote interface — a cellular network connection the UConnect system offered via a tie-up with Sprint.
Stage-3: Remote Code Execution using cellular connection (but HU only)
UConnect supported various ‘connected’ scenarios such as remote diagnosis, navigation, real-time traffic and weather updates, etc. using a Sprint cellular network. The researchers now set about exploring if it was possible to run arbitrary code on the device remotely by leveraging its cellular connectivity.
This stage revealed some gaping issues with both the UConnect devices and their configuration on the cellular provider network and, eventually, attackers were able to exploit them. This is how they did it:
- First, using a signal booster appliance (a Femtocell), which served as a Man-in-the-Middle (MITM), they were able to probe the device and examine how it interfaced with the cell network. (The Femtocell basically posed as a cell tower to the device and as a device to the actual Sprint tower. It had its own set of rich tools that could be used to reach the UConnect system from a console running on the Femtocell.)
- They found that the D-Bus service was bound to the cellular network interface as well. As a result, they could send commands to run on UConnect from the Femtocell console!
- The UConnect system was assigned an IP address by Sprint to support mobile internet. However, that address was not reachable from a computer. The researchers first tried using a Sprint mobile phone which was in the same cell tower as the device. To their surprise, they could reach the UConnect system via the IP address from their phone as well and issue the same commands from the phone. But to do so on an arbitrary vehicle, they would need to know the IP address Sprint assigned to the specific vehicle’s UConnect system. This appeared tricky because even their own device seemed to get a new IP address each time.
- However, this hurdle went away after they observed that — even though it changed — the IP-addresses was always from a fixed CIDR range. It seemed that Sprint had reserved a block of addresses for vehicles on its network and would always allocate IPs from the same block.
- Further to that, they also found that it was possible to reach a specific UConnect system from any other location on the Sprint network so long as the phone was a Sprint phone (one need not be in the same cell tower zone).
- These two observations basically implied that they could reach an arbitrary UConnect device (any random Jeep Grand Cherokee) in the continental United States from anywhere using a phone or other device on the Sprint network!
- This finding had staggering implications. They could literally scan the address range and enumerate all IP addresses within the CIDR-block which responded to probes and could choose any of those as targets of remote attacks.
With these new capabilities in their arsenal, they now embarked upon the final frontier of their effort. Indeed, it was scary enough that they could literally run commands of their choice on any vehicle’s UConnect device across the country — however, what those commands could do was still limited to potentially annoying gimmicks on the AV system and the somewhat more worrisome interior/HVAC-related things like windshields, wipers, headlights control. They wanted more!
Stage-4: Sending arbitrary CAN messages using cellular connectivity
Recall that the UConnect system interfaced with the vehicle’s other ECUs on the C-CAN bus via another processor — the VCPU. Most of the exploits they had so far were based on code running on the main CPU (CCPU).
Although the UConnect system had the ability to read messages or observe activity on the C-CAN bus, the capability to transmit messages was very limited. The VCPU ran its own, proprietary firmware image which was not as easy to reverse engineer. Moreover, the messages that the CCPU could request the VCPU to transmit were burned into the VCPU firmware image with very little scope for influencing the message content. (This showed that a decent bit of thought had gone into the security design — at least piece wise!)
The researchers’ goal was to be able to send arbitrary messages to the C-CAN bus remotely — because those would lead to the scariest attacks in the spectrum!
Let us see the steps they took to conquer this final frontier:
- Just like the CCPU, the VCPU also supported system updates. However, the only way to perform an update was via a USB stick for which the whole system went into a special mode where the CCPU also had limited functionality just enough to facilitate the VCPU’s firmware update.
- Interestingly, because the design assumed CCPU’s role as intermediary (and perhaps that a technician would be physically in the vehicle for updating the VCPU), the update mechanism for the VCPU did not require signed firmware!!
- Building on their early observation about the MMC area in the image address space, they knew that if they could (a) write an arbitrary shell script on the device in the MMC area and (b) trigger a reboot into VCPU update mode, they could get the system to run the arbitrary scripts during the update process.
- After much painstaking effort, they were also able to decipher the firmware image enough to find a couple of opportunities where the VCPU could be made to transmit messages on the C-CAN using message ID and payload that was sent from the CCPU over the serial link. Now they had everything they needed!
- Using a clever combination of these and the exploits accomplished until Stage-3 (whereby they could run arbitrary scripts remotely on the CCPU), they were eventually able to send any CAN message on the C-CAN bus! This was basically game over as now they could accelerate/decelerate the vehicle, apply brakes, control the steering and make the vehicle do all sorts of other dangerous actions.
Studying an effort of this magnitude helps us appreciate how persistent attackers can be and how their perseverance pays off eventually. The research report mentions several interesting points about secure coding and design. We will look at the salient takeaways about IoT system design here.
Lessons Learned:
- For such a feature rich embedded system, one must keep “attack surface reduction” top of mind every day from the inception of the project till its completion. This involves carefully monitoring the footprint of the system and regularly asking questions such as: (a) why do these binaries exist in the image? (b) do these services need to be running by default? (c) what ports are open on which interfaces? (d) does this service need to be listening on the network?, etc.
- Security of system updates is always a challenge in such settings— especially when the situation involves the main system (CCPU) playing intermediary to support firmware update of another system (VCPU). Requiring signed (and forward-only) updates should be a bare minimum. Besides that, the update workflow and the state machine involved should be closely scrutinized to eliminate any indeterminate states or other windows of attack.
- In a system where multiple sub-systems work together to deliver end-user scenarios, just component-wise (or piecemeal) threat modeling is not sufficient. A system level view is important and should be created out of ‘end-to-end’ threat modeling. It is not uncommon for secure components to interact with each other in insecure ways when composed together. The system integration stage has to be examined with a keen eye.
- Lastly, CAN networks and vehicle ECUs would be akin to children or pets exposed to IoT in the previous (“IoT at Home”) article. These components have not been battle-hardened in an internet environment. Automotive ECUs have existed for generations in a context that needs them to be dependable and resilient in the face of the occasional error but that is a context that does not assume presence of malicious attackers on the network. Thus, designers and developers need to be super-careful and adopt a rigorous ‘what if’ mindset in all the layers between the internet and such ECUs.
Let us now look at a few other things we encounter on the road.
Intelligent Transportation & Related Infra
Cities around the world are aspiring to become ‘smart’ and — transportation being one of the key areas — investing in making the transportation experience seamless and efficient for residents. This involves lots of inter-connected systems that have to work together towards that end. The ecosystem is comprised of diverse entities such as connected vehicles (like the one we just saw), autonomous vehicles, cameras and other sensors for real-time traffic monitoring, real-time traffic signal control, dynamic message signs, automated ticketing and toll systems, tourist info kiosks, etc. As you can tell, the implementation landscape for all this comprises of a broad array of device types and diverse systems — which creates a similarly diverse, rich and enormous attack surface. We will look at just a few samples here...
Traffic Signs
Hacking traffic signs has long been a hobby of crackers. Generally, these incidents are regarded as mischief but the prosecutors weren’t amused and awarded a 2-year jail term for 3rd-degree felony when someone hacked a traffic sign which was supposed to display “WORK IN PROGRESS” with a message that said “DRIVE CRAZY Y’ALL”. Police nabbed the person soon after because a few people had seen him tinker with the sign while walking his dog. Apparently, he just guessed the device password and was able to get through to the interface that let him change the message!
In a more worrisome incident late one evening, commuters at Washington DC’s Union Station were shocked when — all of a sudden — many large advertising displays started playing pornographic content. The content played for a good 3 minutes before someone figured out how to turn the displays off. Investigators are still unsure how someone hacked into the displays.
In another attack, the San Francisco transit authority had to allow free rides to passengers because all their ticketing stations were taken over by crypto ransomware resulting in both — customer inconvenience and loss of revenues.
Weather Alert Systems
In cities that are prone to weather uncertainties (like tornadoes), weather alert alarm systems are used to warn residents (who may be outdoors) to seek shelter in case of an impending weather hazard.
In April of 2017, residents in Dallas were bewildered to hear the entire city’s weather alert siren system go off in the middle of the night in what seemed like totally normal weather conditions. Puzzled at this, people resorted to panic calling and ended up jamming all the 9–11 lines. The city officials — quite puzzled at what was happening and unprepared for such a contingency — had to deploy a task force to manually turn off the alarms/repeaters at 156 different locations!
The weather alert system was controlled by the National Weather Service (NWS) and only the NWS could send radio signals from a hub to trigger the alarms. Apparently, someone in Dallas had worked out the radio signals that activated the alarms and played them repeatedly as mischief that evening. It is believed that the person may have recorded the radio signals in a previous actual communication and replayed them.
The entire alarm system was taken offline for the weekend but then put back in service quite soon — this time with a ‘security mode’ enabled that involved encrypted communication. Interestingly, just two years later, a similar ‘Siren-Jacking’ incident happened in two other Texas cities which did not have the encryption-based protection enabled yet.
Drive Through Car Wash
Of all things, even drive-through car wash (CW) systems have been blessed by the IoT angels!
In an incident stemming from an accidental error during remote configuration, the scrubbing arm of a car wash smashed into the windshield of a vehicle and sprayed the interior with lots of soapy water. The driver panicked and sped out causing much damage both to the vehicle and the CW apparatus itself.
This story attracted attention of a few security researchers who decided to investigate internet-connected CW systems and they were not disappointed in their efforts. Firstly, the system they looked at offered a friendly HTTP server running on Windows CE(!). Aided by a manufacturer-configured default password which was typically never changed by CW owners, simple HTTP GET/POST commands allowed access to all operations performed by the CW system. The commercial API (setting charge rates, discounts, etc.) and the industrial control/config API were both available via the same interface. That is, one could get a premium car wash for free and, equally easily, swing the arm or slam the CW bay door with malicious intent.
Who would think that the drive through car wash might also be on the internet!
Lessons:
- Deployment of safety critical components must be done with higher standards in the ‘book of work’ and one of the most important first steps should be changing default passwords to something non-trivial to guess.
- When a system does have options to use it with better security, it pays to study and enable them in deployment. In the case of the weather alert system, just the fact that the weather department could get it back online with the ‘secure mode’ enabled after only 2–3 days of downtime begs the question about why the ‘secure mode’ wasn’t enabled in the first place?
- It is hard to explain how those two other cities (that too in Texas itself!) could have missed the Dallas sirens incident. That is like a developer fixing a buffer overflow bug in one function after a malicious hack and not bothering to fix the same bug just a few lines away in same code!
What next?
So there you go. We have explored some interesting cases of internet-connected systems we might encounter ‘on the road’.
In the next part of the series, we will look at how IoT security has fared in medical systems and hospitals.
Until then, stay safe and healthy!
