7 Security Threats Nobody Tells You About

By Yali Sela, CTO at Cymplify Security

TL;DR
We made a security-oriented tool to scan a firmware image, ran it on thousands of images, and found a myriad of issues.
Firmware images are publicly available for download, analysis, and [potentially malicious] modification. Firmware design omits and sometimes precludes security. All system components — starting with the kernel itself, through common services, and ending with 3rd party libraries — are all extremely outdated, dangerously misconfigured, and ripe for exploitation.

The fact is that 99% of IoT devices are vulnerable. Not just legacy ones, but modern ones as well, and it’s true for all industries and sectors. From home appliances, through network equipment and industry 4.0, to automotive and medical devices. Although security professionals are aware of the problem and have been publishing numerous examples (e.g. the DoD IoT report, the ZDnet report about the huge router attack in Brazil, the Microsoft report about Russia exploiting IoT devices to breach enterprise networks, and this Sophos report about critical flaws in Amcrest security cameras), device manufacturers seem to overlook these instead of investing in something very important — security by design. Nipping the problem in the bud; tackling the security issues before coding has even begun.

In the following report we will try to shed some light on major security by design vulnerabilities we found in our latest research. This data is mostly relevant to Linux-based embedded devices and relies on a few thousand device firmwares we scanned and analyzed in our lab (to better understand our method of work read below or contact us).

Without further ado, here are our main findings:

1. No one creates their product with security in mind. The first step we took in our labs already exposes a glaring flaw: The very fact we could look at the firmware images. They must be (but are never) properly encrypted, and shouldn’t be publicly available unless necessary. Updates for online devices do not fall under this strict definition of “necessary”: Updates should be pushed over-the-air by the vendor, the whole process being transparent to the end user.
2. Images can be unpacked, modified, and re-packed by a third party. There are entire global communities dedicated to firmware modding. Modding introduces the capability to customize the firmware (which is generally seen as a good thing), but also the immense risk of bricking your own device, voiding warranty and support, and introducing supply chain attacks and malware. A simple asymmetric cryptographic signature validation should be performed by the device to prevent modding and keep the device functioning as it was tested and verified in the factory. Imagine a firmware image is custom modded and repacked with malware inside, essentially making it part of a botnet. This image (which no one will ever scan or really look into) will then be shared online via forums, websites, and torrents, infecting all the modders and populating someone’s botnet with slaved devices. Hikvision earns a mark here for being the only vendor whose images tried to put up a fight when we tried unpacking them.
3. System components are ancient. sshd, telnetd, ntpd, bluetooth, even java, are all generally run with very old versions with known vulnerabilities. This state of affairs can persist as there is no regulation or customer backlash; in fact, customers rarely even know what runs on their network devices unless they utilize some vuln-oriented visibility solution. Even then, there is hardly an actionable process on the client’s side to handle the issues. Let’s put aside the fact that someone runs java on their IoT device: We’re talking over 79% of devices running externally-facing services with known vulnerabilities, often 5+ years old. Think about it — 4 out of 5 devices are just sitting ducks for a random script kiddie’s publicly available, 1-click exploit.
4. “Passwords”. 40% of user:password credential pairs on devices were either available online or bruteforced successfully within 10 seconds. Login credentials are often identical between devices. The egregious abuse of this is evident with notorious malware like Brickerbot, Mirai, and Silex.
5. Kernels are a misconfiguration disaster. Our reports show that a very decisive 100% of kernels can be — in our opinion — easily improved with simple reconfiguration. Inside the industry, this decision is obvious: “If it works, don’t touch it.” Often times the vendor’s industry is tiered and the kernel is not the vendor’s decision, but rather a customized kernel compiled 10 years ago by his board supplier. This leads to vendors omitting basic security features, for example stack canaries, which protect kernel code from simple buffer overflows. They also enable dangerous configs such as /dev/mem and /dev/kmem, allowing processes to examine the memory of the system, and kexec, which allows you to hotswap your kernel during run-time. The most common reply we received when pointing this out to vendors was “oh, oops, let’s turn it off” — showing this situation can be easily fixed if decision makers were only made aware of it.
6. Kernels are ancient. We’ve seen myriad versions in the wild, running the whole range from 2.x to 4.x, but heavily leaning towards 2.x. ~19% of images use a 4.x kernel, whereas ~25% use a 3.x kernel and a whopping 55% of images use a 2.x kernel (including 5% that use a 2.4 kernel!). The last of these ancient kernels have been written over a decade ago, with maintenance and support (even for the long term support versions) having run out years ago. The result? Exposure to dozens of publicly available 1-click exploits.
7. Permissions and capabilities are not utilized. It is simple good practice that every process on a system should do what it does with the minimum permissions required to do it. This limits the damage any single compromised process can cause. This is not a practice that exists in IoT devices. In IoT, as a general rule, everything — every binary, every process, every closed-source 3rd party component — runs as root; Any compromised component can take over the whole system. For example, a compromised web server could lead not only to your device sending browser exploits to users, but also to a database leak, re-writing of the flash memory to brick the device, or act as a staging ground for further exploits into the device to extract private keys from an on-device hardware security module. An impressive 26% of our analyzed firmware granted root privileges to 1 or more unnecessary users. Usually, it’s most if not all users, but specifically, users associated with services processing external input.

Conclusions

The vast majority of devices we analyzed suffered from vulnerabilities that could have been solved if device manufacturers had incorporated security early in the development cycle. Basic good practices such as the concept
of minimal required privileges, or segregation of user and kernel space, are ignored. The internet of things is struggling and developing, with no one paying any attention to the cybersecurity domain, resulting in global-scale
attacks around once a week.

IoT security looks like this

What to do

Device manufacturers must incorporate security elements early in the product development life cycle. This include at least the following 3 items:

• Designing your product in advance to work with OTA updates. This removes the need for client-side manual updates/patching and the presence of the device firmware online.
• If possible, have each device burned with a unique password (possibly printed on the label). If not possible, then choose a strong password, and for god’s sake change it if it leaks to the internet. For client interfaces, force the client to change the default password after installation. Use a good password policy.
• Before releasing your product, update it: Can you recompile the kernel with stricter configs? Are you using recent versions of libraries and services? You should.

Another, more modern approach that tackles this problem is an on-device, real-time protection solution. This can be implemented during the development of a new product or at a later stage using standard
software updates. To learn more about this solution, visit our site: www.cymplify.io

How we conducted the analysis

We recently started large-scale analysis of firmware images. That is, the letters_and_numbers.bin file that you can download from vendors’ sites to update your router, your drone’s remote control, and so on. This firmware file essentially contains everything that’s going to be on the updated device: filesystem, kernel, passwords and keys, and included in these are of course the actual processes that run on the device. Having gathered a modest few thousand firmware images available online for research purposes, we ran our analysis tool on them to generate reports.

This analysis tool was carefully crafted and honed, based on our offensive cyber experience, to find stepping stones on the way to compromising a device. Each report contains actionable items, useful for either a theoretical attacker or for a security team. Each report contains, among other things, CVEs for externally-facing services that run on the device, credentials extracted from the image, misused functions, and dangerous configurations. We aggregated thousands of these reports and queried our dataset to discover some awful facts and trends regarding the security state of the internet of things, as evidenced by the incidence rate of the findings.

Statistics were derived by percentages for each relevant subgroup with N > 200 samples.