Supply Chain Security Speculation

Everything thrown at the wall that seemed to stick

thaddeus t. grugq
Oct 4, 2018 · 4 min read

Bloomberg accuses the PLA of hardware tampering supply chain attacks. If this is at all true, it is a pretty big deal. If it is completely false, it is still a pretty big deal (but thats between Bloomberg’s lawyers and SuperMicro, the company allegedly shipping the hacked server boards.) Supply chain attacks are a scary vulnerability because the root of trust has to start somewhere, and if it starts in a no-name Chinese subcontractor factory…it’s maybe not the ideal foundation. I’ve attempted to collect as much actual information as I can based on the Bloomberg statement:

The illicit chips … were connected to the baseboard management controller

Before the wild speculation though, it must be mentioned that the story is short on evidence and high on flat out denials.

Update: more evidence from an earlier Ars Technica article seems to support the Bloomberg report.

Update: Amazon is pretty emphatic that everything Bloomberg said about them and Supermicro is wrong.

Update: In 2016 Apple did have security issues with Supermicro, but the circumstances are far from clear. It looks like maybe Apple is bluffing Supermicro about a bad firmware, then ghosts. If they actually did find a problem, engage in a coverup, then dump the whole problem on the .gov, it explains the weird messaging going on.

Update: Apple comes out swinging with another “nope!”

Update: ServeTheHome has a good write up on BMCs, but I think they may be attributing too much technical coherence to the Bloomberg article. The hypothetical attack – altering the password verification routine – is not particularly practical for an attacker. A backdoor with direct memory access, and just a few operations (read, write, jump) would be simpler, more robust, and much more useful.

Update: put Supermicro in the emphatic “nope” column.

Update: worth mentioning that Bloomberg (and these reporters) have a couple erroneous infosec stories that should have been retracted, but weren’t.

Something is rotten in the state of supply chain attack reports

Bloomberg claims that the circa 2015 modchip, about “the size of a grain of rice,” was discovered by a third party security auditor. I can think of people who are capable of detecting this sort of modchip hack. I cannot think of a reason why a due diligence audit of a server would go down to that level.

On the other hand, Baseboard Management Controllers (BMC) and the Intelligent Platform Management Interface (IPMI) protocol are a horrendous tire fire for cyber security. That’s why Amazon’s statement about the audit rings true to me.

The pre-acquisition audit described four issues with a web application (not hardware or chips) that SuperMicro provides for management of their motherboards. All these findings were fully addressed before we acquired Elemental. The first two issues, which the auditor deemed as critical, related to a vulnerability in versions prior to 3.15 of this web application (our audit covered prior versions of Elemental appliances as well),

Auditing multiple versions of the same server is already a lot of work, scouring them for camouflaged grain of rice sized backdoors seems a little excessive. The four issues:

  • Two critical issues in the BMC web server (accessible over IPMI)
  • Two non critical ones (probably about encryption or lack thereof) that were mitigated by Amazon’s planned deployment

These findings ring true to me, this is what a typical infosec due diligence analysis is going to do — look at the interfaces and ports, see what functionality there is, what bugs there are, and what needs to be hardened/fixed.

Stripping the boards and hunting for tiny camouflaged rogue modchips is pretty intense for an audit. However, if the modchip was buggy and alerted the auditors to dig deeper, then it is certainly possible. Things that could tip the auditors off:

  • firmware errors when reflashing the modchipped unit (checksums?)
  • unusual network traffic (e.g. beaconing) generated by the modchip
  • anything else weird and unusual that raises redflags

Supply chain attacks exist. Is this article accurate? It feels a little off, but I don’t know.

What do we know?

  • Supermicro boards have third party BMC hardware to handle IPMI
  • There are at least three hardware providers: ASPEED, ATEN, and Nuvoton
  • ASPEED and Nuvoton use AMI software. ATEN has their own software stack
  • All Supermicro IPMI controllers appear to provide an extensive range of functionality that would be useful for an attacker

See the full range here, but the highlights include:

  • Keyboard Video Mouse (KVM) over IP
  • SSH
  • Serial over LAN (SOL), and SSH over SOL
  • Web server (default login: ADMIN:ADMIN)
  • Remote power management…

Servers get hacked via exposed BMC without a modchip all the time, just scan for the IMPI web console and use the default password. There are other ports to check for as well:

  • TCP 80, 443: web interface
  • TCP 3520, 5900: KVM access
  • TCP 623: menu access, allowing full control of the hardware

Good supply chain attack?

So, what’s the deal?

The real takeaway from this is that IPMI is a raging tire fire, BMCs are Satan spawn, and never ever expose IPMI interfaces to the Internet. Unless you want hackers, because that’s how you get hackers.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store