Pax Technica: afternoon panel, security

Notes from a CRASSH event by the Technology and Democracy project. Updated through the day. Following on from first session.

John Naughton: now addressing the security and privacy issues which are in the public perception of IOT. With Chris Doran, Director of Research Collaborations at ARM, Jon Crowcroft, and Phil Howard again.

Jon — really struggling with business cases for IOT. the markets in the internet are either two sided (facebook, google, advertising!) or subscription…

hard to see how the two sided stuff will work in IOT

also hard to see business case for machine learning in the long run too. Analytics on silo of data, fine. Eg DeepMind + Moorfields eye health machine learning — but no inventive step here, OSS convolutional neural net, cheap compute time plus a surgeon’s expertise. The scanner could run on a smart phone and detect and therefore stop a lot of macular degeneration. Or, canonical IOT fridge stupid example — could also use a shopping list — spot dietary issues that imply diabetes and correlate that with the smart phone, stop people losing sight. Wow so useful! But no company makes this. Because there’s no business case.

Big startup with Irene Ng looking at this, trying to find the business models, very hard.

Privacy invasion is one issue — see next panel — and in the startup (Hub of All Things) they can try business models. But privacy is NOT a selling point (unless people have just had ID theft). Do you know what your facebook data is worth per year? based on their revenue it’s about $18/y. Maybe that goes up with IOT data, maybe more if there’s health data, but it’s not a lot. hence no business case.

Things work in siloes. The NOT — network of things. Maybe you save a bit of money, say, replacing meter reader with a connected electricity meter. It’s not a lot of value, it’s a little incremental saving because it’s not an internet, it’s just a connected meter. Slightly improve bus timetable by knowing how many people are at the stop. THEN race to the bottom because the Things have to be so cheap. So there’s no incentive to secure them. hence loads of security exploits on embedded systems. real nightmare. And they don’t maintain or update it at all either. eg: Philips Hue lighting, change the colour of your LED light on the internet; massive security problem. They had an update process, applied it, fixed in a day (and made future attacks harder, by changing key architecture). (why security risk? strobe lights to affect epileptics. Malware could have propagated across paris in 30sec). Of course now you can only use official Hue bulbs… DRM… so paying 3x standard price. Maybe less worrying DRM than in medical devices.

You can make these cheap 10c webcams, and you could have nice security but the PhD who knows how to do that is too expensive. Sometimes companies get burned and then go fix it, but you have to have a structure to afford that (eg be big like Philips).

verifying software in new systems is expensive, but can be done and sometimes necessary.

Chasing down who is liable for security attacks may help, get CEOs thinking about it. But ultimately ISPs will start to restrict access — no internet for the $3 webcam that’s likely a security hole.

Chris — representing own opinion not official ARM.

ARM acquired by Softbank, to do IOT, basically. these sensors, devices, data, actuators, transforming all our lives for good. that’s the high level pitch.

Not sure there’s that much money just in the data. Not vastly more revenue. Area gets more interesting when you have robots and actuators — like precision agriculture, surgery, etc. Real value will be the companies who actually DO stuff, not just move data. probably a good thing. But brings more security issues! Physical effects!

Everyone has their favourite example of why IOT is completely insecure today. Chris’s is the sex toy hack example in Berlin.

Security is an afterthought, if done at all. And the race to the bottom on price is a problem. ARM think there are solutions — eg start with device, and design security in from scratch (in a more robust way than has been done to date, where focus has been on power and energy efficiency). So working a lot on new architectures for security. Also Computer Lab at Cambridge working on this. Expect big change in how chips are architected — both the big chips and the small ones. That’s not enough of course, people will still fail to implement them well, or won’t update them.

Long lifetime devices are an issue eg cars. Software industry is bad at fixing old code — end of life comes pretty soon. Cars last 10–15y. Hacked car will be dangerous however old it is. Security for full lifetime will be a real challenge but essential. Driver will be legislation and liability. Manufacturers will be required to do security from the ground up very soon. And companies, new business models, on guaranteed security stuff. Not data business models.

ARM is a broad ecosystem play, not siloes. But that may not work well with the need for end to end security. May need something new. Look to academia to help here.

Phil — on use cases. IOT doesn’t seem to be a tool of disruption. It’s a tool for insurance companies, or large firms managing complete product chains or complex labour forces.

Such as a large coffee firm in Seattle. looking to manage complete arc from fields to labour assignment in a cafe and mapping IOT enabled coffee mugs flowing through train stations eg Penn in NYC. Goal to figure out traffic flows of people in the station to time delivery of drinks, arrival of beans, labour in cafes. Can they make it more efficient? They think it could be significantly improved.

Insurers. looking at people who may have bad habits and insurance rates that can go up or down. May be a model, but it’s not disruptive.

It’s about big companies making big investments.

John: so are fears of IOT as unstoppable juggernaut overblown?

Chris: hype started 8–10y ago. It’s just more machine to machine comms. Slow, partly because of security fears. Until secure you don’t get the business models.

Coffee store example is interesting one, will that actually translate to value? Because they are already pretty efficient actually.

John: Hue example seemed good practice, but in terms of policy, how do you make it always happen?

Jon: set a bar for best practice. If something bad happens, there’s liability. Law is operationalised. When you get to court you learn what is meant. Internet talk about best practice. Reverse port checking to stop DDOS — you can imagine a liability chain in a DDOS attack involving the ISP who didn’t detect that and the folks selling the shrink wrapped devices. If you bought a car, it doesn’t matter where BMW got the engine management system from, if it breaks, it’s BMW’s fault, they assembled it. Just the same as in analog! if steering wheel falls off it’s BMW’s fault even if they bought the wheel in. No wonder insurance industry interested, they want a toe in the water. People are treading carefully, no one wants to be the first to be doing the liability thing… wait and see what happens. There’s best practice for safety for a component, and then there’s extra complexity around software.

The Needham award winner, doing GPUs and 3d image recognition. Qn was about what about the trolley problem? Well, if the thing runs down a 5 year old, by then, it will be clear, because we’ll be using verifiably secure and provably correct stuff and the specification will be to blame not the code.

John — devices that can make things happen in the real world, could make dangerous, lethal, counterintuitive things happen. When Investigative Powers bill was in the works, scary bit was equipment interference section, clearly from security services, where IOT offers new opportunities for security services to do stuff, eg hack into car and cause accident. But they needed legal cover to do this. But that’s the difference in this stuff, making things happen.

Chris — not that different. Italian Job (original) — adjusted traffic lights — you could do that even then, if you were minded to do so.

What’s new with IOT is the price, so cheap, easier to put fully functioning computer into a device than some specific hardware. So things have more compute power than they need.


Q: business models is tough. Main thing is big corps doing servitisation, equipment as service. Often engineers solving a specific problem, not widespread crosscutting internet-ness.

A: Chris — yes, servitisation extends to all white goods in the future. No reason why you’d throw out a washing machine instead of fixing. The challenges are if your business is selling washing machines you don’t want to be a service business that delivers washing services. Companies looking to reposition, but hard for them. Potential big win in efficiency, reuse, not disposable economy.

Jon — colleague in US, got email from Audi that his car in UK had emailed the garage to say that the parking sensors had failed. One day car will drive itself to garage. Ancient Subaru, clock failed, garage couldn’t fix; found internet fix, pull out clock and reflow solder. Fixed it. Both rich person end, and poor person end, this stuff works. Get info, can address problem, repair things. Maker community could say: when a car with all this safety critical stuff is no longer maintained or something, or third hand in Nairobi, demand open source of all the systems. Still need verification of the code, or reputation network, but could be done more automagically with IOT!

Q: unclear where data will be useful in these business models. But security is the risk — only one device needs to breach and you have a risk everywhere. The data value may not be direct, but 4th/5th person buying it — like the value of Russian interference in elections with causal effect much later — so business model somewhere there?

Chris — nuclear industry, tech is v old, so risk is low :) not connected, yay.

talk about herd immunity. if most are ok, one gets hacked, not too bad. The majority might even detect that one is broken, and for a sensor could ignore it or fix it (reflash). Not necessarily an option in a hospital setting. Can design into the hardware protections that will stop stuff jumping to another device.

Dumb router or smarter router that will catch things..

pushing responsibility onto consumer won’t work. When you get a popup saying certificate not secure, no one is going to make a good decision there — have to move the decision elsewhere, do the right thing. Never easy. but solvable.

John — some techies had a way to control temp of shower from smartphone. Could do that from anywhere, because the data had to go via the cloud, making a risk that someone miles away could change the temperature. IF the data had stayed in the house, this would be less of an issue. is the architecture — this clueless design — to blame?

Chris — yes. we designed the internet so anything could communicate with anything so… and could set router to say, no data goes out without authorisation.

Jon — Bottom up approach — TCPIP is fast, lashup, then find home router unreliable, so need GPRS backup; then you batch via a hub… makes sense for the widgets monitoring security sensors etc remotely. but at the SmartTV end, be smart about what people are watching, record it locally. can all happen locally. Content provider still needs to know what you watch, but the smarts can be local. But then people want to combine things from different bits in the home, and that means connecting all the different clouds… hard. So need to combine in the home first, so that might be an evolutionary path. And for GDPR may want to keep all the data locally anyway, avoid risk. Like databox project — all in your home, optionally in a secure enclave in the cloud if you want it — experiment to see if users want this stuff.

Q: would it be safer to have a single function smart thing, rather than general purpose computer in every device? Also, what about IOT companies going out of business — who is responsible for security ongoing at that point?

Jon — refer you to Doctorow War On General Purpose Computing. But believe ARM story — open source OS for ARM — can do checks for physical limit safety etc, which is good. Someone looking after a smart home hub could have shims to look after the wrapper of a device that is no longer supported. Or, you agree to replace it, because no longer safe — just as you do with other old gear like boilers that emit too much CO. Whitelist vs blacklist models. Think whitelisting may become a more common model, which is sad, less open internet.

John: who is driving good practice in the internet?

Chris — journalists, politicians, standards bodies, engineers… Open standards are safer, if everyone can examine and see it’s safe. Society has done this many times, ways to drive best practice; there will always be mistakes, and some people who try to avoid doing it, but it will get there.

John — this doesn’t sound like the disruptive capitalism of silicon valley.

Chris — sounds about right. It’s about big traditional companies. Banks, who are used to keeping data secure, could be brokers, looking after keys. Insurers. Need big players. New companies will emerge. IOT appstore will appear. New companies will take advantage though not do the backbone infrastructure. They will play within existing standards, not create new ones.

Phil — interviewed IOT engineers. German design lab, were they threatened by china? no, felt they would design robots for china and design chips for china.

Chris — countries should play to their strengths. China shouldn’t be underestimated. Think about what your competitors do. China leading the way in machine learning — not doing cheap knock offs, and not stealing IP because caught up.

John — will china set some of the standards? Do they need us as much as we need them?

Chris — have to involve China. if they want to sell into western markets they must follow our standards. They comply with standards today. the broken stuff today is in areas with no standards.

Q: you can set a standard, but hard to test against it.

A: Jon — it’s a research topic… secure enclaves, run encrypted and encrypted memory. Attestation of keys — small step for open standards in this area. Intel, Microsoft interested here. Trustzone, lots of pieces of this story, even if not all there yet. Open APIs, no lock in. Security companies and banks can be part of certificate transparency etc. Optimistic! lots of creative good work here, even if it will take a while.

Q: will consumers, prosumers, start to contribute to design process?

Chris — chip design is very specialist :) not a consumer game generally. Difficult, tedious, need specific kind of mind. Important companies think about open standards — and about their monopolies/dominance — benefits to consumer of stable platform even if there’s a monopoly. Cannot imagine consumers feeding back to chief architect. But ARM’s customers feed back a lot.

Phil — plausible scenario of connection between social media and hardware data streams.

John — open source is everything from the long tail on github with 2 occasional contributors, to Linux. There’s a core, quite small, of kernel developers. Open, but not a crowd. Those people are an elite, and treated as such.

Jon — exceptions — an open RISC5 processor, FPGA programming widely used in network acceleration etc. Hard to program though!

Q: National Grid is a big IOT install. There’s sensors, actuators, and wide control eg load management. The home is part of this. Should there be control, eg to stop all the kettles being turned on nationwide and crashing the grid? use that as legal framework to drive improvement

Chris — hard to make a case that net connected toy is critical national infrastructure. UK can’t go it alone, need global standards. whilst we know there are IOT security issues, the number of actual bad things is small, and usually not serious (consumer toys). So it’s sort of not falling apart. No point in scaring politicians.

Q: are there any secure IOT devices?

Chris: there’s no provably secure. You just have to be ahead of the attacker. There are some good things coming out, newer devices with trustzone. Depends on the value of the attack too. Games consoles pretty secure.

Jon — should always say : what is the threat model? if you have smart meters and generators in all the millions of homes, there’s a threat model. The smart cat flap isn’t that threat model. Need to write out the threats, and cost benefits. Earlier question about longer term attacks — that’s a big thing. people are deploying stuff, even though there’s no great security rush. Multivector attacks could make a big thing happen, multiple zero days at once! Risk is: lots of small annoying things, not a big bang. Like in the cloud, the leaks of PII that keep happening, lots of little losses of money refunded by credit card providers… Smart meters locked down so bad they are siloed. Except for the network deployment single key issue, which could have had a nice attack where all the appliances turn on at once if they thought electricity was 0p…. the generator companies would totally sue, right?

Q: IOT not really about data, it’s not the social media stuff we heard about earlier. Really it’s boring. Conflating tech, threats, everyday concerns about facebook. IOT isn’t that, it could be but the business models aren’t there. The data companies in IOT give you insight, but more is not necessarily better… Telematics in cars is interesting, data is funneled to a cloud and surely lots of people would like that data, although it’s vertically oriented today. …

Phil — security in terms of personal security in this topic. But not national security… political elites don’t think about business models. The Kremlin doesn’t think about them. Elites will make inferences from whatever data they can get, including to create social control. We need to get ready for applications of new tech in ways designers could not anticipate. (We are seeing this now with social media.) With IOT we can think about it before it becomes a real problem. A good project for us now is understanding the political implications of the data flows.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.