What Would AI Look Like If We Built It Using System Safety?
How does the science of safety fit into software development?
For a couple years now the area I have been most interested in as a software engineer has been safety. Specifically how to account for product safety in the software development process.
There are a lot of industries that have mature — if not effective — process around safety, but the software industry barely considers it. This is particularly troubling given the impact software has on privacy, criminal justice, medicine, transportation logistics, and other areas where decisions become life or death super quickly.
What should software engineers know about safety? There’s no better introduction than the complications and challenges around artificial intelligence. It’s hard to discuss safety with teams that are building web apps that may seem cutesy and harmless, but with artificial intelligence the threat to safety is already here.
What is System Safety?
Research around system safety pops up under different terms and in different industries. Human factor design, is a common one, as well as ergonomics but I avoid that term because for most software engineers ergonomics is about keyboards that won’t give you carpal tunnel and Eames chairs. A specific subset of system…
