How Apple Missed An Opportunity With Siri

A few months ago Sal Soghoian announced that his position as Product Manager of Automation Technologies at Apple was being eliminated due to business reasons. Sal integrated and grew AppleScript support within existing Apple applications, and advocated for AppleScript use within third-party applications as a way to empower users. That work resulted in the development of Automator.

In late 2002, Soghoian joined an ad hoc team of engineers developing an application for creating and running automation workflows. After a year and a half of difficult but steady development, Soghoian showed the application to Steve Jobs. It was just one week before Apple’s 2004 Worldwide Developers Conference (WWDC). Jobs was impressed with the new tool and asked Soghoian to appear at the WWDC keynote to demonstrate Automator for the first time to attendees.

I was fortunate enough to hear Sal speak at a Macworld event many years ago. He spoke with a passion for giving power to users, not just the pro users, but the everyday users. He believed that “the power of the computer should reside in the hands of the one using it”, and he was working on tools and integrations to support that belief.

As an engineer, everything about automation makes sense to me. We can’t, and shouldn’t expect software and interfaces to be designed around every possible use and need we might have. The magic lies in the ability to extend software easily to do our bidding. Automator was a good step in the right direction, but I think solving the problem is difficult for a few reasons:

  1. You have to have an entire ecosystem–it’s no good if I can move data out of one app, but can’t move it into the other. That goes both for Apple software, and third-party software.
  2. The entry point was off. Automator was okay for one-off tasks, but could even feel a little clunky for those. When you needed a repeatable task, you had a lot of bad options: workflows, applications, services, etc. All of those are useful but you end up with a persistent “thing” you have to find and interact with (an app, a service menu, etc).

Apple has experience building ecosystems–the app stores (mac, iOS, AppleTV). Apple also has experience in cross-app integration with iOS and Mac app plugins. The ecosystem problem is solvable, but what about the entry point thing?

Hey Siri

Voice is the next big way to interact with technology. You’re seeing it already with Siri, both on the Mac, and on iOS. You see it in cars, in the home with Amazon Echo, Google Home, AppleTV, etc. The problem with voice is that it’s really hard to get right all the time–and if you get it wrong, it’s hard to gain back a user’s trust or faith in the interface.

Apple has announced that Siri APIs will be available to developers, and that goes a long way towards making voice more powerful, but it’s stopping short of fully empowering the user. It gives app developers the ability to build in “shortcuts” for tasks that their users might want to perform, but it doesn’t give the user the ability to decide on what those are.

If you could build your own Siri command to “resize the selected picture”, or “cleanup my downloads folder”, the user would be in control of the computer in a more intuitive and powerful way–and they aren’t waiting for an app announcement about how Siri now has 200 new commands (most that they’ll never use).

Voice is a also hidden interface–it doesn’t take up visual space. It doesn’t rely on icons or menus, and that makes it limitless. It’s also a smart interface–you can teach it that when you say “clean my downloads folder” it’s the same as “archive my downloads”.

Another benefit is that commands or workflows built by a user have context, that means Siri gets it right more often. Siri knows you’re in a given app, so it should look for relevant app commands first–which makes getting voice right a little easier.

In Closing

I think voice actually moves the “automation” needle in a big way. With Siri so heavily integrated into OS X it seems like an obvious move. If anyone could pull it off in a natural way it would be Apple. All that being said, Apple’s move of ditching Sal and potentially the entire Automation team is a bit of a mistake in my opinion, and ultimately points towards moving away from user automation, not towards it.

I think there’s a reason Steve Jobs pushed Automator to a worldwide stage a week after he saw it–there was something there. It seems like Apple was looking for a sweet spot in making everyday users feel as powerful as the engineers that built the system–AppleScript seems to be evidence of that. I think they got a lot closer than people realized, just wish they hadn’t given up.