Future Computers Speak English

Natural Language Interfaces, an Introduction

Science Fiction paints a nearly unanimous picture — in the future, computers speak English. Or at least, they speak whatever language the protagonist speaks.

The Computer responds to Spock’s queries with a soothing monotone. Jarvis is capable of understanding commands even when they’re dripping with sarcasm. The Star Wars droids use language to communicate with the humans and (inexplicably) one another.

The fiction is clear — in the future, nobody will need to learn to use a computer. Computers will be intuitive (or sentient) enough to understand us. And it seems like this future is coming true, in some senses. Siri is a long way off from Jarvis, and Cortana definitely can’t compare to, uh, Cortana. But we’re getting there, right?

Sure you are.

Kind of. The thing about these interfaces from TV and movies is that they’re not designed to be intuitive to the user, the’re designed to be intuitive to the audience. Typing is boring and reading is hard — spoken-word interfaces are the best for moviegoers. But can you imagine how confusing it would be if on the Enterprise flight deck, multiple people ever needed to use the Computer at once?

In fictional situations where lots of people need to be working on computers at the same time, the spoken-word interfaces are always replaced with something more practical and individual.

We could debate the practicality of huge monochromatic touch screens, but you get my point.

In the movies, it appears that are ease-of-use and practical usability mutually exclusive. But that doesn’t need to be true in real life. I submit that the best way for real humans to interact with real computers does involve natural language interfaces, but in a much more practical way: the Natural Language Command Line.

A Natural Language Command Line (NLCL) is an keyboard-driven interface that understands text written the way that people naturally communicate, and can do what it is told.

The keyword here is naturally. Natural communication is varied and ambiguous. There are lots of ways to say the same thing, and sometimes one sentence could be interpreted in various ways based upon the context. There are thousands of different languages, and they each have quirks and complexities that make it difficult for computers to parse them. It’s not easy to teach computers to do understand us in this way, but once they do, the result is tremendously easy, efficient, and expressive. After all, that’s what the future will really be like: computers doing the hard work for us, not the other way around.

The real innovation is just not making everything blue.

I am going to set forth a list of guidelines that a good NLCL should follow. This is loosely based upon this list that Jono DiCarlo wrote in 2008, when working on the Mozilla Ubiquity project.

Ease of Use

• The user should not need have any specific knowledge to use the interface, other than fluency in their language.
• The interface should give feedback about how it understands any given command before the command is executed, especially when the input is ambiguous
• The interface should provide its understanding feedback while commands being typed, not waiting until they have been entered completely
• The interface should suggest possible ways to complete any partially-entered command
• The interface should be equally natural for all languages

Efficiency

• The user should be able to autocomplete their input with a keypress
• The interface should provide suggestions based upon the external state of the system (e.g. filesystem, current selection, music library, calendar)
• The interface should, where appropriate, allow the user to enter input that is abbreviated or slightly misspelled.
• The interface should sort suggestions based upon input history, but this sorting should never betray the user’s expectations
• The interface must be quick in responding to the user’s input

Expressiveness

• The interface should accept commands with more than one argument
• The interface should accept ‘arguments’ of various data types, not just plain strings
• Users should be able to add new commands and extend existing commands in an easy and safe way
• The interface should allow systems to work together if it makes sense linguistically, even if those systems are disconnected
• The interface should allow users to create shortcuts to commands that they choose (e.g. text aliases, keyboard shortcuts, or GUI buttons)

The Plan

This is knowledge that I have gained while working on Lacona, an all-purpose Natural Language Command Line for Mac, which is currently on Kickstarter. Of course, I believe that Lacona fulfills all of these goals (and you should back it now!), but I hope that these posts will be useful for general knowledge as well.

As always, I stand on the shoulders of giants — especially the things written by the Humanized team while working on Enso, and by the Mozilla team while working on Ubiquity. I owe a great debt to @aza, @toolness, Jono DiCarlo, @themitcho, and many others.