Worlds in Collision

At present, theoretical linguistics and artificial intelligence exist as two separate worlds, and there is very little interaction between the two of them. In fact, they’re on a collision course — they just don’t know it yet.

Five years ago Peter Norvig, Director of Research at Google, picked a widely publicized fight with the leading linguist Noam Chomsky, insisting that the current statistical approaches to Natural Language are highly successful. Yet, if these approaches are as excellent as Peter Norvig claimed them to be, how is it that five years later Mark Zuckerberg still has to state in relation to what he calls “open-ended” commands:

“No commercial products I know of do this today”.

If he hadn’t believed all the hype surrounding AI, Mark probably would not have announced a year ago that, as part of his “personal commitment” for 2016, he would try to put together existing technologies to make a Natural Language Interface for the “Smart Home”. Apparently he was so impressed by the successes of AI in other fields that he thought “unsupervised learning” would also be able to handle “open-ended” commands.

Needless to say he was not able to find such a solution among existing commercial platforms.

And yet, Mark was not actually wrong in his recognition of the important role that traditional AI in general, and “unsupervised learning” in particular, are likely to play in dealing with Natural Language. They just cannot do it alone. It can only be done by working together with experts in theoretical linguistics who are the only people today who know a thing or two about the inner workings of language.

In order for established AI algorithms to be able to learn to understand the meaning of Natural Language, there must be a way for one to communicate this meaning to them. For that a precise agreed-upon format in which the meaning of linguistic expressions can be clearly and easily stated is needed. In addition to being an indispensible prerequisite for using AI with Natural Language, it is also going to be urgently needed as a language which both linguists and AI specialists understand. This ‘lingua franca’ is of paramount importance.

Just how unwilling the leaders of traditional AI are to even consider scientific insights about human language, one can learn from the fight that Google Director of Research Peter Norvig picked with Noam Chomsky. Nevertheless, many developers are gradually beginning to understand that in their quest for Actionable Linguistic Intelligence they’re going to need all the help they can get.

The world of AI is not the only one that must brace itself for impact. The same is true for the world of theoretical linguistics. Very soon it will be impossible to do theoretical linguistics without a massive use of computer algorithms as is the case in many serious sciences.

Consider, for example, high energy physics. The use of computers for the processing of information is absolutely necessary for doing research about elementary particles. But unlike physics, for theoretical linguistics, the numerical computation algorithms are not enough. Instead, algorithms of AI will have to be used.

In speaking about the “collision” of these two worlds, I do not mean infighting and the mutual exchange of put-downs. I speak about the profound ways in which these two disciplines are inevitably going to impact each other in the immediate future.