Computational linguistics

Last edited 5 days ago

There is a broad area of research called computational linguistics that tries to make computers understand and imitate human language. Solving this is the same as passing the Turing Test (the benchmark for AI). While the whole problem is very hard, researchers have made progress in deconstructing the problem and solving parts of it.

Linguists have decomposed the problem of understanding and generating natural language utterances into several layers of analysis (see Figure). One might think of these as equivalent to the OSI stack for networks — you need to solve the lower parts of the stack to get to the layer above.

  1. Phonetics structures sequences of phonemes (the smallest units of sound, equivalent to letters) together into morphemes (roots, prefixes and suffixes).
  2. Morphology structures sequences of morphemes into words.
  3. Syntax structures sequences of words into clauses and then into sentences or utterances (when spoken).
  4. Semantics is concerned with the meaning of sentences, independent of their context of use: how words, phrases and clauses relate to the world, and how the meanings of these constituents can be combined to form the meaning of an entire utterance.
  5. Pragmatics is concerned with those elements of utterance meaning that are context-dependent, and with how language is used by people to achieve their goals.

Programming Languages and User Interfaces

Like human languages, which provide an interface between human to human, programming languages act as an interface between humans and machines — they have similar constructs.

  • syntax i.e. type definitions — data.txt : File, “hello”: String
  • semantics i.e. functions — write (data.txt, “hello”)
  • pragmatics i.e. a program (an executable sequence of functions). Thus { file.close(), f.write(), f.open() } is not a pragmatic program.

The differences are that programming languages are much more limited in their syntax, much more structured in their semantics (strict rules on how to construct a function). They also have a bulk dialog interface (issue a large sequence of messages rather than being back and forth) that reduces the pragmatics problem.

User interfaces (UX) also provide a different kind of interface between humans and computers. They are even more structured than programming languages and command-lines. The syntax is limited to specific interface items (buttons, links, menu items etc.) and the semantics is deduced through standard guidelines (a link click will reveal new information) and experimentation. The pragmatics is the path that the user takes through an app to achieve a certain outcome — these are more two-way and less “bulk-oriented” interfaces than programming languages.

Conversational interfaces

Conversational interfaces lie at the intersection of human and programming languages. Like the latter, they provide a human-computer interface; like the former they are less structured and more two-way interactive.

Conversational Interfaces touch upon the realm of pragmatics. It requires semantics as a base on which to build. Most computing systems interact two layers down the stack — in the domain of syntax. Interfaces between humans and computers are highly structured (button, link, screen, text) so the semantics of each interaction is pre-defined and does not need to be deduced.

So, to build a conversational interface, a system must first solve the problem of semantics — i.e. figuring out what an utterance means. This involves technologies similar to undertanding search queries — first solve syntactic problems (stemming, typos, suffixing, synonyms, homonyms etc) using a vocabulary, then solve the map the utterance to a set of concepts using an ontology. This is tricky e.g. “Do you have the time?” is not really a yes/no question, even though it seems so, whereas “Do you have a watch ?” is a yes/no question. Distinguishing between these needs understanding the semantics (meaning — concepts).

Once the semantics problem is solved (hard enough), then you get into pragmatics — where we attempt to stitch together multiple utterances to communicate intent. This requires maintaining context across utterances. Search systems have typically not ventured into this area.

Domain

A knife that can be used to cut through the complexity of the computational linguistics problem, is domain-specificity. By limiting the problem to one domain (e.g. interview scheduling, fashion purchase etc.), it greatly reduces the syntax problem (vocabulary), the semantics problem (ontology), as well as the pragmatics.