Lingua — smart language processing for Python

After 2 years of work, we, Lingua are ready to show you a really small demo of our upcoming product that makes communication with machines easier. Unfortunately, our product is at an early testing stage and we cannot publish source code or interactive demos.

Lingua uses more than 50 algorithms (using statistics, heuristics, machine learning), all running using the basic multiprocessing package, that manage to understand all kinds of English expressions. We’ve managed to do:

• Entity separation
• Relationship extraction
• Topic recognition
• Word sense distinction (with stemming)
• Context preservation

An example

Here, we’ve defined a language processor that can parse temperature questions:

By calling ExampleProcessor().parse(“Hi computer, tell me the temperature”) after 100ms we get the following response:

In addition, the <LanguageMatch> argument contains a lot of data about the context, how sure we are about the match, the tone of the expression and some words that might be useful (that are not stop words).

A more complicated example

Let’s observe input/output:

• Input: Hi, I am John!
• Output: Hello, John!
• Input: What’s Sun’s mass?
• Output: Thanks for asking about [<lingua.dictionary.english LightSources>, <lingua.dictionary.english Sun>, <lingua.dictionary.english Stars with proper names>]
• Input: Why did John ask his brother?
• Output: Thanks for asking

As our algorithm improves and our testing stage finishes, we will update you and release a demo or source code. In addition, we will talk more about our algorithms and how we try to optimize Python (2.7.6) ☺