Natural Language Processing Will Dwarf Machine Learning and AI

Forget sliced bread. It’s the greatest thing since the written word.Everything humanity has developed in the past 10,000 years is the result of a singular discovery. More important than the wheel, or the discovery of relativity, or any other great invention, it was the now long forgotten creator of the written word who allowed us to expand on ideas, and keep a record of all of human knowledge.

We abuse that creation today. The written word is so common place that we send through through the air waves, and post them on transient bits of magnetic material so we can write over them. No longer do we carve things in stone, or think long and hard for our words which we lay down in beautiful penmanship.

We use words to spread lies, and falsehoods, and manufactured facts. We use them for marketing, and propaganda, and we cheapen them with hate speech.

Despite all that, we also generate more knowledge, facts, and data points each hour than we did in a millennium back when we carved in stone and drew glyphs on walls. We store more words per second than all of the collective works of the Library of Congress holds in paper and leather.

Machine Learning is a bit of a misnomer. ML is automated pattern recognition. The machine doesn’t learn its ABC’s or how to say friend in Spanish. This is not Sesame Street for computers. AI is automated application of rules based on patterns either observed or prescribed. The computer is not as intelligent as a cockroach.

This isn’t to bash ML and AI. They are important for everything from the traction control system in a performance car like a Nissan GTR that uses an AI and ML pairing to adjust power to each wheel, and the stiffness of each component of the suspension, to stock and bond trading that now happens in nano-seconds.

No, ML and AI are important, but it is Natural Language Processing that is going to change the world in ways we had never before imagined.

We can’t store all of human knowledge. We can’t codify it in to ML or AI. It is a really big set of data and it grows constantly. But we can extract it. We can boil it down to the core knols of information and make it available. That is what NLP can do.

Consider how much redundancy there is in all of the text books for teaching Fourth Grade Science. Six different text book companies make a book that is 250 pages long, and those books all teach the same information that ends up on the standardized test. Now imagine how much of that same information is in the High School, and College text books.

Consider how many News Papers re-report the same information. Plus blogs. Plus Facebook Posts. All of that redundancy takes up space. Space we don’t really have. So we delete it, it disappears from the internet, and supposedly the important stuff stays on because of popularity. But what about important stuff, nobody knows is important? What if 5 kids who live in a given apartment over 20 years all develop lung cancer? What if a Sixth Grader discovers a new method for solving a hard math problem and nobody notices?

NLP is not just about building a Siri, or a Voice Assistant. It is not just a means to mine data for better targeting spam. While it is those things, NLP is really the method by which we will compress all of human knowledge in to something that is indexable and searchable.

NLP will allow us to determine if something is “new” information. To gather all of the information on a topic, and only get rid of redundant information. NLP is what will power the Google Killer. It is what will change how news is distributed. It is what will make knowledge workers and middle management obsolete.

ML and AI will draw on the resources of NLP to make decisions, but until computers and people can understand each other, that won’t happen. Dolphins and Primates are really intelligent. They learn, they interact, but they lack the ability to communicate with us (or in very limited ways) and so they are not a part of our society. In short Until we have NLP we won’t have a C3PO, or a Star Trek style voice interface for doing programming of our holodecks. The world just won’t be as awesome until that day.

Photo Credit: