The Tower of Babel (and machine learning as the wrecking ball)
Facebook says it is abandoning the Bing translation system it has been using since 2011 and will replace it with its own translation system based on machine learning.
With the right tools, machine learning is now an increasingly important sustainable competitive advantage that is being used in more and more areas.
Companies that want that competitive advantage will have to start familiarizing themselves with tools like TensorFlow, FBLearner Flow, AzureML, WhizzML,SystemML, Amazon Machine Learning and others that will be used to build the engineering and processes of the future. These are tools that tend to lower entry barriers and put machine learning in the hands of people who are not necessarily data scientists, although it is important to understand and interpret data correctly, as well as being aware of its potential.
At the same time, this shift highlights the importance of data bases, which are used to feed and educate algorithms. Facebook is able to develop its own translation system thanks to its huge stock of translated updates that have been evaluated by users through a simple, five-star system that keeps the process lively and that can generate a huge flow of versions to process. It is the essence of machine learning: each error, each unsatisfactory version, is learned by the system as part of a continual learning process that a machine cannot forget, and that absorbs knowledge far more efficiently than the human brain and that can also carry out other iterative processes. Facebook generates more than two billion updated translations per day in 40 languages and 1,800 linguistic pairings.
Finally, there is the question of understanding translation as an example of what this process means. We all know what the old translation systems were like: imperfect because they translated word for word that would often come up with “amusing” results that themselves often required translating. We have now moved toward systems able to evaluate a term within a context and to choose a meaning based on that, which while still not perfect, have notably improved the result.
What we now have is a machine able to take a sentence, compare to an infinite number of previous uses that have been evaluated and then generate dynamic rules similar to those we would construct in our brains to carry out a similar task. It’s been a long road, but the results are significantly better, allowing us to work with language that have very different grammatical structures and that will help understand how many things work.
Why does a self-driving car based on machine learning work better than a human? It’s not just that it has better vision, instant reflexes or doesn’t get tired, distracted, drunk or fly into road rage, but it is also able to learn from any error and remember it, alongside the context it took place in, storing it in a memory system so that it will not be repeated.
The companies of the future will be able to apply this type of efficiency based on machine learning to all their processes.
(En español, aquí)
