Small Data is Big in AI : Train-spotting at France is AI
The third edition of France Is AI, major meeting of the French ecosystem of artificial intelligence, was held at Station F Paris on October 17th and 18th and was organized by France Digitale. The event gathered more than 65 renowned speakers and 2000 attendees.
Exchange, discovery and innovation were on the program of this event that brought together researchers, developers, entrepreneurs, managers and enthusiasts.
The first day was dedicated to technology and science with talks from Heads of AI Labs at large corporations including Twitter, Huawei, Criteo, Netflix, Facebook, Google. While the second was around the AI ecosystem and future strategies.
As France is AI 2018 Conference closes its doors, the AI community realizes clearly that today’s AI state of the art comes with as much advancements as questions about the future, regarding technology trends and data usage.
As an AI/NLP researcher, I will be talking about these trends that arise from this conference. I will also try to highlight the future AI trends and issues.
Bias/Context aware Machine Learning
Despite the different tasks tackled by each company, the issues were not that different. Netflix head of AI, Yves Raimond talked about recommendation systems and particularly on how we shouldn’t fall in the trap of overfitting the training data which represent the past. When predicting the future we should take into account the fact that everything changes. One way to avoid overfitting the past is to add contextual variables so the future will be predicted having the current context and the user contextual variables. In that case, time is embedded as a continuous variable in the system. He also talked about integrating causality in the recommender system by using the client feedback in the prediction.
IBM AI CTO, Guilhaume Leroy-Meline talked about fairness in AI and particularly how to remove and detect bias in machine learning models. He presented their open source library AIF360 which performs this task and allows also to learn to act in order to favor the unprivileged features in the learning process like gender for example.
Next, the Digital Vision Center director Nikos Paragios explained why AI can’t, currently, go beyond human in the medical field. The main reasons are the variability in cases, the lack of data and also their quality. He recommended to set 5 features in order to apply ML properly in medicine. You need to have a well defined task, the right data (quantity and quality), the right methods, the right team and the right expectations too. He also talked about the robustness issue of ML systems where performance drops when the input changes.
Olivier Grisel, a software engineer at Inria, presented the new release of Scikit-learn and its major improvements. The main enhancements are about the feature engineering flexibility and the early stopping feature which has been added to many classifiers. This last release is also scalable to parallel computing allowing people having clusters to benefit from the parallelism.
Is Deep Learning enough for NLU?
A common NLU issue was tackled by many speakers : deep learning is purely data driven and performs nor shallow reasoning. As confirmed by Mila’s director Yoshua Bengio, learning distributional representation of words will never lead to understand the language. If we want to learn an alien language we will observe what aliens do, the context in which they speak and try to get their intention. That’s what lacks actually in current NLU techniques. In the opposite side of ML NLU, we find symbolic AI/NLU which has been among the first techniques used in NLU. Symbolic NLU uses structured data such as knowledge bases. Models are able to learn from structured data by seeing only few examples. However, such data is expensive to produce and doesn’t model uncertainty. The current Facebook AI research conjecture, presented by Antoine Bordes, is to benefit from both worlds.
To this end, it is possible to learn continuous representations of knowledge bases (KB) in the same way we learn continuous representations of words. In the KB case, the vectors will represent entities and a similarity function will represent the relation such “is part of”.
Will Nano Neurons kill Transistors?
CNRS-Thalès were also there and Julie Grollier, a research director, talked about the future hardware nano neurons. Actually, when comparing hardware processors to our brain, hardware is less efficient than our brain. One of the main differences is that in our brain, memory and processing happen in the same place while in machines processors and memory are separated. So recent works are trying to go beyond the current transistors which are just switches to create new hardware neurons where memory and processing are fully closer. These new nano neurons are non linear nano radios composed each of two small magnets that transfer information through their spinnings.
An edgy subject was addressed by the Microsoft France CTO CSO, Bernard Ourghanlian, who talked about quantum computing and how fast it can solve combinatorial problems that current computers take ages to solve. He also thinks that there is no chance to apply deep learning as it is now on quantum computers which perform mainly linear operations. So he recommends, to start thinking of new techniques or adapt current ones to quantum computing as quantum computers are not too far away.
Small AI : the new AI using Small Data
The main observation I made during these intensive presentations is that data availability is a major AI issue slowing down advances in different tasks. Researchers can’t agree more on the necessity of rethinking the current AI and ML techniques in order to exploit smaller datasets. Deep learning does a good job in many tasks but requires so much data so it is not applicable in all levels/organizations. So the future direction in AI will definitely be the “small data AI” and the goal will be to achieve firstly, the same performance using less data.
On top of that, a clear consensus emerged on the necessity of extending current deep learning NLU approaches using reasoning on knowledge corpora as in Symbolic AI. This latter has a significant ability to achieve language understanding that purely data driven machine learning prevents from exploiting. It seems that some research need to be done in order to combine both methods while keeping low computational time.
We are ALL still searching for the right tools and concepts to improve AI. We are all equal in this quest from small startups to big corporations. Events such as France is AI are an opportunity to go further by exchanging points of views and identifying scientific obstacles. Undoubtedly, everyone has a chance to bring a significant contribution to the scientific community and solve part of the challenge. That being said, collaboration remains the shortest way to tackle scientific issues.
