An Understandable Language Processing
While recent advances in language processing with Deep Neural Networks (DNNs) present high-quality translation and classification of the texts, the Holy Grail of the language learning remains missed. That is, while humans appear capable to acquire languages in unsupervised way based on everyday conversations easily, the DNNs require extensive supervised training. Moreover, the humans are capable to acquire explainable and reasonable rules of connecting words into sentences based on grammatical rules and conversational patterns and have the grammatical and semantic categories of words well understood, with all that synonyms and homonyms. On the opposite, the very advanced DNN models remain black boxes not being understandable and inspectable.
That is why we are looking for Understandable Language Processing (ULP) which would let acquisition of the language, comprehension of textual communications and production of textual messages in reasonable and transparent way. One of the directions of our studies is Unsupervised Language Learning (ULL) project being executed by SingularityNET. The goal of the project is to enable acquisition of the language grammar from un-labeled text corpora programmatically in unsupervised way, with the learned knowledge stored in a human-readable and reasonable representation in form of Link Grammar dictionary, as it has been explained in our previous publication on the matter.
The latest update on the Unsupervised learning project has been delivered on AGI-2019 conference in Shenzhen, China. The paper called Programmatic Link Grammar Induction for Unsupervised Language Learning (with slides) has been presented. The following video shows that grammar can be learned based on the high-quality parses provided as the input, but the problem of getting these parses in unsupervised way needs more work ahead. One of the possible advances may be achieved with implementation of the curriculum learning approach so the learning curve is built up gradually, adding more complex training sets after the simpler texts are acquired by the learner system. See the following full video tutorial on the latest advance in this work.
The key interest in the Understandable Language Processing relies on that the study falls under the domain of the Explainable Artificial Intelligence (XAI), which targets to have the learned knowledge models to be stored in a way comprehensible by humans rather than “black-boxes” or at least have the “black-box” models transferable to reasonable and explainable representations. As the following illustration suggests, the same recognition of prediction model can be kept either in “black-boxed” coefficients of connections between multiple DNN layers (on the right) or in weight or truth values on the links in a graph of a bayesian, probabilistic or fuzzy logic network (on the left) used for inference. In such a case, there is a possibility to explain decisions made or solutions found by an artificial intelligence system (“explain”). Also, it opens the door for Transfer Learning, where the knowledge learned by one artificial intelligence (AI) system can be transferred to another system, or this knowledge may be entered into an AI system after being created by human expert in formal way.
Further extension of the understandable language learning approach can be seen as the natural language technology based on complex hierarchical patterns encompassing semantic, lexical, grammatical and morphological information for a language or specific domain, as it has been suggested in our earlier work (with slides) and practically implemented in the Aigents framework. The following video presents how the complex sequentially-hierarchical patterns can be employed for practical Natural Language Processing (NLP) tasks such as text classification, entity extraction and property attribution.
The next NLP development beyond grammar learning, classification, entity extraction and property attribution is text production, which can be addressed in “understandable” way as well. Let say, we have the whole set of the ontological and linguistic knowledge encoded in the enormous hyper-graph incorporating semantic and grammatical relationships between words, categories of the words, parts of speech, simple and compound concepts, languages, moods and intents incorporated into symbolic network like it is shown on the following illustration. The important part of the network would be aggregative symbols for disjunctions (“OR”), conjunctions (“AND”) and sequential conjunctions (“SEQ” with either straight -> or reverse <- order), so the combinations of the words and symbols can be assembled into lemmas, synonyms and homonyms as well as phrases or templates of the phrases or semantic relations.
In such a network, the language production can be achieved with allocation of attention in a graph, i.e. activation spreading across the links in a network, accordingly to the connectivity values on the edges of the links (which may vary depending on the kind of the network). For instance, the following illustration shows how the attention spread in the network above to specific compound concept, language and the mood, the combined attention leads to specific phrase being identified for production by the system.
Further, stay tuned our work being done open-source at SingularityNET language learning project on GitHub and Aigents News Monitoring framework based on Aigents project on GitHub.
