DL & NLU

Note: Please pardon any incompleteness, for I wrote the following conjecture from my phone in a parking lot.

Idea for performing NLU with neural networks:

An LSTM network that generates a parse forest from text input. Then, separately perform A* search on the network output to construct the k-best parse trees with concurrent semantic reduction of semantic trees, grammatical conjugation of display text, et cetera.

The first component is essentially a neural network trained to represent a fuzzy, probabilistic grammar (PCFG), and processes input like a shift-reduce parser. In fact, this is more similar to a (Markov) state-transition table, like an MDP. Perhaps, reinforcement learning (RL) is better suited for NLU (to generate parse forests) than LSTMs.

Though, this is untested and likely incorrect. Afterall, Google’s SyntaxNet generates parse trees with a simple feed-forward network without recurrence. But, these parse trees are simple POS tagging, and lack much of the information necessary for complete NLU. Note, these are preliminary musings. I will not know until I try to implement a complete solution by testing multiple architectures.

The second component, parse forest search, would manually ensure output is grammatically and semantically correct. These NLU tasks require precision; fuzzy logic does not suffice. LSTMs (and DL in general) are efficient for NLP, where precision is inessential. But, (far harder) NLU needs precision for recursive grammars, grammatical conjugation, anaphora resolution, semantic and textual disambiguation, semantic reduction and validation, boolean logic, and much more.