News: new collaborative project with CISCO on large language models

I’m really excited to report that we have a new collaborative project with funding from Cisco. The title and the brief of this project are as below. More interestingly, I’m looking forward to working with Ali Payani from Cisco, as well as Daga and Antonio from our side here in Edinburgh, on this.

Logically consistent and coherent querying from large language models

Large language models play a crucial role in the new generation of generative AI technologies. Many of these models are complex and incredibly extensive, comprising millions of parameters that are difficult to comprehend. Anecdotal evaluations suggest that these models often provide inconsistent answers. They do not yield mathematically or logically consistent answers, although they do exhibit some level of reasoning. Some of this reasoning is by training on deductively closed data and in other cases, clever prompting as well as chain of thought style inference is often useful to obtain coherent answers.

In this project — a collaboration between Cisco (Ali Payani) and the University of Edinburgh (Vaishak Belle, School of Informatics) — we are investigating new frameworks and challenge problems to consider logical consistency in the era of large language models. Among other things, we hope to use large language models as effective encoders to transform natural language to symbolic formulations so that symbolic reasoners can be used as oracles.