DeepMind & UCL Fine-tune a 70B Parameter LM to Generate Statements Agreeable to Humans with Diverse Opinions

Large-scale transformer-based large language models (LLMs) have revolutionized the field of natural language processing (NLP), achieving humanlike fluency while learning various homogeneous human preferences from their massive training data. People today however are hardly homogeneous — it could be argued that we are more heterogeneous than ever. Might it be possible to leverage LLMs as mediators tasked with generating statements that will find agreement among people with diverse views?

A DeepMind and University College London research team tackles this challenge in the new paper Fine-tuning Language Models To Find Agreement Among Humans With Diverse Preferences, fine-tuning a 70 billion parameter LLM to generate statements that maximize agreement among a human group with diverse written opinions. The team’s top model achieves a more than 65 percent preference rate compared to the best human-generated opinions.

The team first creates a corpus of questions related to social and political issues in the United Kingdom, such as, “should we raise taxes on the rich?” They…