[Research Paper Summary] ReAct: Synergizing Reasoning and Acting in Language Models

ReAct: Synergizing Reasoning + Acting is a research paper that explores the potential of large language models in solving complex tasks that require both reasoning and decision-making capabilities. The paper proposes a novel approach called ReAct, which combines internal knowledge with external knowledge to support reasoning and decision-making.

The authors of the paper start by highlighting the limitations of current language models in solving complex tasks. While language models like GPT-3 have shown impressive performance in language-related tasks, they still struggle with tasks that require reasoning and decision-making. The authors argue that this is because current language models lack the ability to interact with external environments and retrieve knowledge from them.

To address this limitation, the authors propose ReAct, a framework that combines internal knowledge with external knowledge to support reasoning and decision-making. ReAct operates in a question-only setup, where models only receive the question/claim as input without access to support paragraphs. The framework uses a simple Wikipedia web API with three types of actions to retrieve external knowledge.

METHODS

The authors test ReAct on two types of tasks: reasoning tasks and decision-making tasks.

1. For reasoning tasks, the authors alternate the generation of thoughts and actions so that the task-solving trajectory consists of multiple thought-action-observation steps.
2. For decision-making tasks, the authors let the language model decide the asynchronous occurrence of thoughts and actions for itself.

The authors evaluate ReAct on two benchmark datasets: HotPotQA and FEVER. The results show that ReAct outperforms several baselines on both datasets, demonstrating the effectiveness of the proposed framework. The authors also conduct an ablation study to analyze the contribution of different components of ReAct to its performance.

RESULTS ANDOBSERVATIONS

ReAct’s structure improved groundedness and trustworthiness but reduced flexibility, causing more reasoning errors. Informative knowledge retrieval was crucial for ReAct’s success. Combining ReAct and CoT-SC methods yielded the best results for prompting LLMs in HotpotQA and Fever tasks.

PaLM-540B prompting results with respect to number of CoT-SC samples used.

CONCLUSIONS

The paper concludes by highlighting the potential of ReAct in scaling up to train and operate on more tasks and combining it with complementary paradigms like reinforcement learning. The authors believe that ReAct can further unlock the potential of large language models in solving complex tasks that require both reasoning and decision-making capabilities.

References

1. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022). React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.