Salesforce’s CodeRL Achieves SOTA Code Generation Results With Strong Zero-Shot Transfer Capabilities

Large-scale pretrained language models (LMs) have shown promising results on simple code generation tasks, but they have several limitations: training models with only next-token prediction objectives leads to accumulating errors, and neglecting potentially meaningful signals from unit tests results in poor generalization capability when facing complex unseen coding tasks.

A Salesforce Research team addresses these issues in the new paper CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning, proposing CodeRL, a novel framework for program synthesis tasks that employs pretrained LMs and deep reinforcement learning (RL) and achieves state-of-the-art performance on the challenging APPS benchmark while demonstrating impressive zero-shot transfer capabilities.

The team extends the Salesforce CodeT5 (Wang et al., 2021) unified pretrained encoder-decoder transformer architecture as CodeRL’s backbone. Although CodeT5 pretraining tasks such as masked span prediction (MSP) can benefit code understanding tasks, they do not necessarily align with program…