Meta AI Open Sources Code Llama: A SOTA Code-Specialized Llama 2

Large language models (LLMs) have gained popularity in wide spread real-world applications, their impressive natural language processing capability enable them to be commanded and prompted to handle different tasks. Moreover, recent studies found that LLMs’ efficacy can be boosted significantly by utilizing large, in-domain datasets, some of the prominent user cases includes code generation and infilling.

In a new paper Code Llama: Open Foundation Models for Code, a Meta AI research team releases Code Llama, a family of code-specialized Llama 2 models for code generation and infilling, which achieves state-of-the-art performance against open models on code benchmarks.

The team releases a family of code-specialized Llama 2 models with three main variants in three sizes (7B, 13B and 34B parameters):

1. Code Llama: a foundational model for code generation tasks.
2. Code Llama — Python: a version specialized for Python.
3. Code Llama — Instruct: a version fine-tuned with human instructions and self-instruct code synthesis data.