Instruction back translation is a scalable method to build a high-quality instruction following language…
LLaMA 2 Long is a series of long-context LLMs built through continual pretraining from LLAMA 2 with…
As larger models require pretraining on trillions of tokens, it is unclear how scalable is curation of…
Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn…
PaLM 2 is the successor of PaLM. It’s more compute efficient and is pre-trained on a more multilingual &…
Alpaca is fine-tuned from Meta’s LLaMA 7B model. The Alpaca model is trained on 52K instruction-following…
