Member-only story
Easily Train a Specialized LLM: PEFT, LoRA, QLoRA, LLaMA-Adapter, and More
Training a specialized LLM over your own data is easier than you think…
Due to the surge of interest in large language models (LLMs), AI practitioners are commonly asked questions such as: How can we train a specialized LLM over our own data? However, answering this question is far from simple. Recent advances in generative AI are powered by massive models with many parameters, and training such an LLM requires expensive hardware (i.e., many expensive GPUs with a lot of memory) and fancy training techniques (e.g., fully-sharded data parallel training). Luckily, these models are usually trained in two phases — pretraining and finetuning — where the former phase is (much) more expensive. Given that high-quality pretrained LLMs are readily available online, most AI practitioners can simply download a pretrained model and focus upon adapting this model (via finetuning) to their desired task.
“Fine-tuning enormous language models is prohibitively expensive in terms of the hardware required and the storage/switching cost for hosting independent instances for different tasks.” — from [1]
Nonetheless, the size of the model does not change during finetuning! As a result, finetuning an LLM — though cheaper than pretraining — is not…