Get Insight from your Business Data — Build LLM application with PEFT (with LoRA) using 🤗 Hugging Face

In our second blog on LLM’s series ( Build LLM application with fine tuning), we have discussed how we can build LLM application using full fine tuning with our own business data. There were two issues with that approach.

1. As full fine tuning update all the weights of the model so it requires enough compute and memory to train the model as in pre-training. In general LLM models have parameter in billions (like GPT3 has 175b and even smaller model like falcon-7b has 7b). Even training also adds other parameter apart form training weights like Optimizer States, Gradient, Forward Activation and Temp Memory which increase the total parameter size from 12 to 20%. So we need a way to fast track the training with less compute and memory.
2. Other issue is Catastrophic forgetting in which the full Fine tuning can increase the performance of the model on a particular task but on the other hand that may lead to reduction in ability on the other task. As in PEFT, most of the weights of LLM are not changing, it’s less pron to catastrophic forgetting. Even we can have different PEFT weights per task.

PEFT (Parameter Efficient fine-tuning)