Fine tuning LLM using PEFT
What is LLM?
Asking Chat GPT3 😄 “Large language model refers to neural network-based model that has been trained on large amount of test data to understand human like language.”
Or more technically, it is a probabilistic model with transformer-based architecture trained on massive dataset to predict the subsequent word in a text sequence based on the preceeding words
How to make LLM work for specific task?
One recent technique for using Large language models to solve tasks is zero-shot or few shot prompting. The technique formulates a task based on text language model has seen during training. e.g to classify a sentiment of a movie we give model the text “Classify this review: I love this movie! Sentiment: __” and be asked to complete the sentiment as positive or negative as shown in the picture below.
Although prompt engineering has good performance on some task but it requires very careful prompts to design task to look like data that model has seen during training. Also it performs well on some task but not on all tasks. Is there another way? Here comes Instruction tuning