We as AI practicioners love to fine-tune language models. After GPT arch becoming mainstream (and so llama), many fine-tuned models were created trying to distill GPT-3.5 or even GPT-4 right from the api. But, fine-tuning language models is tedious and time consuming and not always works as expected. 1 billion parameter models excel on good performance after typically after 3–5 epochs. So, how do we get faster convergence?
Introducing palmer-001
palmer constitutes a collection of language models, each encompassing approximately ~1 billion parameters. These models are finely tuned to serve as foundational models, eliminating the need for tailored prompts in various tasks. Is a middle-step to make your chatbots better in less time.
Why it matters? Versatility of being trained on no prompts allows for further fine-tuning using specific prompts and additional data or direct utilization in downstream tasks, akin to any conventional base model.
palmer embodies a harmonious blend, possessing a degree of predisposition to function as an assistant while also possessing the capability to predict the next word based on its extensive internet knowledge base. As a llama 2 model, it seamlessly integrates with your preferred tools and frameworks like llama.cpp or AutoGPT-Q.
You can download palmer-001 here: https://huggingface.co/appvoid/palmer-001
