Does SIZE (of LLMs) Matter?
Let’s compare the performance and capabilities of Tiny LLM and verify when too Small is really too much. — Part 1
There is an ongoing trend in the Hugging Face community, to create and opt for small sized Language Models. A few of them are called Tiny LLM, for other they are simply Small Form Factor language models.
But why? I mean, we all know that you need a Big model to perform good… isn’t it?
The reality is that the mentioned above statement is not true 100% anymore. Let’s see together why!
Mini, Tiny, Small, Sheared & Quantized
I did an interesting (at least for me) exercise on Hugging Face: I looked for all the models less than 3 Billion parameters in quantized version.
- Mini Models are usually models with less than 1B parameters
- Tiny/Small Models are models below 1.5B parameters
- Sheared models are pruned models from 7B or higher reduced between 1.3B and 2.7B parameters
- Quantized models, from 3B parameters onward…