How to decide which LLM to use for an application

Learn about the differences between types of LLMs to navigate the tradeoffs and choose the best LLM for your application.

Training an instruction tuned (instruct) and chat-based LLM. The figure was adapted from LLMs: a premier and gastroenterology applications, 2024.

Once you identify a problem that is suitable to tackle using a Large Language Model (LLM) and build a proof-of-concept (PoC), then you need to decide what LLM is a good choice for your problem. Usually, in the PoC stage you want to demonstrate that the problem you identify can be solved with an LLM and for that it is better to start with a large foundational model. However, in production you might not want to foundational model due to their longer latency or their cost. In this post I summarize the factors you might want to consider when deciding which LLM to choose from based on their size of type.

LLM parameter sizes

The size of an LLM is determined by the number of parameters that were used in the transformer network. The parameters are the values that are “learned” or adjusted during learning. In theory, the larger the number of parameters, the more complex the relationships it can “understand” in the language. The number of parameters in an LLM can go from hundreds of thousands to trillions. For…