This is the first part of a deeper dive into Ollama and things that I have learned about local LLMs and how you can use them for inference-based applications. In this post, you will learn about —
- How to use Ollama
- How to create your own model in Ollama
- Using Ollama to build a chatbot
To understand the basics of LLMs (including Local LLMs) you can refer to my previous post on this topic here.
Some background
In the space of local LLMs, I first ran into LMStudio. While the app itself is easy to use, I liked the simplicity and maneuverability that Ollama provides. To learn more about Ollama you can go here.
tl;dr: Ollama hosts its own curated list of models that you have access to. You can download these models to your local machine and then interact with those models through a command line prompt. Alternatively, when you run the model, Ollama also runs an inference server hosted at port 11434 (by default) that you can interact with by way of APIs and other libraries like Langchain.
As of this post, Ollama has 74 models, which also include categories like embedding models.
