Learning Generative AI

Make Local MCP and Run on Local LLM

Experimenting Generative AI Without The Need of Internet

Photo by Domenico Loia on Unsplash

We explored ChatGPT, a powerful large language model (LLM) hosted in the cloud. We also learned about MCP (Model Context Protocol) — the next-generation API for AI, often dubbed the “USB-C of AI.” Like ChatGPT, MCP services are also cloud-based.

But what if you want to understand the concept, experiment with it, and build your own solutions — all without relying on paid cloud access?

How can you connect your own API, and create a local MCP setup that works with your locally running LLM?

Yes — this article shows you exactly how to do that.

Setup local LLM

An LLM (Large Language Model) is simply a model that allows you to interact with a computer using natural human language — and receive responses in the same way. ChatGPT is one example of this.

If you have an LLM running locally, you can carry out these natural language conversations entirely offline. This is made possible with tools like Ollama, which lets you download and serve models such as DeepSeek and LLaMA 3 directly on your machine.