Building A Local LLAMA 3 App for your Mac with Swift
Recently, Meta released LLAMA 3 and allowed the masses to use it (made it open source).
Wanting to test how fast the new MacBook Pros with the fancy M3 Pro chip can handle on device Language Models, I decided to download the model and make a Mac App to chat with the model from my Menu Bar at all times. In this article, I will you teach you, YES YOU, how to deploy your own language model locally onto your Mac.
Setting Up Ollama / Deploying a Local Model
This part is relatively simple. Thank you Ollama developers!
First step, ollama.com and follow their download and set up instructions.
Namely, you will download the Ollama App, after opening it, you will go through a set up process that installs Ollama to your Mac.
After installing Ollama, we can download and run our model. For this article, we will use LLAMA3:8b because that’s what my M3 Pro 32GB Memory Mac Book Pro runs the best.
To do that, we’ll open the Terminal and type in
ollama run llama3This will download the model and start a Text Interface where you can interact with the model via the terminal.
Since I have run this command before, using this command simply starts the model and does not cause it download once more.
You can exit this interface with the command:
Control + dShow Me The Code / Making A Mac Menu Bar App To Chat With The Local Model
By the end, our app will look like this:
If you want to see the full project, it’s on my GitHub, here https://github.com/carlosmbe/MyMacLLAMA
I will be updating the project there with new features every once in a while. So this article will focus on the basics and essentials.
The Data Model and Fun Logic:
Things to Note:
1. The address, http://127.0.0.1:11434/ is the local host at port 11434. Which the default port that Ollama runs on. What we’re doing here is doing an API call to our own device as the server.
2. Furthermore, since this is an API Call, we need explicitly request permission for incoming and outgoing network calls in the App capabilities.
import Foundation
// Struct to decode the JSON response
struct Response: Codable {
let model: String
let response: String
}
// Class for managing application data and network communication
class DataInterface: ObservableObject, Observable {
// Store the current prompt as a modifiable string
@Published var prompt: String = ""
// Store the response to the prompt as a modifiable string
@Published var response: String = ""
// Track whether a network request is currently being sent
@Published var isSending: Bool = false
// Function to handle sending the prompt to a server
func sendPrompt() {
print("Started Send Prompt") // Log the start of sending a prompt
// Prevent sending if the prompt is empty or a request is already in progress
guard !prompt.isEmpty, !isSending else { return }
isSending = true // Mark that a sending process has started
// Define the server endpoint
let urlString = "http://127.0.0.1:11434/api/generate"
// Safely unwrap the URL constructed from the urlString
guard let url = URL(string: urlString) else { return }
// Prepare the network request with the URL
var request = URLRequest(url: url)
request.httpMethod = "POST" // Set the HTTP method to POST
request.addValue("application/json", forHTTPHeaderField: "Content-Type") // Set the content type to JSON
let body: [String: Any] = [
"model": "llama3", // Specify the model to be used
"prompt": prompt, // Pass the prompt
"options": [
"num_ctx": 4096 // Specify context options
]
]
// Encode the request body as JSON
request.httpBody = try? JSONSerialization.data(withJSONObject: body)
// Start the data task with the request
URLSession.shared.dataTask(with: request) { data, response, error in
defer { DispatchQueue.main.async { self.isSending = false } } // Ensure isSending is reset after operation
if let error = error {
DispatchQueue.main.async { self.response = "Error: \(error.localizedDescription)" } // Handle errors by updating the response
return
}
// Ensure data was received
guard let data = data else {
DispatchQueue.main.async { self.response = "No data received" } // Handle the absence of data
return
}
let decoder = JSONDecoder() // Initialize JSON decoder
let lines = data.split(separator: 10) // Split the data into lines
var responses = [String]() // Array to hold the decoded responses
// Iterate over each line of data
for line in lines {
if let jsonLine = try? decoder.decode(Response.self, from: Data(line)) {
responses.append(jsonLine.response) // Decode each line and append the response
}
}
print(responses) // Log all responses
DispatchQueue.main.async {
self.response = responses.joined(separator: "") // Combine all responses into one string
print(self.response) // Print the full response
}
}.resume() // Resume the task if it was suspended
}
}The User Interface
The Menu Bar:
We will go to the App file to make the app operate solely in the Menu Bar. For more information on the Menu Bar Section. Check out this article, its what I used as a foundation.
https://sarunw.com/posts/swiftui-menu-bar-app/
import SwiftUI
@main
struct MyMacLLAMAApp: App {
var body: some Scene {
//Create instance of Data Interface for the app
@StateObject var appModel = DataInterface()
//Create a Menu Bar for our app and have it have the brain icon
MenuBarExtra("My Mac LLAMA Bar", systemImage: "brain"){
//When Clicked The Menu Bar Will Show the content View
ContentView()
.environment(appModel)// Pass the appModel environment object to ContentView
}
//This modifier lets us show a Window in the Menu Bar
.menuBarExtraStyle(.window)
}
}Content View:
To make the user interface, I will make a simple Content View with a TextField, a Submit Button and a Text View for the Response.
struct ContentView: View {
// I will use the EnvironmentObject property wrapper to share data between this view and others
@EnvironmentObject var appModel: DataInterface
var body: some View {
VStack {
// TextField for the user input .
TextField("Prompt", text: $appModel.prompt)
.textFieldStyle(.roundedBorder)
.onSubmit(appModel.sendPrompt) // Send the prompt to Ollama and get a response
// Divider draws a line separating elements
Divider()
// Use an if statement to conditionally display a view depending on if appModel.isSending.
if appModel.isSending{
ProgressView() // Display a progress bar while waiting for a response.
.padding()
}else{
Text(appModel.response) // Display the response text from appModel if not currently sending.
}
HStack{
// Button to send the current prompt. It triggers the sendPrompt function when clicked.
Button("Send"){
appModel.sendPrompt()
}
.keyboardShortcut(.return) // Assign the return key as a shortcut to activate this button. Cause Mac.
// Button to clear the current prompt and response.
Button("Clear"){
appModel.prompt = "" // Clear the prompt string.
appModel.response = "" // Clear the response string.
}
.keyboardShortcut("c") // Assign the 'c' key as a shortcut to activate this button. So Command + C
}
}
.padding()
}
}Links:
Final Project: https://github.com/carlosmbe/MyMacLLAMA
Ollama: https://ollama.com
Tutorial on Menu Bar Mac Apps: https://sarunw.com/posts/swiftui-menu-bar-app/
