Running a Large Language Model is computationally expensive and requires a lot of memory, but through research and optimizations, these limitations are starting to fade away, to the point where we can run a LLM on a device that is 5 years old.
Size and quality
Microsoft’s Phi models are an example of these advancements, defined by them as “Tiny but mighty”, trying to find the balance between good output and memory requirements. Phi models come in different sizes, and I decided to try the latest Phi-3-mini with 3.8 billion parameters, which is the smallest available. To make it more interesting, I decided to try it on my iPhone 11.
The iPhone 11 is a five-year-old device, released in September 2019, with 4GB of RAM, which nowadays is not ideal for loading and running LLMs. Despite this limitation, and after a considerable amount of patience, I was still able to obtain an answer from the assistant.
Granted, it’s barely usable, and I only did it for fun, but this shows what current and next-generation smartphones will be capable of with increased memory and more computational power.
Most probably, we will have local LLMs running on Apple devices very soon, especially considering the rumors about Apple’s plans for its local LLMs on iOS 18 and the…
