Ollama v0.1.27 AI Benchmark

Ollama first released native Windows Preview version in v0.1.25. However, there were some bugs to let the native version run 8X slower than WSL2 Linux version. It happened when there is only Intel CPU, no NVIDIA GPU.

With some fix starting in v0.1.27 which was released on 2024–02–23. The throughput of native version on Windows 11 is slightly better than WSL2 Linux version on the same Windows machine. Apple Mac M1 chip has a small boost with M1 GPU. It’s faster than Windows because of M1 chip.

The Throughput (tokens/sec) on 3 systems are listed below

Table 1. Ollama v0.1.27 AI benchmark

Tools that we used to test out this benchmark

[Ref] https://github.com/aidatatools/ollama-benchmark

Details of recorded benchmark video on 3 systems are listed below

Video 1 : Ollama v0.1.27 benchmark | WSL2 on Win11 (Ubuntu 22.04)

Video 2 : Ollama v0.1.27 AI benchmark | native Windows 11 application

Video 3 : Ollama v0.1.27 AI benchmark | Apple M1 Mac mini

Conclusion

Apple mac mini comes with M1 chip with GPU support, and the inference speed is better than Windows PC without NVIDIA GPU.