PinnedLocal DeepSeek-R1 671B on $800 configurationsUpdate on April 6,2025: Llama 4 has been released, and along with DeepSeek V3/R1, it marks a step into the top-tier large model platform…Mar 20A response icon17Mar 20A response icon17
PinnedIntel Core Ultra 5 vs. Apple M1 on LLM inferenceI bought an Intel “AI PC” equipped with a Core Ultra 5 125H and 96GB of memory, which means a graphics card with 48GB of VRAM. This mini PC…May 24, 2024A response icon6May 24, 2024A response icon6
Best LLM for 🇫🇮Finnish (low-resource languages) Translation?In the previous experiment, we concluded that google/madlad400–3b-mt is the best choice for running machine translation locally. The newly…May 14May 14
Audio processing with Qwen2-Audio on $600 Intel Core Ultra 5 125H Mini-PCThe inference speed in the previous experiment on video understanding on a Core Ultra iGPU was not ideal, struggling to surpass human…Apr 28Apr 28
Video Understanding with Qwen2.5-VL on $400 (Tesla P40, RTX 2080Ti) GPUsIn the previous experiment, Qwen2.5-Omni’s performance in video understanding was less than satisfactory, and the processing speed of…Apr 23Apr 23
Video Understanding with Qwen2.5-Omni on $600 Intel Core Ultra 125H Mini-PCI want to build a home robot that can record audio and video around the clock — for example, capturing what we do at our desks and…Apr 10Apr 10
$60 Multi-GPU Power SolutionIn our Local DeepSeek-R1 671B on $800 configurations, we mentioned PSUs capable of powering multiple GPUs. An alternative, cost-effective…Apr 5Apr 5
DeepSeek-V3–0324 for Code Generation Tasks, How many bits of Quantization is enough?In my opinion, the so-called reasoning models like OpenAI’s o1, o3, and DeepSeek R1 don’t really perform logical reasoning in the…Mar 27A response icon4Mar 27A response icon4
Convert PDF to text (markdown) with SmolDoclingSmolDocling is a vision-to-text model with only 256M parameters, so its inference resource requirements should be much lower than olmOCR…Mar 25A response icon3Mar 25A response icon3
Convert PDF to text (markdown) with olmOCR on Windows Mini PC with Intel Core Ultra i5olmOCR is a Qwen2-VL 7B model fine-tuned with academic papers, technical documentation, and other reference content, as well as a toolkit…Mar 4A response icon2Mar 4A response icon2
