How Apple defines on-device AI and future trends — from the perspective of analyzing supported models of Apple Intelligence / Apple如何定義裝置端AI與未來趨勢變化 — 從分析Apple Intelligence支援機種之觀點

The iPhone 15 with the A16 chip cannot support Apple Intelligence, but M1-equipped models can. Therefore, it can be concluded that the key to supporting the current Apple Intelligence on-device model is the DRAM spec, rather than computing power (TOPS).

The AI computing power of the M1 is about 11 TOPS, lower than the 17 TOPS of the A16. The A16 has 6GB of DRAM, lower than the 8GB of the M1. Therefore, current Apple Intelligence on-device AI LLM require about 2GB or less of DRAM.

The demand for DRAM can be verified in another way. Apple Intelligence uses an on-device 3B LLM (which should be FP16, as the M1’s NPU/ANE supports FP16 well). After compression (using a mixed 2-bit and 4-bit configuration), approximately 0.7-1.5GB of DRAM needs to be reserved at any time to run the Apple Intelligence on-device LLM.

From the above analysis, the following further thoughts and conclusions can be drawn:

1. To enhance existing on-device applications with AI, at least a on-device 3B LLM must be deployed, and based on that, the DRAM spec is determined by the compression method.
2. Microsoft believes the key specification for an AI PC is 40 TOPS of computing power. However, for Apple, integrated with cloud AI (Private Cloud Compute), 11 TOPS of on-device computing power is sufficient to start providing on-device AI applications.
3. Consumers may find purchasing Microsoft’s AI PC confusing (calculating whether it reaches 40 TOPS before purchase), whereas Apple directly tells consumers which models can support Apple Intelligence. Regardless of whether on-device AI applications meet consumer needs, Apple has a clear selling advantage from the start.
4. In the future, Apple Intelligence’s on-device AI will certainly upgrade (most likely to a 7B LLM), which will require larger DRAM to operate. It is worth watching whether Apple will use this strategy to differentiate between high- and low-end models.
5. Whether the user experience is as good as Apple claims still needs to be observed (Google’s Gemini, for example, ever made exaggerated claims in its promotions).
6. Samsung S24’s AI capabilities are limited, and Microsoft’s AI PC still confuses consumers. Apple has successfully defined on-device AI (at least consumers are already aware of the rich AI features and selling points of Apple’s AI devices), which will accelerate competitors’ imitation and catch-up, thereby driving rapid growth in the on-device AI-related industries.

從配備 A16 的 iPhone 15 無法支援 Apple Intelligence，但 M1 的機型可以支援，這推論出能否支援目前 Apple Intelligence 裝置端模型的關鍵應該是 DRAM 大小，而較不是AI算力 (TOPS)。

M1 的 AI 算力約為 11 TOPS，低於 A16 的 17 TOPS。但 A16 的 DRAM 為 6GB，低於 M1 的 8GB。因此，目前的 Apple Intelligence 裝置端AI LLM對 DRAM 的需求約為 2GB或更低。

對 DRAM 的需求可以用另一種方式來驗證。Apple Intelligence 採用裝置端 3B LLM（應為 FP16，M1 的 NPU/ANE 對 FP16 有很好的支持），經過壓縮後（採用 2-bit 與 4-bit 的混合配置），隨時需要預留約 0.7-1.5GB DRAM 來運作 Apple Intelligence 的裝置端 LLM。

從上述分析可以延伸出以下進一步的想法與結論：

1. 若要透過 AI 強化既有裝置端應用，至少需部署裝置端 3B LLM，並以此為基礎再根據壓縮方式決定了DRAM規格。
2. 微軟認為 AI PC 的關鍵規格是 40 TOPS 算力，但對 Apple 而言，搭配雲端 AI（Private Cloud Compute），裝置端有 11 TOPS 的算力已足夠開始提供裝置端 AI 應用。
3. 消費者若欲購買微軟的 AI PC 可能會感到困惑（還要自行計算是否達到 40 TOPS），而 Apple 則是直接告訴消費者哪些機型可以支援 Apple Intelligence。不論裝置端 AI 應用能否滿足消費者需求，Apple 在銷售上一開始就具有明顯優勢。
4. 未來 Apple Intelligence 的裝置端 AI 肯定也會升級（最有可能升級到 7B LLM），屆時需要更大 DRAM 才能運作。Apple 是否會以此作為高低階機種的產品區隔策略值得觀察。
5. 使用者體驗是否如 Apple 宣稱的那麼美好仍需觀察（Gemini 在宣傳上就曾犯過言過其實的錯誤）。
6. Samsung S24 的 AI 功能有限，微軟的 AI PC 目前仍讓消費者感到困惑，但 Apple 成功定義了裝置端 AI (至少消費者已清楚知道Apple的AI裝置功能豐富且有哪些賣點)，而這會加速競爭對手的模仿與追趕，並進而帶動裝置端 AI 相關產業的快速成長。