The Cerebras Wafer Scale Engine stands out as an exceptionally impressive chip and remains one of the few AI chip solutions from a startup that has achieved commercial success. In march, the company announced a third iteration of the chip, known as the Cerebras WSE-3. This latest version utilizes further process miniaturization to increase density beyond that of its predecessors.
Cerebras WSE-3 chip has some incredible specs. It has 4 trillion transistors, 900,000 AI-optimized cores, 125 petaflops of peak AI performance, and 44GB of on-chip SRAM, all made using TSMC’s 5nm process.
The company has not publicly revealed the prices of its chips, but they are estimated to be in the range of $2–3 million from multiple sources.
In direct comparison to H100:
Cerebras says the WSE-3’s die is 46,225mm², which is 57 times larger than NVIDIA’s current H100 AI GPU, which is 826mm² .The WSE-3 also supports external memory options of 1.5TB, 12TB, or an amazing 1.2PB, which can train AI models with up to 24 trillion parameters.
Nvidia’s current H100 AI GPU has 16,896 cores and 528 Tensor Cores, but the new Cerebras WSE-3 has an incredible 900,000 AI-optimized cores per chip, a huge 52 times more than the H100. Truly amazing advancements.
It has 21 petabytes per second of memory bandwidth, which is 7000 times more than the H100, and 214 petabits per second of fabric bandwidth, making it 3715 times faster than the H100. The chip includes 44GB of on-chip memory, which is 880 times more than NVIDIA’s H100 AI GPU. These numbers show how far ahead Cerebras is compared to others.
Anshuman Tripathi from NSAB believes that wafer-based chips will soon surpass Nvidia’s non-wafer GPUs
TSMC aims to bring more powerful Wafer-Scale chips by 2027, expect a wave of Wafer-Scale computers
