Groq challenges Nvidia’s hegemony
Chatbots have become a familiar product for many people. However, waiting for 5–7 seconds to generate a response in chatGPT is not comical. It is extremely long for technological adoption. The matter is in the hardware used. Although Nvidia chips work wonders, but GPUs were not customized for generative AI tasks.
A new generation of LPU (language processing unit) chips from Groq, a startup founded in 2016 by Jonath Russ, the creator of Google’s TPU processors.
LPUs are custom chips (similar to ASICs in bitcoin mining), customized for the task of processing and generating text. Groq chips handle chatbot tasks on average 10 times faster than Nvidia! Processing over 300 tokens per second!
And that’s on 2022 (14nm) chips. What will be the gap when they launch 4nm in 2024?
The success is possible because of the LPU architecture — the RAM is right in the chip, which significantly reduces the time to signal transfer. Moreover, Groq has solved another problem as well. It does not depend on Taiwan’s TSMC. Groq chips are 100% designed and manufactured in the US.
Today, Groq’s business model is built on selling chips and providing their facilities in the cloud on inference as a service. They make money on each iteration of a chatbot.
If early investor Chamath (the king of SPACs) is to be believed, Groq’s developer audience grew in a month x3 to 10k at the public API launch. At the same time, the company plans to deploy 1 million chips by the end of 2025!
Groq is rumored to be unprofitable for now, but a $360M cushion of VC funding (last C round was in April 2021 at $1.1B) and scaling plans will help with margins
Plus, Groq acquired an AI business solutions startup Definitive Intelligence, targeting Aramco-level big business (already partners) and government agencies
The advantage of Groq’s chips is clear. LPU has the potential to revolutionize the AI market by providing more efficient and faster hardware. It seems the stuffing of data centers in cloud services (and beyond) could change very soon. The battle is on!
