Compute and reduce emissions!
How the startup Groq can solve our accelerating computational needs while reducing our carbon footprint drastically.
Data is the new oil
Data is definitely the new oil. It’s the key ingredient for training machines (e.g. computers) to infer valuable results, sometimes life-saving results. This method is known as machine learning. Whether it’s predicting the right hydration of plants, assisting drivers in staying in their lanes, or enabling gene therapy for cancer, machine learning is having a significant impact on our daily lives already. This trend will only accelerate in the coming years.
The demand for petabytes and petabytes of data that are required for machine learning already puts an enormous demand on the computation power of our modern data centers. Data centers already use an estimated 200 terawatt-hours (TWh) each year, or about 1% of global electricity demand, and contribute 0.3% to overall carbon emissions. The carbon footprint for data centers could soon reach a double-digit percentage of the world’s electricity by 2025.
The only solution to this problem is to simplify compute, everywhere.
We need a simpler and more scalable processing architecture that can accelerate the performance of compute-intensive workloads while reducing our carbon footprint.
The chipset maker Groq offers a solution to this problem. Its approach is quite revolutionary and unique.
Starting from scratch
Instead of adding more cores to its central processing units, like Intel, Groq is taking an opposite approach. The industry standard is putting many smaller processors working together on one piece of silicon that can carve up a problem and solve it quicker than one more performant chip. What many don’t know is that Moore’s law is actually slowing because the smaller the transistors get, the more heat the chip produces. That’s why Intel has introduced the dual-core processor and many multiple n-core-infrastructures.
Groq is simply starting from scratch.
Three advantages of simplifaction
Instead of improving existing designs that are already reaching their limits, Groq has three key areas of innovation:
- Software-defined hardware: Groq started with developing the software and then created a programmable chip that’s relatively simple but fast. In their new chips, the control of execution and data flows is moved from the hardware to an accompanying compiler (software that turns computer programs into instructions that the chip can execute). This is freeing up valuable silicon space for additional processing capabilities.
- Silicon innovation: Groq’s simplified architecture removes any unnecessary circuitry from the chip. It leads to a more efficient silicon design with more performance per square millimeter.
- Maximizing developer velocity: The simplicity allows developers to focus on the compiler, enabling software requirements to drive the hardware specification and not the other way around.
This architecture has lead to the development of one of the fastest chips on the market:
Groq’s TSP100 chip allows for one quadrillion operations per second!!!
Focus on inference
It is essential to understand that machine learning models require two things:
- Training: This enables us to develop the required AI model.
- Inference: This enables us to get the needed results from the model.
Many would assume that the company would focus more on training AI models, but Groq’s CEO Jonathan Ross makes a case for inference:
“Inference is an inherently larger market. Training scales with the number of machine learning researchers you have, inference scales with the number of queries or users. Training is compiling, inference is running” — Jonathan Ross, CEO of Groq
Therefore, Groq will manly target operators of giant data centers and car manufacturers. Datacenter operators can run with Groq their operations more efficiently and save electricity, while self-driving cars will have reliable computational capabilities and save on energy consumption at the same time. Its chip will be mostly used where latency and computational power is critical.
Funding from TDK
Due to the high inference performance and the potential reduction in IT infrastructure cost, TDK Ventures, the venture arm of TDK, decided to invest in Groq. This investment was announced on August 5th and also perfectly aligns with the mission of TDK Ventures “to invest in innovative startups that focus on digital and energy transformation and contribute to sustainability”.
Concerning the investment, Nicolas Sauvage, Managing Director, TDK Ventures, stated the following:
“In line with TDK’s environmental transformation mission, it is critical we support and invest in solutions that drastically reduce energy needed for very high levels of compute power. Groq brings the highest AI compute for a given amount of energy, typically measured as Tera Operations Per Second (TOPS) per watt, demonstrated by its first product, the TSP100, with further projected improvement in their next gen platform.”
Stealth-mode origin
This funding is an important milestone for this company. The company was founded in stealth-mode by several prominent Googlers who have worked on the TPU (Tensor Processing Unit) in 2017 — most prominently, Jonathan Ross, who helped invent the TPU.
Since then, the company has raised two funding rounds that were both led by the investor Chamath Palihapitiya. The first round was around $10 million in 2017 and the second $52 million in 2018. It might seem a contrarian bet to invest in a startup that is developing a modern chip because it is very capital-intensive and there is potential competition from larger organizations e.g. Intel, Nvidia, and Amazon. But Groq could provide in Chamath Palihapitiya’s words the “fundamental building block for the next generation of computing”.
The funding by TDK proves the increased interest in powerful and sustainable computing. Groq is tackling both problems and has proven its technological viability. It will be gratifying to follow Groq’s further path in simplifying computing, everywhere.
