The Rise of AI Accessibility
By Gary Zhu
It often feels hard to keep up with the field of artificial intelligence. In the past year alone, we’ve seen generative AI win awards at major artistic competitions, pushing the boundaries of digital creativity. We’ve seen the creation of large language models so sophisticated that it makes us question what it means to be human. We’ve seen the advent of artificial intelligence so good at coding that it can beat programmers at their own game. Now, with companies like Google, Amazon, and Microsoft continuing to invest billions into AI research, it seems that the AI industry is poised to usher in a new golden age of technology.
Except, there’s just one problem. Behind every large-scale AI model lies an enormous amount of computing power, limiting their creation to the select few that can afford to build the required infrastructure.
Artificial Intelligence — The Resource Problem
It’s clear that the development of AI is an extremely expensive process, requiring exponentially more resources year by year. Open AI has reported that the amount of energy used to train AI has been doubling every 3.4 months, a trend that seems unlikely to stop any time soon. The development of AI models has also greatly contributed to the existing chip scarcity, creating a lack of the fundamental circuits that power the majority of electronics used today.
So why is AI such a resource-intensive process? Most complex models are composed of “neurons”, individual logical units that receive inputs and create outputs which are fed into other neurons for further processing. As a result, the number of interconnections within a model grows rapidly with more neurons, often reaching billions in number. This forces scientists to use multiple chips in parallel to keep up with the computational demand. Without this process, a model similar to GPT-3 would take 355+ years to be trained on a single GPU cloud.
Furthermore, parallelism becomes less and less efficient as more cores are involved. With every additional core, more time must be dedicated to communicating between them, creating a bottleneck in computational efficiency and sustainability. Eventually, Open AI estimates that ChatGPT will require over 30,000 Nvidia GPUs to function at peak capacity. For the average aficionado, such a price is completely out of reach. All this means that the AI revolution will likely continue to exacerbate current societal issues, and with it, our hopes for a sustainable and economical future.
IBM NorthPole — A Solution to AI Sustainability?
Recently, IBM has announced the creation of NorthPole, a chip specifically built to enable artificial intelligence algorithms. NorthPole has outstripped every other chip on the market, outperforming even Nvidia’s flagship H100 GPU while being 5 times more energy efficient.
However, what makes NorthPole unique is not merely its energy efficiency. Rather, IBM’s NorthPole represents a fundamental shift in the chip-making process, succeeding the chip TrueNorth and being the newest in their line of brain-inspired processors.
Traditional computers are built according to the von Neumann architecture, where processors and memory are separate entities. To transfer data between these two components, a system of circuits known as a bus is used. Unfortunately, as data sets become larger and larger, a bottleneck forms when transfer rates fail to keep up. This phenomenon is known as the von Neumann bottleneck, where processors spend more and more time idling, waiting to receive information from central memory.
To overcome the Von Neumann bottleneck, IBM has integrated memory within the chip itself, eliminating the need for off-chip memory. Each core mimics a human neuron, capable of performing computations and storing information simultaneously. This style of chip architecture maximizes the efficiency of parallel computing by lowering throughput (the amount of data being transferred), resulting in improved energy efficiency and computational speed. As a result, NorthPole is especially useful in the field of artificial intelligence, where the ability to execute millions of smaller tasks is more valuable than one or two larger tasks.
The Future of AI Development
Although NorthPole has not yet entered mass production, it has already opened new doors for the future of artificial intelligence. For one, it will likely decrease the cost of training larger AI, making it more accessible to the general public and promoting open-source development.
Experts also believe that NorthPole’s efficiency allows it to forego the need for bulky cooling systems, allowing artificial intelligence to be integrated into everyday appliances. As a result, these chips may find use among applications where space and convenience are major concerns — such as autonomous vehicles and robotics.
Finally, given that artificial intelligence is projected to consume some 85.4 terawatt-hours of electricity annually, NorthPole architectures will play a key role in reducing the carbon footprint of the digital world. NorthPole is also likely to reduce the chip industries’ reliance on rare metals, aiding in environmental conservation.
As revolutionary as NorthPole is, Modha, the lead researcher behind the NorthPole project, believes that there is plenty of room for further improvement in semiconductor technology. He hopes that their success will inspire future innovations, showing that chip architecture is just as important as computational power. While NorthPole may just be a small step forward in improving artificial intelligence, it makes clear that novel innovations are key to pushing the boundaries of what’s possible.
