What you should know about GPUs — Part 3, The Paradigm Shift of Compute

5 min readDec 26, 2023

Part 3 — The Paradigm Shift of Compute

Innovation Pushing to Change the Paradigm of Compute

Quantum Computing

To understand quantum computing you have to think back to chemistry classes when you were studying the locations of elections in atoms and looking at diagrams like this —

Superposition — To understand quantum computing we will analogize a transistor (which is either a 1 or 0 in traditional computing) to the yellow orbital in the top left. The location of an electron (referred to as a qubit) at the time of measurement is either a 1 or 0, based on what side of the x axis it is found. The concept of superposition in electrons means that they are probabilistically in both states at once until measured (Schrödinger’s cat style). Quantum algorithms can exploit this superposition to explore many possibilities simultaneously, leading to parallelism. What that means, idk, but I’m just going to leave that there …

Entanglement — Quantum computers can leverage entanglement, a phenomenon where the state of one qubit becomes linked with the state of another, even if they are physically separated. Changes to one qubit will instantaneously affect the other, no matter the distance between them [aka 0 latency information transfer].

One of the the challenges is quantum gates (the quantum version of logic gates aka AND, OR, NOT) are error prone because the control of electrons is not a perfected science.

State of Tech — You can use quantum computers if you would like to spend approximately $1–5k/hour. Long term, it is unlikely that quantum will scale to the accessibility and generalizability of a GPU in the next 20 years given the cost to manufacture. Quantum computers will compete on the supercomputer market, not on the ‘everyday inference’ or ‘everyday training’ markets. There is a question to be asked if they may be used for training of the largest of models and this is a possibility. To unlock this technology there will need to be a CUDA equivalent for quantum, not out of reach but definitely a pain.

Few startups will have the funds to compete in the quantum computer space. On top of the cost there are also challenging incumbent dynamics. Microsoft owns 49% of OpenAI and their own quantum computer. Neither of these has stopped the money flowing into the field.

Startups — A recent startup Extropic raised 17M to create an AI processor using what appears to be photonic technology.

Photonic Computing

Concept — Photonics is a technology that replaces electrons with photons in chips. Researchers are attempting to develop photonic versions of all sorts of components, photonic memory, photonic data transfer, photonic compute (as we are discussing here).

How does photonics work? Traditional compute uses transistors (0s & 1s). Photonics uses wave propagation to do calculations. The wave property means that photonics can be analogue or digital. Light waves have additive and subtractive properties when interacting with other light waves, 0s and 1s are represented using light or absence of light. Measuring the outputted light can allow you do calculations at the speed of light without the latency induced by transistor switching. Check out this video for an in-depth explanation.

When using electrons, the speed of data transfer is limited by how fast electrons move. Since the speed of light is more than 100 times faster, moving data using photons can significantly accelerate computing and data transfer.

We all know the saying- “you’re only as strong as your weakest link”, here it’s more like “you’re only as fast as your slowest component”. Photon-izing the GPU, runs the risk of being bottlenecked by other components that can’t move at the speed of light. The key here is to design processes that can run phonically with out needing access to other systems, e.g memory. To experience the true lift of this technology, we need photonic systems not just photonic components.

Startups — Lightmatter was founded in 2021 originally targeting photonic processors. The company eventually pivoted to work on interconnect, aka connections intra chip. If I were to hypothesize this shift, I’d imagine that photonic interconnect is a low hanging fruit problem that likely can be sold to many chips on the market, data throughput is a universal issue. It is still to be seen what comes of light matter’s photonic compute technology, but it seems to be on the back burner.

Analog photonics was working to make photonic sensors starting with lidar, combining both the memory advantages of analogue (avoiding intermediary binary conversions and capturing more information in each calculation), and the speed of photonics.

State of Tech — Photonic sensors and fiber optics have proven themselves to be viable and useful technologies. Photonic compute & memory companies need to prove they can create reliable functional products, then they need to create an economy of scale to make the costs viable. If they can execute on then above they have the potential to blow silicon GPUs out of the water with 100,000x better processing power. Certainly worth paying attention to.

Opinions

Innovations in traditional silicon are not going to move the needle

A paradigm shift is coming in compute and I think its going to be analogue, photonics, quantum or some combination of the above. The 3 big AI chip startups are poised to compete with NVIDIA over the next 5–10 years, but we are reaching the true limits of current silicon technology and the next big step is going to be fundamentally different, analogue, photonic, or quantum. I lean towards the first two.

I hope we see some exits soon

Significant amounts of deep tech venture money is locked up in the big 3 AI chip startups. These investments were made in around 2016 or earlier, this means some of this money is 7 years, deep. The real game changers in this field are going to come from the more speculative research driven spaces. For the sake of the advancement of this space, I hope we see some good outcomes and money being re-allocated into the next generation.

If you are interested in or working on AI chips, Photonics, quantum or the like, please reach out.