What is PCIe and why should you care?
PCIe, which is an acronym for Peripheral Component Interconnect Express is a chip interconnect technology.
Below is a picture of a PCB (printed circuit board) that mechanically supports and provides a base for electrically connecting different electronic components. You can think of it as the muscular and skeletal system behind all electronic devices.
The orange arrows point to the PCIe interconnect on this board which is serving to shuttle data between components.
We could describe PCIe as a protocol. But first let us understand what a ‘protocol’ actually is; Think of it as a well-defined set of rules and procedures (language semantics) that allows 2 or more different hardware components in a device to talk to each other.
Let’s look at a scenario where you may have wanted to play a high-resolution PC game like the Assassins’ Creed or Metro Exodus that typically requires more graphics computational power than what a regular PC ships with. So you went ahead and bought an NVIDIA GeForce graphics accelerator to plug into your PC.
Now how do we ensure seamless integration between the PC and your newly bought hardware accelerator, where both devices can talk to each other at a data rate of 128 Gbps (Gigabits per sec — viz is 128 x 1,000,000,000 bits per sec (bps)) ?!
This is where PCI Express saves the day!
You would insert the NVIDIA graphics accelerator into one of the PCIe slots on the motherboard of your PC which would look like the picture below:
[For those who are interested in how we got 128Gbps: A typical graphics processor available today in the market has a Gen 3 x16 interface, which means it has 16 lanes, each operating at 8Gbps. So 16*8Gbps = 128Gbps.]
Below you see the specs for an NVIDIA video card which highlights the PCIe Gen3, 16 lane interface.
PCIe also lets the central processing unit (CPU) in a device talk to several other components within the same device(called ‘on-board’ vs the ‘add-in-card’ example of graphics accelerator). This can be the processor accessing the DRAM (Dynamic Random Access Memory) or an on-board NIC (Network Interface Controller). The latter by the way lets you connect to the world wide web(www) and do all of the wonderful things you are able to do through the internet.
Furthermore, a majority of the HDDs (Hard disk drive) and SDDs (Solid State Drive) used for data storage in data centers and in all of the PCs move data into and out of them through the PCIe.
So you can think of PCIe as a nervous system connecting the brain(processor) to all the vital organs (onboard and add-in components capable of specialized actions) and additionally as a self-learning, interfacing organ able to connect with and exchange information with new entities.
As you read this, the ASCII characters that make up the words you are reading on the display of your screen have journeyed through several devices and the PCIe has been the primary, on-device, bus which has carried it. All of these bitstreams traveling within a few milliseconds.(0.001 second)
With the explosion of Machine learning (ML), Deep Learning (DL) and Artificial Intelligence (AI) applications comes a guzzling need for specialized processors that can train, model, and predict. They also need large memories to store ‘big-data’ that they process and call for them to be available within ~50 ns (0.0000000001) latencies.
A scalable fabric that is already ubiquitous, has a well-integrated hardware development process, a widely available software driver ecosystem and engineers already trained on the complexities — holds within itself the answers to allow disaggregated, fungible and specialized hardware components to coordinate and solve the problems of the next generation of AI computing.
PCIe is ubiquitous in computing, communication, and storage devices used in the cloud, enterprise, PC, embedded, IoT, automotive, and mobile market segments for the last two decades.
You may wonder who owns PCIe technology. If they did, they might have a revenue pipeline of royalties. PCIe® technology is an open non-proprietary technology that is co-developed by several companies collaboratively (even competitors ! ). Today, any company could get the specifications(blueprint) for the technology and use it to build a device which uses PCIe on their board. However in reality, since this is a mature technology, there are vendors who have developed PCIe IPs (Intellectual property) and mass-market it to achieve economies of scale. A few examples of such vendors are Cadence, PLDA, and Synopsys.
Semiconductor chip makers like Intel, Apple, Broadcom, Qualcomm, and Marvell have a choice of buying PCIe IP off the shelf for a price tag and then integrating into their own devices. Most do, as they do not want to reinvent the wheel and can instead focus on their core competencies.
Now that we know what PCIe is, let’s get a glimpse of its backstory. PCIe is an evolution of older hardware interconnect technologies of PCI, PCI-X, and AGP, which were the name of the game, prior to 2003. In 2003, the 4 heavyweights of personal computing (PC) and server industry — Intel, Dell, HP, and IBM teamed up to define and create PCIe.They would then go on to donate this to an open standards body, the PCI-SIG.
Today’s PCI-SIG i.e PCI Express Special Interests Group is a consortium that develops the specification, owns, and manages the open industry standard. It offers compliance workshops to test and ensure interoperability between products from different vendors and also holds developers’ conferences in the US, Israel, APAC, and Europe.
The standards body has currently about 800 member companies, who have adopted the PCIe interface on their hardware design. Open, technical contributions to the non-proprietary protocol from some of these companies have consistently been pushing the envelope of innovation. Some companies hold greater sway in shaping the standard and bonus points for guessing which ones :).
The first generation of PCIe, which supported a data rate of 2.5G bits/sec per lane (x1) was introduced back in 2003. Today after 17 years the protocol has consistently been breaking new frontiers, with the Gen 6 specification defining a data rate of 64G bits/sec per lane (x1). Gen 6 is planned for release in 2021. Here is how the evolution looks on a time chart.
The table above stands as a testament to the success of the technology, in that it has doubled its bandwidth 6 times in the last 17 years, adopted universally, and is serving as the bed-rock for future innovations for chip technology.
If you are interested to know more about the engineering technology behind PCIe, here are some resources:
- A great ebook -PCIe Technology from Mindshare https://www.mindshare.com/Books/Titles/PCI_Express_Technology_3.0
- The PCI-SIG.This would be where you’d find 1300 page specification which lays out the rules/procedures to manage the complexity on manufacturing a device that can speak “PCIe”.
