Downloading Hardware: The Quiet Revolution in Networking
Networking is boring
Networking is boring. It is dominated by big players like Arista, Cisco, and Huawei. These vendors have been building equipment for decades, languishing in years of backwards compatibility and support for old features (and bugs). They compete with each other on the basis of size, customer lock-in (“nobody ever got fired for buying Cisco”), and suites of enterprise management solutions aimed at large corporations, governments and military networks. Boring.
But, there is a quiet revolution going on in networking which threatens to turn all of this boredom on its head. The revolution is powered by a new class of small and secretive telecommunications companies and by a relatively new type of flexible, programmable hardware called a Field Programmable Gate Array or FPGA.
Private networks: quick as a flash trade
In the popular book “Flash-Boys”, Michael Lewis discusses the curious story of “Spread Networks”. Spread Networks is a small, private company that built the shortest, straightest fiberoptic network between Chicago and New York. The construction cost a staggering $300 million!
Building long range fiberoptic networks used to be the preserve of governments and large telecommunications companies like AT&T. But this network was different, and it had a very specific aim: to shave 13 thousandths of a second off the time that it takes to transmit data between the Chicago and New York, which, not incidentally, happen to be two of the world’s largest financial centers.
The time to transmit data between two points is known as network latency. The aim of Spread Networks was to build the fastest or lowest latency network. Technologically advanced financial market traders use these fast networks move data between financial exchanges and perform ‘High Frequency Trading’ or HFT.
The need for speedy network hardware
Since the time that Spread Networks built their optical fibre routes, many other low latency network players have sprung up. Some, like Spread, are also telecommunications providers who have financial customers in mind, others are HFT firms themselves who’ve decided to build their networks in-house.
To make the most of these low latency networks, the networking equipment must also be low latency. The problem is that the large, established (boring) network vendors have no desire to invest in the creation of new specialised networking technologies for companies with a demand of (maybe) a few hundred devices.
Enter Exablaze: Exablaze is one of a handful of new networking companies that build boutique networking hardware for specialised low latency networks. The devices built by Exablaze are at the very extreme end of latency performance. To established network players, they wouldn’t even look like network devices at all. These devices operate at a scale of a handful of billionths of seconds, or ‘nanoseconds’. By comparison, the fastest devices from the established players operate at speeds of 10x-100x higher latency.
Building it big and slow
Established networking vendors like Cisco spend considerable time and cost designing complex, custom silicon chips to implement their networking functions. A typical custom chip takes 3–5 years to produce, requires teams of hundreds of staff, and costs millions of dollars in R&D and fabrication costs. Building chips like this is serious business.
Because of the extreme cost and risk of producing a new chip, the big networking vendors are cautious with their feature sets, and slow to innovate. The advantage of these chips is that, once perfected, they can be produced in their thousands for almost no cost. This is highly profitable for businesses that ship products in quantities of millions.
Building it small and fast
For Exablaze the demand for networking equipment is in the thousands, not millions of devices, and blazing speed (low latency) is the highest priority. In this market, cost is no object and extreme innovation and agility are the key differentiators between the players.
To overcome the rigid limitations of custom silicon chip design, Exablaze builds all of its products on a chip technology called Field Programmable Gate Arrays. FPGAs are a relatively new type of silicon chip which implements logical units that can be ‘rewired’ in hardware to implement different functions, all of which can be upgraded, modified and debugged even after they’ve been deployed to customers. To update an FPGA, you simply download an new circuit description into it and suddenly it behaves like a brand new chip.
Very recently, FPGAs have become large enough, and fast enough, to replicate many of the functions of the custom silicon chips used by the traditional networking vendors. This innovation brings a fast, iterative, and software-like approach to the stale and boring world of networking. And, it gives Exablaze an edge over its more traditional counterparts. Using the power of FPGAs, small companies like Exablaze have been able to use small, smart engineering teams to produce incredibly low latency devices for deployment into high performance networks (such as those used by specialist network providers like Spread).
The rate of innovation in this space is truly staggering. When exablaze announced its first switching product, it had a headline latency of 110 nanoseconds. Over the time it would take a traditional vendor to produce one new chip revision, Exablaze’s device has undergone four . At each revision, the latency of the device has improved and the feature set has expanded. The current iteration has a latency of just 49ns. Even more impressive, these upgrades have not forced customers to buy any new expensive hardware. Exablaze simply puts a new link on its website and customers can now download a new hardware description that rewires the FPGA inside of thier already installed device.
The flexibility to specialise
Using the same FPGA technology, Exablaze has also produced a complete network switch product that is comparable in features to those of the traditional vendors. This switch operates operates at a latency of just 95 nanoseconds, about 3x faster than traditional silicon chip based devices.
Exablaze also uses FPGA technologies to build network adapters (NICs). The headline adapter has a significantly lower latency than the nearest competitor who also uses traditional chip designs. Again, Exablaze has released several revisions to this device in the time it has taken competitors to produce just one revision. Again, these revisions have increased performance and feature sets without requiring customers to buy new hardware.
Using FPGAs has given the Exablaze engineers the flexibility to experiment with and build some very unusual network adapters. For example, one adapter can listen to timing signals from GPS satellites to precisely synchronise clocks inside and between datacenters. Exablaze has also used its FPGA technologies (and some of its own smarts) to produce a network capture device capable of capturing and timing network events that happen in intervals of just 0.25 nanoseconds (250 picoseconds). By comparison, traditional devices have 25x worse time resolution. Again this device was produced rapidly, with the entire product cycle taking only a fraction of the time of traditional approaches.
The flexibility is not limited just to Exablaze. Exablaze have also given customers access to the FPGAs inside their devices. This means that customers are now free to build their own business and application specific innovations inside of the devices that they own. Often the modifications are small, so small that a large networking company would never offer to build them. For example, “I really wish my network card could send two different packets at exactly the same time”. These small modifications are just the sorts of innovations that bring value to businesses.
Although networks may seem old and boring, behind the scenes the story is much more exciting. Powered by the wealth of sophisticated finance companies and using new silicon chip technologies, companies like Exablaze are at the forefront of a quiet revolution in networking, producing unique and unusual devices and out pacing and out innovating traditional vendors.