Benchmarking the Raspberry Pi 3 B+
The launch of the shiny new Raspberry Pi 3 B+ offers a chance to revisit the entire history of the Pi family, benchmarking each device in turn from the original Raspberry Pi Model B launch board with its somewhat limited 256MB of RAM right through to the shiniest and newest board. This post collates the results from a range of different benchmarks, demonstrating how the power of the Pi has changed over the years.
If attempting to replicate the results yourself, there is one key fact to note: the Raspberry Pi has enjoyed somewhere in the range of a 30 percent performance uplift in the last couple of years through software and firmware optimisation alone; comparing the same benchmark run on a Pi using the latest Raspbian operating system today with results gathered a year or more ago will give a false reading, which is why all these results have been gathered using the same firmware and software revision.
Update: The benchmarks in this article have been updated after it was discovered that the Pi Zero on test had developed a fault. The Pi Zero benchmarks have been re-run on a new board and updated, while the Pi Zero W’s SysBench score has also been updated to reflect improvements brought about by the latest firmware release. The figures in this article are now accurate and up-to-date as per the version of Raspbian released on the 14th of March 2018.
SoC: Broadcom BCM2837B0 quad-core A53 (ARMv8) 64-bit @ 1.4GHz
GPU: Broadcom Videocore-IV
RAM: 1GB LPDDR2 SDRAM
Networking: Gigabit Ethernet, 2.4GHz and 5GHz 802.11b/g/n/ac Wi-Fi
Bluetooth: Bluetooth 4.2, Bluetooth Low Energy (BLE)
GPIO: 40-pin GPIO header, populated
Ports: HDMI, 3.5mm analogue audio-video jack, 4x USB 2.0, Ethernet, Camera Serial Interface (CSI), Display Serial Interface (DSI)
Dimensions: 82mm x 56mm x 19.5mm, 50g
The overall specifications are largely unchanged from the Pi 3 bar the inclusion of gigabit Ethernet connectivity with optional Power over Ethernet (PoE) support when fitted with the appropriate add-on HAT, a tweaked BCM2837 design with new metal-encapsulated package, and the addition of 5GHz Wi-Fi connectivity along with a shift from a chip antenna to a ground-plane antenna. One area of note is the weight: a move to a thicker circuit board, coupled with the additional metal on the board, has increased the weight of the Pi 3 by 10g to 50g.
Thermal Management Benchmark
The original Pi 3 design suffers from a high system-on-chip (SoC) temperature, particularly when running CPU-intensive applications — a by-product of the distance between the CPU cores and the temperature sensor on the chip’s die. These thermal images, taken after a ten-minute CPU benchmark run, show the difference the Pi 3 B+’s new SoC packaging and improved PCB design make: the heat is more evenly spread throughout the whole SoC package — which means that the on-board temperature sensors will be considerably more accurate — while the peak temperature is lower. The new package also enjoys better bonding to the circuit board, which in turn is able to handle the heat better: note how the heat is spread almost entirely through the board, compared to the one-sided heating of the original Pi 3 design.
The classic Whetstone benchmark measures a processor’s floating point performance, in Whetstone instructions per second (WIPS). Here the performance is given in millions of WIPS, or MWIPS. There’s little here to surprise: architecture improvements in the Pi 2 onwards give the best boost, while the Pi 3 B+’s extra 200MHz clock speed gives it the edge over the otherwise very similar Pi 3.
Where Whetstone offers floating-point performance measurement, Dhrystone looks at integer performance. The measurement is given here in millions of instructions per second, or MIPS. Again, no surprises: the Pi 3 B+ comfortably takes the lead.
SysBench CPU Benchmark
The SysBench application offers a range of synthetic benchmarks, and its CPU test demonstrates the difference multi-threading can make. Here two measurements are given: the number of seconds it takes to complete the benchmark in single-threaded and multi-threaded modes, with the latter only available to the quad-core Pi 2, Pi 3, and new Pi 3 B+ models. This offers a glimpse at how parallelism can mean more than raw performance: despite being an older SoC design and running at a lower speed, using all four cores of the Pi 2 is significantly faster than using a single core of the Pi 3 B+ — though using all four cores of the Pi 3 B+ is faster still, of course.
Designed for use in high-performance (HPC) computing environments, Linpack is a more generalised test than either Dhrystone or Whetstone. Here three readings are taken: double-precision complexity, single-precision complexity, and a special version of the single-precision benchmark compiled to use the NEON accelerated instruction set available on the Pi 2, Pi 3, and Pi 3 B+. All measurements are in millions of floating-point operations per second (MFLOPS). As before, this benchmark highlights how making clever use of the features available in a given SoC is key to getting the best performance: switching your program to use NEON on a Pi 3 will give you better results than simply taking the same program and running it on a Pi 3 B+ without NEON.
Python GPIO Benchmark
Many Raspberry Pis spend much of their life running Python applications, so this custom benchmark measures each model’s capability at exactly that workload. A simple program switches a pin on the general-purpose input/output (GPIO) port on and off as quickly as possible, while an external frequency counter measures its switching rate. The result is express in kilohertz (kHz), with 1kHz equalling a thousand switches per second. It’s a fairly synthetic test — it’s unlikely you’d actually want to simply turn a pin on and off repeatedly without doing anything useful inbetween — but does offer a hint of how shifting to each model up the rankings will affect CPU-bound GPIO applications.
SysBench RAM Benchmark
The CPU isn’t the be-all and end-all of performance, though, and here the SysBench utility measures the memory throughput for reading and writing data. Figures are given in megabytes per second (MB/s). This test, possibly more so than any other, shows the dramatic difference the launch of the Pi 2 and its successors made.
The Ethernet port on the Raspberry Pi B family remained largely unchanged across all models, until the launch of the gigabit-capable Pi 3 B+. This benchmark pushes incompressible data through the Ethernet port from a nearby networked machine and measure the throughput in megabits per second (Mb/s); models without an Ethernet port show no result. The Pi 3 B+ is the first model to feature anything better than 10/100Mb/s Ethernet connectivity, and while it falls well short of the 1,000Mb/s theoretical limit of gigabit connectivity — the new network chip still shares a single USB channel to the SoC, as in previous designs — it’s a comfortable boost over earlier models.
Power Draw Benchmark
Performance comes at a cost, and the power draw benchmark shows exactly what that cost is: energy usage. Each model is configured with an HDMI-connected display, USB wireless keyboard dongle, and Ethernet where available, then two measurements are taken: one where the Pi is sat idle at the Raspbian desktop, and another while running a CPU-intensive application. Unsurprisingly, the Pi 3 B+ draws the most power here, meaning that if you’re power-constrained you’ll almost certainly be better off with something like a Pi Zero W.
This benchmark is a head-to-head between only two models, the Raspberry Pi 3 and the Raspberry Pi 3 B+, to demonstrate the improvement available from the new radio module and ground-plane antenna design. The iwlist utility is used to scan for nearby wireless access points and rate their signal quality out of a possible 70 points, which is then charted. Cells 9 through 13 inclusive are only visible on the Pi 3 B+, thanks to its new dual-band 2.4GHz and 5GHz radio; all other cells show improved reception thanks to the new ground-plane antenna, based on the one developed for the Pi Zero W, with the sole exception of Cell 8.
Thermal Throttling Benchmark
When a Raspberry Pi’s system-on-chip — or, more accurately, the thermal sensor on the SoC — reaches a pre-configured maximum temperature, the Pi automatically reduces its clock speed to prevent thermal damage. This is known as “thermal throttling,” and is an area in which the Pi 3’s hot-running SoC has struggled in the past.
This graph compares a Pi 3 to a Pi 3 B+ in two regards: the clock speed at which the processor is running, graphed along the left-hand axis and measured in megahertz (MHz), and the temperature reported by the thermal sensor, graphed along the right-hand axis and measured in degrees Celsius. Both Pi boards are booted, allowed to settle, then given a CPU-intensive task to run for eight minutes with measurements taken every second.
The dashed lines represent the clock speed, and initially both go up to their maximum to accommodate the task’s demand: 1.2GHz for the Pi 3 and 1.4GHz for the Pi 3 B+. The solid lines, meanwhile, chart the temperature rise of the SoC. Both the Pi 3 and Pi 3 B+ hit begin throttling reasonably early on in the process, but the Pi 3 B+ does so more stably: aside from a brief period when it bounces between 1.2GHz and 1.4GHz, the Pi 3 B+ sits at 1.2GHz to manage its temperature while the Pi 3 is both running considerably hotter and constantly varying its processor clock — as low, towards the end of the process, as 926MHz.
For those who have found the Pi 3 a trifle too hot to handle, the Pi 3 B+ is most definitely the answer: it never gets as hot as the Pi 3 did, even without additional cooling, and throttles to the same speed as the Pi 3 runs at normally — an impressive achievement, and real proof that the clever new SoC packaging and other tweaks are delivering on their promises.
The Pi 3 B+ is a worthy upgrade over the original design. While not as big a jump as the Pi 2, it offers improved performance — in the case of Ethernet throughput, considerably improved — for the same price as its predecessor. While that performance does come at the cost of increased power draw, the new thermal management techniques introduced in the design of the Pi 3 B+ mean that it runs cooler than its hot-headed predecessor — and that alone is worth the cost of entry.