Rendering Video With Flash Drive RAID Volume— Is The Performance Better?
RAID provides a storage solution that can offer redundancy or high performance by using multiple disk drives (minimum of two). A volume can be created from them using macOS RAID. This is a software based implementation that is a part of the operating system’s features. Various types of external storage can be used for RAID, even USB flash (i.e. thumb drives) actually work. This is not a robust solution by any means for production purposes, but it does offer something for editors on the go. It can be a sort of makeshift solution for optimizing performance.
In this case I have two USB thumb drives of different storage capacity. On production RAID systems used for enterprise applications, it is not recommended to use drives with different storage capacities because the RAID system will only be able to create a volume from the total size of the lesser total drive capacity. For example I have one thumb drive that has 8 GB, while the other has only 2 GB. I can therefore only have a maximum RAID size volume of 2 GB. The two drives are connected to a USB 3.1 hub (Gen 1, 5 Gbps, 900 mA).
Normally I would not be using thumb drives for this particular configuration, but it is good to try it out. In production environments, RAID are more hardware based devices that attach to the network in the case of a NAS. It can also be directly connected to the computer via the USB port consisting of an enclosure unit with many disk drives. Those devices are usually much faster and does optimize performance for tasks that require fast write speeds. In this case USB thumb drives are more like a last resort if anything.
The macOS provides a RAID feature that can use Level 0. This makes use of a stripe set, which uniformly writes data across the number of drives you have configured for the volume. Thus, if there are two drives on the RAID stripe set, the data is written across both drives as if it were a single volume. The advantage of this is to improve performance by optimizing write speeds. What happens is that the RAID system builds a single volume that multiplexes two separate USB connections into one.
Objective
Sometimes it is not necessary to purchase a fast SSD drive if the budget doesn’t permit or if it will only be temporary. For example, you need to render video quickly while on the road but you are running low on space. You also don’t have time or any place to buy an external SSD drive. Instead, RAID 0 can can offer a solution using existing drives, like USB thumb drives. I want to build a RAID volume using the USB thumb drives to see if it will improve performance significantly. This is not a complete benchmark or performance report, so it is informal. This is merely to evaluate my own personal experience to test the workload and record my observations.
Testing Methodology
I am going to render a file using Da Vinci Resolve. I will first try Full HD 1080p and then ProRes 422 to see if there was any significant performance increase using RAID 0. The test will use the media files for rendering the video stored on an external USB 3.1 harddrive. The MacBook Pro used in this testing is a mid-2015 model with the macOS 10.13.6 High Sierra.
The RAID volume was created using two USB thumb drives. One drive has a storage capacity of 8 GB while the other has 2 GB. Thus, the effective storage for the stripe set is 2 GB. Both drives are actually USB 2.0 but connected to a USB 3.1 hub.
I also used Blackmagic Disk Speed test to get the disk write/read speed on the volume. So, I constructed this table to show the results from three volume sources including the macOS RAID.
Internal Macbook Pro SSD: 1806 MB/sec READ/ 600 MB/sec WRITE
External non-SSD HDD (USB 3.1): 114 MB/sec READ/ 112.7 MB/sec WRITE
macOS RAID 0: 28.8 MB/sec READ/ 3.8 MB/sec WRITE
Note: MB/sec = Megabytes per second
The test will do two rounds of rendering using two video formats. First round is MP4 (*.mp4) and second round will be in Quicktime (*.mov). The CODEC for MP4 will be H.264 and for Quicktime it will be ProRes422 HQ. For all testing, the resolution will be kept to Full HD 1080p.
All media files used for the rendering will be coming from the external HDD used in the testing. Keeping all the media files is meant to act as a form of control for the test from which we can gather some insights from.
Another important note to consider, I did not use any GPU, other than the graphics card with the MacBook Pro (Intel Iris Pro Graphics). There was no discrete GPU hardware acceleration involved in the process.
Results
First I will render a project from Resolve and output to a file. The details are indicated with the results.
Rendering Round 1
Format: MP4
CODEC: H.264
Resolution: Full HD (1920 x 1080)
Frame Rate: 30 fps
File Size: 182 MB
Rendering Round 2
Format: Quicktime
CODEC: Apple ProRes422 HQ
Resolution: Full HD (1920 x 1080)
Frame Rate: 30 fps
File Size: 1.83 GB
Observations
There were variations in total rendering time. Most of the results were expected, based on hardware specifications. However, there were also some findings which were not expected. Given that MP4 uses more compression than Quicktime, what we get as a result are different file sizes for the exported video.
Round 1 (MP4) uses some lossy compression, set by the H.264 CODEC. The best performance would come from the external HDD, which was somewhat not too surprising. I had the media files stored on the same external HDD, so the processing does not require as much I/O from one device to another. In the case of the internal SSD and the macOS RAID, there would be a lot of data going from the external HDD to the PCIe bus on the internal SSD and to the USB bus on the macOS RAID. The output file size remained small at 182 MB.
The rendering times were quite close in the first round. Despite having the fastest read/write performance, the internal SSD was the slowest to render at 2:43 (2 minutes and 43 seconds). The macOS RAID did good at 2:21, though it still was slower than the external HDD despite using a stripe set that writes data across two drives with what would appear as 2 combined USB 3.1 channels (total bandwidth of approximately 10 Gbps in theory). It shows the advantage of having the media files on the same volume where you want to export the output file.
Round 2 (Quicktime) uses the ProRes422 HQ CODEC, so this is optimized for Apple systems. This was why the MacBook Pro’s internal SSD had the overall advantage in rendering time, at a speedy 1:16 for a 1.83 GB file. That was really fast, but shows how Apple code optimizes their CODEC to work better on Apple systems. The external HDD showed another good result, though not as fast as the internal SSD. The worst performance in this round was from the macOS RAID, at 5:43. I have tried it several times just to make sure, and it had been consistent.
You can blame the slow RAID performance on my choice of devices. Rather than thumb drives, a better RAID storage solution would have used fast external SSD or even higher capacity HDD. The bottleneck in the RAID I would assume would be the thumb drives since I have seen external USB drive arrays configured as a RAID volume perform much better. Another reason it took longer was the amount of data that needed to be processed and stored on the RAID volume. 1.83 GB is quite huge, and thus the larger the file size the greater the time required for rendering.
Recommendations
I do not recommend using thumb drives for configuring a RAID volume for production use. SSD and HDD drives are much better and more reliable for this solution. Another thing to do would be to use drives that have the same capacity because if they are not equal, RAID will only use the smallest drive’s capacity. That means some disk space will be wasted and that is not the most efficient utilization of resources.
Many professionals do use RAID configurations, but more complex than what can be configured on the macOS. These RAID systems are sold by third party vendors, and provide not just a stripe set for multiple drives, but also fault tolerance with parity i.e. RAID Level 5(stripe set with parity). Some of these systems are also not prone to a single point of failure because of redundancy. Others also have dedicated RAID controllers which means that the operating system does not need to manage the device. An interface will be provided for users. However, they are expensive for the average content creator and are best for production studios.
There are personal desktop-based RAID devices that provide simple connections to a computer. A simple USB or Thunderbolt RAID drive can provide a solution for high performance and fault-tolerant system for typical editors. Independent content creators can render their video without using RAID with a regular external HDD or faster SSD drive. It will depend on the budget, as SSD costs are much higher than HDD, but as SSD become more available their prices will drop. As the testing shows an external HDD (non-RAID) can still provide tremendous performance with a USB 3.1 interface or higher (e.g. USB 3.2, Thunderbolt 3).
The ideal solution is a local fast internal SSD that stores the media files on the same volume, but disk space is not infinite. Typical computer systems can only store so much locally, which is why professionals use external or network drives (e.g. cloud, NAS, SAN). RAID is highly recommended for performance optimization, but choose the solution that will work by considering whether fault-tolerance is just as important as speed. If that is out of someone’s budget, a macOS RAID 0 is only recommended if fault-tolerance is not a priority and existing resources are available (e.g. old HDD, SSD, etc.).
