Optimizing array formatting with filtering in JS

When you need to convert a small array of data from one format to another, you hardly ever think about performance because latest PCs with their computing capabilities perform such tasks in no time. However, when handling a vast amount of data even the slightest delay can cause an unpleasant surprise. That’s why such cases require particular deliberation and thorough research.

Let’s try to find the best way to convert array items from ItemSource to ItemTarget.

Additional requirement: the resulting array must contain unique elements only.

Let’s compare the performance of different possible solutions. We will create a test data array containing 531678 elements, including 8150 unique ones, and use it to determine execution time of every algorithm. In order to minimize the measure of inaccuracy, we will run each algorithm 100 times and then calculate the average execution time.

Formatting

The most obvious solution is to use

items.map(…)

However, let’s try to find the best iteration by using different loops. Test source.

loop speed test

Turns out, the fastest loop is for, while the “obvious” solution produces the worst results.

Filtering by id

Since we need to both format data and filter it by id, let’s create a new array that stores unique processed data. We will use the filter method to filter data of the map loop. Test source.

filtering by id with array speed test

As you see, the for loop produces the best results again. But we won’t stop here and will try to reach an even higher optimization level.

Optimizing filtering by id

Given that item.id is a unique value, let’s try using the Set object instead of an array. Test source.

filtering by id with Set speed test

Awesome!

Now, let’s save the formatted values to a key-value object and get the final result by running Object.values. Test source.

filtering by id with key-value object speed test

The results show that the combination of for + ids as Set has the best running time.

Summary

The most effective way to format data is using for to iterate the array and using Set instead of an array that stores the unique processed elements. As a result, we get the combination that runs 60 times faster than map + filter! Test source.

Filter fast and prosper!