Remove Duplicates From Two Arrays

Photo by Aedrian on Unsplash

At some point you might come across the scenario where you have two shallow arrays that need comparison for duplicates. This usually shows up as an original array and the updated one. The updated one could have items added or removed and you’re tasked with comparing the two so you can remove the duplicates and be left with only the new, unique items. How can we go about this?

On an array with only primitive types we can utilize a couple different methods.

const originalArray = ['James', 'Amanda', 'Arturo', 'Jasmine', 'Lucy'];
const updatedArray = ['James', 'Arturo', 'Steve', 'Trudy'];

const uniqueArray = originalArray.filter(name => {
  return updatedArray.indexOf(name) === -1;
});

console.log(uniqueArray);
// ['Amanda', 'Jasmine', 'Lucy']

Above we can use the filter method on the original array. The filter function takes in a predicate function (A function returning a boolean). It will then return us an array containing every element that returns true from the passed in predicate function. So for each name in the originalArray, we compare it against the updatedArray by calling the indexOf method on the updatedArray. The indexOf method returns the first index that matches the passed in element (in this case our name at the current iteration), or -1 if it is not there. So we check using the strict equality operator (===) to see if it returned to us a negative one (-1) or not. This means with the current name we’re looking at in our originalArray, if we cannot find it’s index existing in the updatedArray (a returned -1), then it must be unique, so return that name. And so we are left with [‘Amanda’, ‘Jasmine’, ‘Lucy’].

We have a method we can use to make the above syntax a little more human readable and easier to understand. This is the includes method.

const originalArray = ['James', 'Amanda', 'Arturo', 'Jasmine', 'Lucy'];
const updatedArray = ['James', 'Arturo', 'Steve', 'Trudy'];

const uniqueArray = originalArray.filter(name => {
  return !updatedArray.includes(name);
});

console.log(uniqueArray);
// ['Amanda', 'Jasmine', 'Lucy']

Here we see the filter function taking in the name at each iteration of the originalArray with its predicate function. Then we call the includes methods on the updatedArray. The includes method checks to see if the passed in value is an entry in the array it is called on and returns true or false. Now this reads much nicer. For each name in the originalArray, if the name is NOT in the updatedArray, then that means it is not a duplicate (it’s unique), so return it. And again we are left with [‘Amanda’, ‘Jasmine’, ‘Lucy’].

Additionally this could be inverted if you wanted the duplicates instead of the unique items.

const duplicateArray = originalArray.filter(name => {
  return updatedArray.indexOf(name) !== -1;
});

// OR 

const duplicateArray = originalArray.filter(name => {
  return updatedArray.includes(name);
});

Taking this one step further, we can compare arrays that hold more complex data types such as objects. To do this we can utilize yet another array method called find.

const originalArray = [ 
  {name: 'James', employeeId: 123, title: 'Custodian'}, 
  {name: 'Amanda', employeeId: 444, title: 'Operations Manager'}, 
  {name: 'Arturo', employeeId: 897, title: 'Salesman'}
  ];

  const updatedArray = [ 
  {name: 'Arturo', employeeId: 897, title: 'Salesman'}
  ];

  const uniqueArray = originalArray.filter(employeeObject => {
    return !updatedArray.find(updatedEmployeeObject => {
      return updatedEmployeeObject.employeeId === employeeObject.employeeId
    })
  })

  console.log(uniqueArray);
    // [ 
    // {name: 'James', employeeId: 123, title: 'Custodian'}, 
    // {name: 'Amanda', employeeId: 444, title: 'Operations Manager'} 
    // ]

This looks complicated at first, but using what we know we can add the find method to solve our problem. The find method takes in a comparison function to return the element matching the condition, or if no element is found, returns undefined. So above, we loop over our originalArray with filter and for each employeeObject we want to check our updatedArray to try and FIND the updatedArrayEmployeeObject with the matching employeeId. This means we found a duplicate. Then we add the logical “not” operator (!) to say we don’t want it. We only want it if the id of that object is unique. In this way we use a unique property within an object to be able to compare our originalArray against another array and remove duplicate objects.

Additionally this could be inverted if you wanted the duplicates instead of the unique items. Just remove the logical “not” operator.

Hopefully this helps you if you are ever in need of comparing arrays.

I’m Cam Remesz, an Angular/Typescript developer. I’m self taught in Front-End Development and a bootcamp graduate. I write about JavaScript and try to make technical content more digestible. You can follow or get in touch with me on LinkedIn here!