5 ways to remove duplicates from an Array of objects

Test Data

Here is the sample data that is retrieved from backend through API call. We would like to remove the duplicate authors based on their name.

const data = [
  { post: { id: "1001", author: { id: "1", name: "Rocky" } } },
  { post: { id: "1002", author: { id: "2", name: "David" } } },
  { post: { id: "1003", author: { id: "1", name: "Rocky" } } },
  { post: { id: "1004", author: { id: "3", name: "Bob" } } },
  { post: { id: "1005", author: { id: "2", name: "David" } } },
];

Goal

To remove duplicate authors from the array. And it should look something like this.

const data = [
  { post: { id: "1001", author: { id: "1", name: "Rocky" } } },
  { post: { id: "1002", author: { id: "2", name: "David" } } },
  { post: { id: "1004", author: { id: "3", name: "Bob" } } },
];

1. Use reduce( ) Method

We can use reduce() method to iterate through objects and see if we already have an object added to the accumulator with the same author name as the current object.

const result = data.reduce((acc, curr) => {
  if (!acc.find((item) => item.post.author.name === curr.post.author.name))
    acc.push(curr);
  return acc;
}, []);
console.log(result);

2. Use Map Object

Using the Map is a more performant technique, but it could not preserve the order of the original array.

Here we are storing [key, value] pairs in the map with the key being the author name and value being the whole object, So only unique authors are stored in the Map. Then we spread the map.values() in the result array.

const result = [
  ...new Map(data.map((obj) => [obj.post.author.name, obj])).values(),
];
console.log(result);

3. Use filter Method + Set

We use the Set and its add and has methods.

Here we check if the author name is present in the set inside filter method and return boolean based on the condition.

const set = new Set();
const result = data.filter((obj) => {
  const duplicate = set.has(obj.post.author.name);
  set.add(obj.post.author.name);
  return !duplicate;
});
console.log(result);

4. Use filter Method + findIndex Method

we are using the findIndex method. This is probably the least performant method out all since we are looping twice. which is O(n²)T.

Here findIndex method returns the particular index if the condition is met and -1 if not.

const result = data.filter(
  (value, index, self) =>
    index ===
    self.findIndex((item) => value.post.author.name === item.post.author.name)
);
console.log(result);

5. Use loadsh.uniqueWith( )

If you have used popular library called Lodash, Then uniqueWith() function might be a suitable and fast solution for you.

const _ = require("lodash");
const result = _.uniqWith(
  data,
  (arrVal, othVal) => arrVal.post.author.name === othVal.post.author.name
);
console.log(result);

Hope this helps in you. Happy coding :)