There are multiple ways to remove duplicates from an array with JavaScript. But, it’s very important to know the details of how efficient they, especially if you’re dealing with large arrays.
The easiest way is to use ES6’s (ECMAScript 2015) Set
object. This let’s you store unique values of any type. It will automatically remove duplicates for us, isn’t that sweet?
const fruit = [
'apple',
'orange',
'avo',
'pear',
'cherries',
'strawberries',
'avo',
'avo',
];
let uniqueFruit = [...new Set(fruit)];
console.log(uniqueFruit); // ['apple', 'orange', 'avo', 'pear', 'cherries', 'strawberries']
Here we had to use the spread
operator to get an array from Set
again.
We can use the built in filter method of ES5 to achieve the same:
const fruits = [
'apple',
'orange',
'avo',
'pear',
'cherries',
'strawberries',
'avo',
'avo',
];
const uniqueFruit = fruits.filter(
(item, i, arr) => {
return arr.indexOf(item) == i;
}
);
console.log(uniqueFruit); // ['apple', 'orange', 'avo', 'pear', 'cherries', 'strawberries']
What we’re doing here is not that complex. We’re going through each element and check if the first position of this item in the array is equal to current position. Since these two positions are different for duplicate items, they will get filtered and we end up with a unique array 🤩.
This approach is to place each item in a hashtable
and then check for it’s presence. This gives a linear time, but has at least two pitfalls:
['1', 1]
will return [1]
.function unique(fruits) {
var seen = {};
return fruits.filter(item => {
return seen.hasOwnProperty(item)
? false
: (seen[item] = true);
});
}
A better approach combines both approaches. It uses hash lookups for primitive types and linear search for objects:
function unique(fruits) {
var prims = {
boolean: {},
number: {},
string: {},
},
objs = [];
return fruits.filter(item => {
var type = typeof item;
if (type in prims)
return prims[type].hasOwnProperty(item)
? false
: (prims[type][item] = true);
else
return objs.indexOf(item) >= 0
? false
: objs.push(item);
});
}
Another option is to sort the array and then remove the items equal to the one right after:
function unique(fruits) {
return fruits
.sort()
.filter((item, pos, ary) => {
return !pos || item != ary[pos - 1];
});
}
This item is similar to hash table approach, it does not work with objects. And more importantly we have to change the original array which is a side effect and not a good way to go about this.
const uniqueFruits = fruits.reduce((a, b) => {
if (a.indexOf(b) < 0) a.push(b);
return a;
}, []);
console.log(uniqueFruits); // ['apple', 'orange', 'avo', 'pear', 'cherries', 'strawberries']
This approach uses an accumulator to see whether the item is previously inserted into it or not. Very similar to filter approach we introduced earlier, a bit faster though.
These are just a couple of ways of doing the same operation. But as you can see, very different in nature and support in browsers. Pick carefully and happy coding.