Remove duplicate arrays from a set of arrays in Java

This post will discuss how to remove duplicate arrays from a set of arrays in Java.

The arrays in Java inherit from the Object class and use the default hashCode() and equals() methods. Using arrays as keys in a HashMap or a HashSet doesn’t work as expected. This is because the hashCode() method depends on the reference and there is no way to override it. For example, consider the following code:

import java.util.Arrays;

import java.util.HashSet;

import java.util.Set;

class Main

{

public static void main(String[] args) {

String[] first = { "A", "B", "C" };

String[] second = { "D", "E", "F" };

Set<String[]> set = new HashSet<>(Arrays.asList(first, second));

set.add(Arrays.copyOf(first, first.length)); // insert copy of first

set.stream().map(Arrays::toString).forEach(System.out::println);

}

Download Run Code

Output:

[D, E, F]
[A, B, C]
[A, B, C]

This results in a set containing three arrays: [A, B, C], [D, E, F], [A, B, C], where the array [A, B, C] is repeated. The task is to remove the duplicate arrays from the set.

1. Use a `Set<List>`

The recommended solution is to use Set<List> instead of a Set<String[]>, which uses a hash code derived from the its elements using the following calculation:

int hashCode = 1;

for (E e : list)

hashCode = 31 * hashCode + (e==null ? 0 : e.hashCode());

}

This way, we can easily eliminate the duplicate arrays by adding them to a Set of Lists. However, this may change the order of the elements in the arrays. For example, the following solution uses a Set<List> instead of a Set<String[]>:

import java.util.*;

class Main

{

public static void main(String[] args)

{

List<String> first = Arrays.asList("A", "B", "C");

List<String> second = Arrays.asList("D", "E", "F");

Set<List<String>> set = new HashSet<>();

set.add(first);

set.add(second);

set.add(new ArrayList<>(first)); // insert copy of first

System.out.println(set); // [[A, B, C], [D, E, F]]

}

Download Run Code

2. Using a `TreeSet`

Another option is to use the TreeSet implementation of the Set interface with a custom comparator, which performs comparison between two elements using its compareTo() or compare() method. A TreeSet is a sorted set that compares its elements using a comparator. We can define our own comparator to compare two arrays for equality based on their elements. This way, we can maintain the order of the elements in the arrays and remove the duplicates by adding them to a TreeSet. For example, the following code initializes a TreeSet object with a custom comparator that compares two String arrays for equality. Now, two arrays that are deemed equal by the custom comparator are also considered the same from the standpoint of the set.

import java.util.Arrays;

import java.util.Set;

import java.util.TreeSet;

class Main

{

public static void main(String[] args)

{

String[] first = { "A", "B", "C" };

String[] second = { "D", "E", "F" };

Set<String[]> set = new TreeSet<>((x, y) -> {

for (int i = 0; i < x.length; i++){

int value = x[i].compareTo(y[i]);

if (value != 0) {

return value;

}

return Integer.compare(x.length, y.length);

});

set.add(first);

set.add(second);

set.add(Arrays.copyOf(first, first.length)); // insert copy of first

set.stream().map(Arrays::toString).forEach(System.out::println);

}

Download Run Code

Output:

[A, B, C]
[D, E, F]

Instead of writing our comparator, we can also use built-in comparator Arrays::compare that compares two arrays in lexicographical order. Here is an example of applying this comparator to a set of string arrays in Java:

import java.util.Arrays;

import java.util.Set;

import java.util.TreeSet;

class Main

{

public static void main(String[] args)

{

String[] first = { "A", "B", "C" };

String[] second = { "D", "E", "F" };

Set<String[]> set = new TreeSet<>(Arrays::compare);

set.add(first);

set.add(second);

set.add(Arrays.copyOf(first, first.length)); // insert copy of first

set.stream().map(Arrays::toString).forEach(System.out::println);

}

Download Run Code

Output:

[A, B, C]
[D, E, F]

That’s all about removing duplicate arrays from a set of arrays in Java.

Remove duplicate arrays from a set of arrays in Java

1. Use a Set<List>

2. Using a TreeSet

Share this:

1. Use a `Set<List>`

2. Using a `TreeSet`