Identify Duplicates in a List in Java

This post will discuss how to identify duplicates in a List in Java.

1. Using `Set`

A simple solution is to iterate through all values in the list and insert each element into a HashSet. If the current element already exists in the set, then it is a duplicate. You can collect all duplicates found in a new list.

This can be easily done using Java 8 Stream. Note that the solution uses the return value of the Set.add() method to determine if the value is already present in the set or not

import java.util.HashSet;

import java.util.List;

import java.util.Set;

import java.util.stream.Collectors;

class Main

{

private static <T> Set<T> findDuplicates(List<T> list)

{

Set<T> seen = new HashSet<>();

return list.stream()

.filter(e -> !seen.add(e))

.collect(Collectors.toSet());

}

public static void main(String[] args) {

List<Integer> values = List.of(1, 3, 2, 3, 4, 1);

Set<Integer> duplicates = findDuplicates(values);

System.out.println(duplicates);

}

Download Run Code

Output:

[1, 3]

The above code is equivalent to the following code (for Java 7 and less):

import java.util.HashSet;

import java.util.List;

import java.util.Set;

class Main

{

private static <T> Set<T> findDuplicates(Iterable<T> iterable)

{

Set<T> duplicates = new HashSet<>();

Set<T> seen = new HashSet<>();

for (T t : iterable) {

if (!seen.add(t)) {

duplicates.add(t);

}

return duplicates;

}

public static void main(String[] args) {

List<Integer> values = List.of(1, 3, 2, 3, 4, 1);

Set<Integer> duplicates = findDuplicates(values);

System.out.println(duplicates);

}

Download Run Code

Output:

[1, 3]

2. Using `Collections.frequency()` method

The simplest way is probably to count the frequency of each list element to determine if it is a duplicate or not. You can use the Collections.frequency() method for this, which returns the number of elements in the collection.

Here’s code using Stream API:

import java.util.Collections;

import java.util.List;

import java.util.Set;

import java.util.stream.Collectors;

class Main

{

private static <T> Set<T> findDuplicates(List<T> list)

{

return list.stream().distinct()

.filter(i -> Collections.frequency(list, i) > 1)

.collect(Collectors.toSet());

}

public static void main(String[] args) {

List<Integer> values = List.of(1, 3, 2, 3, 4, 1);

Set<Integer> duplicates = findDuplicates(values);

System.out.println(duplicates);

}

Download Run Code

Output:

[1, 3]

This is equivalent to following:

import java.util.Collections;

import java.util.HashSet;

import java.util.List;

import java.util.Set;

class Main

{

private static <T> Set<T> findDuplicates(List<T> list)

{

Set<T> set = new HashSet<>();

for (T i : list) {

if (Collections.frequency(list, i) > 1) {

set.add(i);

}

return set;

}

public static void main(String[] args) {

List<Integer> values = List.of(1, 3, 2, 3, 4, 1);

Set<Integer> duplicates = findDuplicates(values);

System.out.println(duplicates);

}

Download Run Code

Output:

[1, 3]

3. Using `Collectors.groupingBy()` function

The above solution calls the Collections.frequency() method for each element of the list. A better solution is to create a frequency map and use that to determine if an element is duplicated or not. The following Java 8 solution uses Streams to filter the items having the frequency of more than 1:

import java.util.List;

import java.util.Map;

import java.util.Set;

import java.util.function.Function;

import java.util.stream.Collectors;

class Main

{

private static <T> Set<T> findDuplicates(List<T> list)

{

Map<T, Long> frequencyMap = list.stream()

.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));

return frequencyMap.keySet().stream()

.filter(key -> frequencyMap.get(key) > 1)

.collect(Collectors.toSet());

}

public static void main(String[] args) {

List<Integer> values = List.of(1, 3, 2, 3, 4, 1);

Set<Integer> duplicates = findDuplicates(values);

System.out.println(duplicates);

}

Download Run Code

Output:

[1, 3]

That’s all about identifying duplicates in a List in Java.

Identify Duplicates in a List in Java

1. Using Set

2. Using Collections.frequency() method

3. Using Collectors.groupingBy() function

1. Using `Set`

2. Using `Collections.frequency()` method

3. Using `Collectors.groupingBy()` function