Find Duplicate Words in a String in Java (Count and Remove)

To find duplicate words in a string in Java, we split the string into words and count each word in a Map, or add each word to a Set, where add() returns false for a word we have already seen. Every word with a count above one is a duplicate.

We need this for interview questions, but also in real code, for example to flag repeated tags on a blog post, to count keywords in a product review, or to catch a typo such as “the the” in user text.

The following example counts every word of a sentence with streams and keeps the words that appear more than once.

String sentence = "alex brian charles alex charles david eric david";
Map<String, Long> counts = Arrays.stream(sentence.split("\\s+"))
        .collect(Collectors.groupingBy(w -> w, LinkedHashMap::new, Collectors.counting()));
                                               // {alex=2, brian=1, charles=2, david=2, eric=1}
List<String> duplicates = counts.entrySet().stream()
        .filter(e -> e.getValue() > 1)
        .map(Map.Entry::getKey)
        .toList();                             // [alex, charles, david]

We can see that the LinkedHashMap keeps the words in the order they first appear. We solve the same task with Java Streams and with plain Collections, handle letter case and punctuation, find repeated neighbors such as “the the” with a regex, remove duplicates, and compare the cost of each approach.

1. The Problem of Finding Duplicate Words in a String

Suppose we have a string with names. We want to find the names that appear more than once, and we may also want the number of occurrences of each duplicate word, or of every word.

String sentence = "alex brian charles alex charles david eric david";

The string contains three duplicate words that occur twice, and two unique words.

alex=2
charles=2
david=2

brian=1
eric=1

Every solution follows the same three steps. We split the text into words, count or remember each word, and keep the words seen more than once. The approaches differ only in the data structure and in how they treat case and punctuation.

2. Find Duplicate Words using Stream

The Stream API lets us split, count and filter in one pipeline. We split the string on whitespace with the regex \\s+, which also handles several spaces in a row, and turn the String[] into a stream of words.

2.1. Using a Set to Detect Repeats

The method Set.add() returns false when the element is already in the set. So a filter that keeps the words for which add() fails returns every repeated occurrence.

String sentence = "alex brian charles alex charles david eric david";
Set<String> seen = new HashSet<>();
List<String> duplicateWords = Arrays.stream(sentence.split("\\s+"))
        .filter(w -> !seen.add(w))
        .toList();                             // [alex, charles, david]

The filter changes a set from outside the lambda, so it is a stateful behavioral parameter. It works for a sequential stream, but a parallel stream can give wrong results, so the counting approach in the next section is the safer default.

2.2. Counting Each Word with groupingBy()

To count the occurrences of each word, we collect the words with Collectors.groupingBy() and Collectors.counting(). The map factory LinkedHashMap::new keeps the first-seen order, because a plain HashMap does not guarantee any order.

String sentence = "alex brian charles alex charles david eric david";
Map<String, Long> wordCounts = Arrays.stream(sentence.split("\\s+"))
        .collect(Collectors.groupingBy(w -> w, LinkedHashMap::new, Collectors.counting()));
                                               // {alex=2, brian=1, charles=2, david=2, eric=1}
Map<String, Long> dupWordsWithCount = wordCounts.entrySet().stream()
        .filter(e -> e.getValue() > 1)
        .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue, (a, b) -> a, LinkedHashMap::new));
                                               // {alex=2, charles=2, david=2}

The second statement keeps only the duplicate words and their counts by filtering the entries of the map. The older form with toMap() gives the same counts as Integer values. The merge function Math::addExact adds the counts when a word appears again.

String sentence = "alex brian charles alex charles david eric david";
Map<String, Integer> wordsMapWithCount = Arrays.stream(sentence.split("\\s+"))
        .collect(Collectors.toMap(w -> w, w -> 1, Math::addExact, LinkedHashMap::new));
                                               // {alex=2, brian=1, charles=2, david=2, eric=1}

3. Find Duplicate Words using Collections

Without streams, the process is the same. We split the string, loop over the words and use the HashSet add() method to tell a new word from a repeated one.

String sentence = "alex brian charles alex charles david eric david";
String[] words = sentence.split("\\s+");
Set<String> unique = new HashSet<>();
Set<String> repeated = new LinkedHashSet<>();
for (String word : words) {
    if (!unique.add(word)) {
        repeated.add(word);
    }
}
String found = repeated.toString();            // "[alex, charles, david]"

Collecting the repeats in a LinkedHashSet reports each duplicate once, even when a word appears three or more times. To count the words in a loop, Map.merge() adds one to the existing count or stores 1 for a new word.

String[] words = "alex brian charles alex charles david eric david".split("\\s+");
Map<String, Integer> frequency = new LinkedHashMap<>();
for (String word : words) {
    frequency.merge(word, 1, Integer::sum);
}
String freq = frequency.toString();            // "{alex=2, brian=1, charles=2, david=2, eric=1}"

The method Collections.frequency(list, word) also counts how often a word appears, but it scans the whole list on every call. Calling it for each word turns a linear task into a quadratic one, so we use it only for a single lookup.

List<String> wordList = List.of("alex", "brian", "david", "eric", "david");
int davidCount = Collections.frequency(wordList, "david");  // 2

4. Ignoring Letter Case and Punctuation

Real text has capital letters and punctuation, so splitting on whitespace treats “Food”, “food” and “food,” as three different words. Before counting, we convert the text to lower case with Locale.ROOT, which gives the same result on every machine, and split on every run of characters that are not letters, digits or apostrophes.

String review = "Great food, great service. Food was hot!";
Map<String, Long> reviewCounts = Arrays.stream(review.toLowerCase(Locale.ROOT).split("[^\\p{L}\\p{N}']+"))
        .filter(w -> !w.isEmpty())
        .collect(Collectors.groupingBy(w -> w, LinkedHashMap::new, Collectors.counting()));
                                               // {great=2, food=2, service=1, was=1, hot=1}

The class \\p{L} matches letters of every alphabet, so words such as naïve stay whole, which the ASCII-only class \\W would split. The filter() drops the empty string that split() returns when the text starts with punctuation.

A review platform can use these counts to find the words customers repeat most, such as “great” and “food” here, and show them as tags above the reviews. Without the lower-case step, “Great” and “great” would land in separate tags.

5. Finding Repeated Adjacent Words with Regex

Sometimes only neighbors matter, for example when a text editor flags “the the” as a typo. A regular expression with a backreference finds those pairs without any collection.

Pattern doubled = Pattern.compile("\\b(\\w+)(\\s+\\1\\b)+", Pattern.CASE_INSENSITIVE);
String draft = "We ship the the order today. Today today it ships";
List<String> typos = doubled.matcher(draft).results().map(m -> m.group()).toList();
                                               // [the the, Today today]
String fixed = doubled.matcher(draft).replaceAll("$1");  // "We ship the order today. Today it ships"

The group (\\w+) captures a word, and \\1 must match the same word again after whitespace. The flag CASE_INSENSITIVE lets “Today today” match, and replaceAll(“$1”) keeps only the first word of each pair. Notice that “today. Today” is not a match, because the period sits between the two words.

6. Removing Duplicate Words from a String

To remove duplicate words and keep the first occurrence of each, we put the words into a LinkedHashSet and join them again. The stream version uses distinct(), which also keeps the encounter order.

String sentence = "alex brian charles alex charles david eric david";
String deduped = String.join(" ", new LinkedHashSet<>(Arrays.asList(sentence.split("\\s+"))));
                                               // "alex brian charles david eric"
String distinctWords = Arrays.stream(sentence.split("\\s+")).distinct().collect(Collectors.joining(" "));
                                               // "alex brian charles david eric"

7. Comparing the Approaches

All approaches split the text first, so they differ in how they count. With n words, the set and map solutions run in linear time, which is what an interviewer expects to hear.

Approach	Time	Gives counts	Keeps order
HashSet.add() in a loop or filter	O(n)	No	With LinkedHashSet for the result
groupingBy() + counting()	O(n)	Yes	With LinkedHashMap::new
Map.merge() in a loop	O(n)	Yes	With LinkedHashMap
Collections.frequency() per word	O(n * k), k distinct words	Yes	Depends on the source list
Regex with backreference	O(n)	No, adjacent pairs only	Yes

For most code, groupingBy() with counting() is the clearest choice, because it gives every count and the duplicates in two short steps. The Set.add() trick is the shortest answer when we need only a yes or no per word.

8. Duplicate Words FAQs

An interviewer who sees a working duplicate-word program tends to change one rule next, such as counting, letter case, removal or the first repeat.

8.1. How do we count the occurrences of each word in a string?

We split the string and collect the words with Collectors.groupingBy(w -> w, Collectors.counting()), or call map.merge(word, 1, Integer::sum) in a loop. Both return a map from each word to its count, as shown in section 2.

8.2. How do we find duplicate words ignoring case?

We convert the string with toLowerCase(Locale.ROOT) before splitting. For adjacent repeats, the regex flag Pattern.CASE_INSENSITIVE does the same job without changing the text.

8.3. How do we find the first repeated word?

We loop over the words and return the first word for which Set.add() returns false. The loop stops at that word, so it does not read the rest of the text.

static Optional<String> firstRepeated(String text) {
    Set<String> seen = new HashSet<>();
    for (String word : text.split("\\s+")) {
        if (!seen.add(word)) {
            return Optional.of(word);
        }
    }
    return Optional.empty();
}

Optional<String> first = firstRepeated("alex brian charles alex charles");  // Optional[alex]
Optional<String> noRepeat = firstRepeated("alex brian");                      // Optional.empty

8.4. Why does split(” “) give empty words?

The regex ” “ matches one space, so two spaces in a row produce an empty string between them. The regex \\s+ matches any run of whitespace and avoids empty words in the middle of the text.

9. Conclusion

Finding duplicate words always starts with splitting the text. A Set tells us whether a word was seen before, and a Map built with groupingBy() or merge() gives the count of every word, from which the duplicates are one filter away.

For real text, we lower-case with Locale.ROOT and split on non-letters, and for typos such as “the the” a regex with a backreference is enough. The same programs list unique words too, by keeping the entries with a count of one. More practice tasks are in the String interview questions.

10. References

• Java 25 API, Collectors
• Java 25 API, Set.add()
• Java 25 API, Pattern
• Java 25 API, java.util.stream package

Happy Learning !!

Source Code on Github