How to Remove Duplicates from Name and Email Lists and Sort Them

When you gather event attendee lists, survey respondent lists or newsletter subscriber lists from several places and merge them, the same person inevitably appears two or three times. This guide explains the order of steps for putting a list one item per line, removing duplicates and sorting it.

The basic pipeline

The built-in recipe runs in this order.

1. Trim leading and trailing spaces — removes spaces at the start and end of each line.
2. Remove blank lines — deletes all blank lines.
3. Remove duplicate lines — removes duplicates with 'Ignore case' turned on.
4. Sort — sorts in alphabetical (natural) order.

minsu@example.com
 jieun@example.com
Minsu@Example.com

park@example.com
jieun@example.com

The input above is cleaned up as follows. Remove duplicate lines keeps the first occurrence, so minsu@example.com remains.

jieun@example.com
minsu@example.com
park@example.com

Why duplicates are not caught

If entries look the same to you but are not removed, it is almost always one of these three.

• Leading, trailing and repeated spaces: Remove duplicate lines has 'Ignore leading, trailing and repeated spaces' on by default, so spacing differences are ignored when comparing. However, the spaces in the lines that remain are unchanged, so put Trim leading and trailing spaces first for a clean result.
• Case: 'Ignore case' is off by default. Email addresses easily get mixed spellings that differ only in case, so turn it on. Turn it off if you need to distinguish names in a list of English names.
• Invisible characters: values copied from the web or a messenger may carry a zero-width space (U+200B) or a BOM (U+FEFF). The ignore-spaces option does not treat these as spaces, so add Clean invisible characters and quotes at the very top of the pipeline to remove them first.

When order must be preserved

If the order of sign-up or submission matters, switch off or delete the sort step. Remove duplicate lines works without sorting and keeps the first occurrence in its original position, so the 'first come, first listed' order is preserved. Every step in the pipeline can be switched on and off, so compare the results and decide.

Comma-separated lists

If the list looks like a@x.com, b@x.com, c@x.com, as when copied from an email recipient field, it has to be spread one item per line before line-based operations work. Add Add line breaks (split) at the very top of the pipeline, set 'Split by' to 'Delimiter' and the delimiter to ,. Spaces at the start of each line are removed automatically after splitting. For a semicolon-separated list, enter ; as the delimiter.

When names and addresses appear together

In a list like John Doe <john@example.com>, where name and address appear together, the same address is treated as a different line if the name is written differently. To remove duplicates based on the address alone, keep only the address using the regex mode of Find and replace. With the search text ^.*<([^>]+)>.*$ and the replacement $1, only the address inside the angle brackets remains. Lines without angle brackets are not changed.

The order in which to merge lists

Remove duplicate lines keeps the line that appears first, so the order in which you paste the lists determines the result. For example, when merging an official member list with an event sign-up list, if you put the member list on top, people who appear in both are kept as written in the member list. Paste the list with the more accurate spelling on top.

To keep only addresses from a specific company, set the line filter to 'Keep only matching lines' with regex on and enter the domain, as in @example\.com$. Adding Number lines at the end after sorting gives you a final list with participant numbers.

Choosing a sort mode

• Alphabetical (natural): alphabetical order for English and ganada (Korean alphabetical) order for Korean, with item2 before item10.
• Numeric value: use it when each line contains a number, such as an employee ID or a score. Lines without a number go to the end.
• Random: use it to decide a draw or presentation order. The result differs every time it runs.

Limitations

• It cannot tell apart different people with the same name. Deduplicating a list of names only may remove a different person, so where possible use a list with unique values such as email addresses or phone numbers.
• Spellings with different spacing, such as Mary Ann and MaryAnn, do not become the same value even with the ignore-spaces option.
• When spreading a list by commas, a value that contains a comma, such as Smith, John, is split into two lines. Skim the spread-out result once.
• You can see how many lines were removed from the change in line count shown on the pipeline step.

Joining back into one line

To put the cleaned list into an email recipient field, add Remove line breaks at the end of the pipeline and enter , or ; as a 'Custom' separator. Cleaning up line by line and joining into one line only at the end is the order with the fewest mistakes.

Checklist

• Spread comma-separated lists onto separate lines first.
• Turn on Ignore case for email lists.
• If order matters, switch off the sort step.

Last updated: 2026-09-23