Remove duplicate lines from any list in seconds
Mailing lists collect the same address twice, keyword exports repeat terms, log files repeat errors and copied spreadsheet columns are full of repeats. The Remove Duplicate Lines tool cleans them up instantly: paste your text, and every repeated line is removed while the order of everything else stays exactly as it was.
You control what counts as a duplicate. Decide whether “London” and “london” are the same, whether stray spaces at the ends of lines matter, what happens to blank lines, and whether to keep the first or the last copy of each repeated line. The tool also lists every duplicate it found with how many times it appeared — handy when you need to know what was repeated, not just get rid of it. It all runs in your browser, so customer lists and private data never leave your device.
How to use the Remove Duplicate Lines
- Paste your list into the input box, one item per line. Text copied from Excel, Google Sheets or a CSV column works directly.
- Choose whether to keep the first or the last occurrence of each repeated line.
- Set the options: tick “Case-sensitive” to treat “London” and “london” as different, and keep “Ignore spaces at the start and end of lines” ticked to catch duplicates that only differ by invisible spaces.
- Decide what to do with blank lines, and optionally sort the result A → Z or Z → A.
- Check the badge and the “Duplicates found” list, then copy or download the unique lines.
Remove Duplicate Lines features
Keeps your order
Unique lines stay in their original order unless you choose to sort them, so numbered or prioritised lists are not scrambled.
First or last copy
Keep the first occurrence (the usual choice) or the last one — useful for logs and exports where the newest entry is at the bottom.
Shows what was removed
A count of removed lines plus a list of every duplicate value and how many times it appeared, most frequent first.
Whitespace and case options
Ignore leading and trailing spaces, and choose case-sensitive or case-insensitive matching.
Unicode-aware matching
Accented letters typed in different ways (a precomposed “é” versus “e” plus an accent mark) are recognised as the same text.
Handles big lists
Tens of thousands of lines are processed in a fraction of a second, entirely on your device.
Which settings should you use?
| Your list | Case-sensitive | Ignore end spaces | Blank lines |
|---|---|---|---|
| Email addresses | Off — emails are not case-sensitive in practice | On | Remove |
| Keywords or tags | Off | On | Remove |
| URLs | On — paths can be case-sensitive | On | Remove |
| Code or config lines | On | Off — indentation can matter | Keep |
| Paragraphs of prose | On | On | Keep |
How duplicates are detected
Two lines are duplicates when they are identical after the options you choose are applied. With “Ignore spaces at the start and end of lines” ticked, New York and New York match, but the line you keep is copied exactly as it was written. With “Case-sensitive” unticked, matching ignores capitalisation, so Priya@Example.com and priya@example.com count as one.
Blank lines get their own setting because they usually separate groups rather than carry data. “Keep all blank lines” leaves them untouched, “Remove blank lines” deletes them, and “Treat like other lines” keeps only the first blank line. Windows (CRLF) and Unix (LF) line endings are both understood, and the result always uses standard line breaks that paste cleanly anywhere.
Tips for cleaner lists
- Pulling addresses out of a long email thread or document? Run it through the Email Extractor first, then remove duplicates here.
- Lines that look identical but are not matched usually contain a tab, a non-breaking space or a zero-width character. Clean them with Remove Extra Spaces and try again.
- To compare two lists and see what is in one but not the other, use the Diff Checker.
- Need the list in alphabetical, natural or numeric order? Sort Lines has more sorting modes and can also remove duplicates while it sorts.
Remove Duplicate Lines FAQ
How do I remove duplicate lines from a list?
Paste the list into the box, one item per line. Duplicates are removed immediately and the unique lines appear under “Unique lines”, ready to copy or download as a .txt file.
Does removing duplicate lines change the order?
No. Lines stay in their original order and only the extra copies are dropped. If you want the result sorted, choose “Sort A → Z” or “Sort Z → A” under “Order of the result”.
Why are some duplicate lines not removed?
They probably differ slightly — in capitalisation, in spaces at the ends of the line, or through invisible characters such as tabs or non-breaking spaces. Untick “Case-sensitive”, keep “Ignore spaces…” ticked, or clean the text with Remove Extra Spaces first.
Can I see which lines were duplicated?
Yes. The “Duplicates found” box lists every repeated value and how many times it appeared, with the most frequent first, and the badge shows the total number of lines removed.
How do I remove duplicates in Excel instead?
In Excel, select the column and use Data → Remove Duplicates; in Google Sheets use Data → Data cleanup → Remove duplicates. Pasting the column here is quicker when you also want case or whitespace options, or a list of what was removed.
Is there a limit on how many lines I can dedupe?
There is no fixed limit. Lists of 100,000 lines or a few megabytes of text work in modern browsers; very large inputs simply take a moment longer. Your data stays on your device.