Remove Duplicate Lines
Dedupe a list, sort it, drop blank lines, and see what was repeated. Runs in your browser — your text never leaves your device.
Repeated lines
How to Remove Duplicate Lines
- Paste your list, one item per line.
- Choose whether to ignore case and trim whitespace when comparing.
- Optionally sort the result and drop empty lines.
- Check the duplicates panel to see what was repeated and how often.
- Copy the cleaned list or download it.
Why duplicates survive a dedupe
Two lines that look identical are not always identical to a computer. These are the usual culprits, and the setting that catches each:
| Difference | Example | Fixed by |
|---|---|---|
| Trailing space or tab | "alice@example.com␣" | Trim whitespace |
| Capitalisation | Alice@Example.com | Ignore case |
| Windows line endings | a line ending in \r | Handled automatically |
| Non-breaking space | pasted from a web page or Word | Trim whitespace (at the ends) |
| Look-alike characters | Latin "a" vs Cyrillic "а" | Nothing generic — they are different characters |
The repeated-lines panel shows exactly which lines matched and how often, so you can check the tool counted what you expected before you trust the result.
Alphabetical versus natural sort
A → Z
file1.txt file10.txt file11.txt file2.txt file3.txt
Natural
file1.txt file2.txt file3.txt file10.txt file11.txt
Alphabetical sorting compares one character at a time, so "10" is decided by its "1" and lands before "2". Natural sort reads runs of digits as whole numbers. Both sorts here are locale-aware, so accented letters sort next to their plain versions rather than after "z".
Specifications
- Accepts
- Text — type or paste
- Gives you
- lines.txt download · copy to clipboard
- Where it runs
- Your browser — the file is never uploaded
- Sign-up
- None
- Cost
- Free, with no usage limits
FAQ
Is my text sent anywhere?
No. Everything runs in your browser as you type. Lists of email addresses and customer names are exactly the sort of thing that should not be pasted into a site that uploads them.
Which copy of a duplicate is kept?
The first one, in its original position. Later repeats are removed. That keeps the list in the order you gave it, which matters when the order means something — a ranking, a log, a priority list.
What does "ignore case" change?
With it on, Apple, apple and APPLE count as the same line and only the first spelling is kept. Off, they are three different lines. Email addresses are the usual reason to turn it on, since the part after the @ is not case-sensitive.
Why does "trim whitespace" matter so much?
Because invisible differences stop duplicates matching. A line pasted from a spreadsheet often has a trailing space or tab, and "alice@example.com " is a different string from "alice@example.com". Trimming removes spaces at both ends before comparing — and in the output.
What is natural sort?
Sorting the way a person would: item2 before item10. Plain alphabetical sort compares character by character, so item10 lands before item2 because 1 comes before 2. Natural sort reads the digits as numbers. Use it for filenames, versions and anything numbered.
How big a list can it handle?
Hundreds of thousands of lines without trouble — duplicate detection uses a hash set, so it takes one pass however long the list is. The practical limit is how much text your browser is comfortable holding in a text box.