Home › Generators & Text Tools › Remove Duplicate Lines
Remove Duplicate Lines
Use this free remove duplicate lines tool to clean up any list in seconds. Paste in names, email addresses, links, or any lines of text, and it keeps the first copy of each line and drops the rest. You can change how it treats capital letters, spaces, and blank lines, sort the result, and then copy it or download it as a plain text file.
How it works
Duplicate lines are easy to miss in a long list and slow to fix by hand. Two copies of the same name in a mailing list can mean the same person gets two emails, and a repeated item in a stock list can throw off a count. This tool finds those repeats: it splits your text into lines, compares each line with the ones it has already seen, and keeps only the first copy of each one.
The comparison is simple, line by line. The tool reads the first line and keeps it. It reads the second line and asks whether it has seen that exact line before; if not, it keeps it, and if so, it drops it. It repeats that check for every line until the end of the list. Because it only ever drops a later copy, the first copy stays in its original place, and the order of the lines that are left is the same order they were in when you pasted them.
Two options change what counts as the same line. Case-sensitive is off by default, so Apple, APPLE, and apple all count as the same line and only the first one is kept. Turn it on when capital letters matter, for example in a list of passwords, codes, or product keys, where Apple and apple really are two different values. Trim whitespace is on by default, and it removes spaces and tabs from the start and end of each line before comparing. That matters because pasted text often has a stray space after some lines, and without trimming, apple and apple with a trailing space would look like two different lines. Trimming also cleans the output, so no line starts or ends with a stray space. Turn it off only if leading or trailing spaces are part of the data itself.
Remove empty lines is on by default. A blank line is a line with nothing on it, and a line with only spaces counts as blank too. With the option on, blank lines are dropped entirely, so a list pasted from a spreadsheet or a document with gaps between groups comes out as one tight list. Turn it off if you want to keep blank lines as gaps between groups. The counts below the result count blank lines that were dropped as part of the lines removed, so the three numbers always add up: original lines equals duplicates removed plus unique lines kept.
Sort lines alphabetically is off by default. With it off, the cleaned list stays in its original order, which is what you want when the order means something, such as a ranking or the order items were added. With it on, the tool removes duplicates first and then sorts the lines that are left from A to Z. Sorting never changes which lines are kept, only the order they appear in, and the first copy of each line is still the one whose spelling and capital letters survive.
The tool updates as you type and every time you change an option, so you can watch the counts move and try a setting without starting over. When the result looks right, press Copy result to put it on the clipboard, or Download .txt to save it as a plain text file you can open anywhere. Everything runs in your browser: the text you paste never leaves your device, never goes to a server, and is never stored.
Worked example
A shopkeeper pastes a short shopping list into the tool with the default settings: case-sensitive off, trim whitespace on, remove empty lines on, and sorting off.
The list has 8 lines:
milk
eggs
bread
milk
butter
eggs
bread
cheese
Step 1: milk is new, so it is kept.
Step 2: eggs is new, so it is kept.
Step 3: bread is new, so it is kept.
Step 4: milk has been seen before, so it is dropped.
Step 5: butter is new, so it is kept.
Step 6: eggs has been seen before, so it is dropped.
Step 7: bread has been seen before, so it is dropped.
Step 8: cheese is new, so it is kept.
The result is 5 unique lines kept, in the original order:
milk
eggs
bread
butter
cheese
The counts confirm it: original lines 8, duplicates removed 3, unique lines kept 5. If the shopkeeper then turns on sorting, the same 5 lines come back in alphabetical order — bread, butter, cheese, eggs, milk — with the same counts, because sorting changes the order and nothing else.
FAQ
Does remove duplicate lines keep the first or the last copy?
It keeps the first copy. When the tool meets a line it has already seen, it drops the later copy and keeps the first one in its original place. That way the order of your list does not change, and the spelling and capital letters of the first copy are the ones that stay.
Is Apple the same as apple when case-sensitive is off?
Yes, when case-sensitive is off, which is the default. Apple, APPLE, and apple all count as the same line, so only the first one is kept. Turn case-sensitive on if Apple and apple should count as two different lines, for example in a list of passwords or codes where capital letters matter.
What happens to blank lines?
Blank lines are removed when Remove empty lines is on, which is the default. A line with only spaces counts as blank too, and it is removed with the other blank lines. Turn the option off if you want to keep blank lines as gaps between groups, and note that the first blank line is kept and later blank lines count as duplicates.
Does sorting change which lines are kept?
Sorting does not change which lines are kept, only the order they appear in. The tool removes duplicates first, then sorts the lines that are left from A to Z. The first copy of each line is still the one that is kept, so its spelling and capital letters stay the same, and the sorted list is easier to scan or compare with another list.
Is my text sent to a server?
No. Everything runs in your browser on your own device. The text you paste is never uploaded, stored, or shared with anyone. You can use it for private lists, such as email addresses or customer names, and close the page when you are done.