Remove duplicate and empty lines

How it works

• Each cleanup step is a separate choice. Only the selected steps are applied; with nothing selected, the text stays the same. Your choices are remembered on this device (your text is not).
• Order: spaces at line start and end → empty lines → duplicate lines. When trimming is on, duplicates are compared after trimming.
• Duplicate lines are lines with exactly the same characters. The first occurrence is kept, and the order of lines does not change. Upper and lower case, and full-width and half-width characters, count as different.
• Empty lines are lines with nothing in them or only spaces. Empty lines are never treated as duplicates, so paragraph breaks stay when you only remove duplicates.
• Spaces include regular and full-width spaces, tabs and non-breaking spaces. Invisible characters such as zero-width spaces are not treated as spaces.

Lines that look the same but are different

Some lines look identical but use a different sequence of characters, for example “é” typed as one character versus “e” followed by a combining accent. These lines are kept as separate lines, and the number of such lines is shown in a notice. The tool does not correct spelling or unify different spellings.

Line breaks and saving

• LF, CRLF and CR are all treated as one line break. The result uses LF. A final line break is kept if your text had one.
• “Save as text file” saves text-cleanup.txt in UTF-8 (without BOM).

Limits

Up to 500,000 characters (UTF-16 units). The list shows the first 2,000 lines; copying and saving always include the full text.