Find duplicate files

How files are compared

A file that is found to be different at one step is not read at the next.

1. Size: files with different sizes have different content, so they are not read. A file with no other file of the same size is not read either.
2. Start and end: files with the same size are compared by their first and last 64 KiB. Files of 128 KiB or less are compared in full at this step.
3. SHA-256 of the whole file: the remaining candidates are read in full, and files whose SHA-256 (a 64-character value computed from the content) matches are treated as identical.
4. Byte-by-byte comparison (optional): if you choose “Also compare byte by byte at the end”, the first file in each set is compared with the others one byte at a time.

File names and dates are not used to decide. Files whose content differs even slightly, such as similar photos, are not looked for. Files with a size of 0 have no content and are listed separately.

Which file to keep

The first file in each set is marked “Keep”. You can choose the rule: shortest path, oldest, newest, or the order you chose them in. It is only a suggestion.

This tool does not delete, move or change any file.

Choosing files

• “Choose a folder” selects everything in a folder, including subfolders. If your browser can’t select folders, choose files or drop a folder instead.
• You can add more at any time. A file with the same path, size and date is counted once even if you choose it twice.

Saving the list

“Save list as CSV” saves the set, role (keep or duplicate), path, size in bytes, modification time (UTC) and SHA-256 as a UTF-8 CSV with BOM. Paths are not changed. If some paths could be treated as formulas by a spreadsheet app, the number of them is shown.

Limits

Up to 20,000 files and 64 GiB in total. Large files are read 8 MiB at a time and never loaded into memory whole. How fast SHA-256 is computed depends on the device; many large files take longer. You can cancel at any time.