Duplicate File Finder
Find identical files — and look-alike photos — across your folders, on your device.
Choose what to search
Add several folders to compare them with each other — a backup and the originals, for example.
No files chosen yet.
Comparing files
Result
Nothing is moved or deleted: tick Keep on the copies you want, then use the list in your file manager.
About the Duplicate File Finder
Choose one or more folders — your documents, a phone backup, an old external drive — and find every file that exists more than once, whatever its name: two files are listed together only when their bytes are identical, checked with SHA-256. Optionally, look for similar photos too: resized, recompressed or lightly edited copies that are no longer byte-identical.
Each group suggests one copy to keep (the oldest, the newest or the one with the shortest path — your choice), shows how much space the others take, and lets you tick the copies you want. Export the list as a CSV or copy the paths of the duplicates for your file manager. The files are read on your device and never uploaded, and nothing on your disk is changed.
How to use it
- Press Choose a folder (add several to compare them with each other), choose files, or drop files and folders onto the page. Subfolders are included.
- Optionally set a minimum size, and tick Also find similar photos to compare pictures by what they show.
- Press Find duplicates. A progress bar shows the quick check of same-size files, then the full comparison of large files.
- Review the groups: the suggested copy is ticked Keep — tick or untick others as you like. Filter by file type and sort by space, number of copies or path.
- Press Download list (CSV) for a record of every file and what you decided, or Copy paths of duplicates to find and remove them in your file manager.
Examples
Photos/report.pdf and Backup/report.pdf (identical, 5,000 bytes) · Videos/clip.mp4 and Backup/clip.mp4 (identical, 300 KB) · Backup/clip-edited.mp4 (the same size as clip.mp4, one byte different)
Two groups: the clip.mp4 pair and the report.pdf pair. clip-edited.mp4 is not listed — its first and last 64 KB match, but the full SHA-256 shows that one byte differs.
IMG_2041.jpg (4000 × 3000) and the 1600 × 1200 copy a chat app saved, with “Also find similar photos” ticked
Not identical files, so they are not under Identical files; they form a group under Similar photos, with the larger original suggested for keeping.
How identical files are found
- Size. Files can only be identical if they have the same size, so files with a size of their own are never read.
- Quick check. For each file that shares its size with another, the SHA-256 of its first and last 64 KB is computed (for files of up to 128 KB, of the whole file). Most look-alikes differ here.
- Full SHA-256. The large files that passed the quick check are read completely and fingerprinted with SHA-256 (FIPS 180-4) — by the browser’s Web Crypto for files of up to 256 MB, and as a stream in 8 MB pieces (WebAssembly) for larger ones, so a big video is never loaded into memory whole.
Files with the same SHA-256 have the same bytes: a different file with the same fingerprint has never been found. Names, dates and folders play no part, so IMG_1.jpg and copy of holiday.jpg are found to be the same file.
Similar photos
With Also find similar photos ticked, every photo is opened and summarised with the perceptual hashes of the Duplicate Photo Finder: a 64-bit pHash (the DCT-based hash from Christoph Zauner’s thesis on perceptual image hashing) and a 64-bit dHash (Neal Krawetz’s “difference hash”), plus a tiny colour thumbnail. Two photos are similar when the pHash differs in at most 4 (Strict), 10 (Normal) or 16 (Loose) of its 64 bits, the dHash in at most 6 more, and the colours are close. The copy with the most pixels — then the sharpest, then the largest file — is suggested for keeping. Up to 5,000 photos are compared this way in one scan.
What this tool does not do
It never deletes, moves or renames anything: browsers can read the files you choose but should not change your disk without care, and a list you can check is the safe first step. Use the CSV or the copied paths in your file manager. For photos, the Duplicate Photo Finder can move the duplicates into a separate folder or delete them, in Chrome, Edge and Opera on a computer.
Limitations
- Up to 200,000 files per scan. Reading is limited by your disk: a full comparison of many large, same-size files (videos, disk images) takes time.
- Choosing a whole folder works in browsers on computers; most phone browsers can only choose files.
- Similar photos: HEIC photos can be opened only in browsers that support them (Safari), and RAW files in none — those are compared as files only. Rotated, mirrored, heavily cropped or filtered copies are usually not recognised as similar.
- System files and folders (.DS_Store, Thumbs.db, desktop.ini, .git, node_modules …) are skipped unless you untick the option.
Privacy
Everything happens in your browser. What you enter or open here is not uploaded or stored by MySmartCoPilot.
Frequently asked questions
Are my files uploaded?
No. The browser reads the files you choose directly from your disk, and the fingerprints are calculated on your device. Nothing is sent anywhere, and the page works offline once loaded.
Can two different files get the same SHA-256?
In theory yes, in practice no: no two different files with the same SHA-256 fingerprint have ever been found. Files listed together under Identical files have identical contents.
Why are two files with the same name not listed?
Names do not matter here — only contents. Two “report.pdf” files with different contents are different files. Likewise, files with different names are listed together when their bytes are the same.
Can it delete the duplicates for me?
No — it lists them. Download the CSV or copy the paths of the duplicates and remove them in your file manager after checking. For photos, the Duplicate Photo Finder can move or delete duplicates in Chrome, Edge and Opera on a computer.
How do I find duplicates between two folders?
Press Choose a folder once for each folder (or drop both). All the files are compared with each other, so copies in the same folder and across the folders are both found.