How to find duplicate files on a Mac

Photos: duplicate photos and videos

1. Open Photos and click Duplicates below Utilities in the sidebar. If you do not see it, hold the pointer over Utilities and click the arrow.
2. If you use iCloud Shared Photo Library, choose Both Libraries from the Library pop-up menu in the toolbar.
3. Select the duplicates to merge: one row, several rows, or Command-A for all of them.
4. Click Merge, then confirm.

One photo stays where the duplicates were, and the others go to Recently Deleted, where they can be recovered for 30 days. On a large library the Duplicates collection can take a while to fill, because Photos has to analyse the photos first. Never delete photos from the library in Finder; see the Photos library is taking up space.

Finder: search by name

Finder has no duplicate search, but it can put likely copies side by side:

1. Open a Finder window, type a word from the file's name in the search field and press Return.
2. Click the Add button below the search field to narrow it, for example by Kind.
3. Switch to list view and sort by Size.

Two files with the same size and a similar name are worth comparing; the same size alone proves nothing, and two copies of a file can have quite different names. Click Save to keep the search as a Smart Folder, which updates itself.

Terminal: check two files are identical

shasum -a 256 ~/Downloads/report.pdf ~/Documents/report.pdf

shasum prints a fingerprint of each file's contents. The same fingerprint means the same contents, whatever the names say. It only reads the files.

Terminal: every duplicate in a folder

find ~/Downloads -type f -size +1M -exec shasum -a 256 {} + | sort | awk '{h=$1; sub(/^[^ ]+  /,""); if (h==p) {if (!shown) {print ""; print pf} print; shown=1} else shown=0; p=h; pf=$0}'

This fingerprints every file over 1 MB in Downloads and prints the identical ones in groups, separated by a blank line. Change ~/Downloads to another folder to search there. It reads every file, so a large folder takes a while, and it deletes nothing. macOS may ask whether Terminal can access the folder; allow it, or those files are skipped. Do not point it at your Library or into the Photos library: identical files there belong to apps, which expect each copy where it is.

What a safe duplicate finder does

• Matches on contents, not names. A renamed copy is still a copy, and two different files can share a name.
• Never offers the last copy. One of each set is kept and is not something you can tick.
• Says which copy it keeps, and why, so you can disagree before anything goes.
• Leaves libraries closed. A photo library or an app package is one thing to the app that owns it; its insides are not loose files.
• Moves to the Trash rather than deleting outright, so File → Put Back still works until you empty it.

Where the app comes in

RefreshYourMac's Duplicate Files finds files over 1 MB whose contents match byte for byte in Downloads, Desktop, Documents, Movies, Pictures and Music, matched on size, then the first 64 KB, then a hash of the whole file, never on the name. One copy of every file is kept and never listed, so cleaning everything shown cannot remove the last copy; it keeps a copy filed outside Downloads, otherwise the oldest, and each row says which one it keeps. Two hard links to one file are not counted, and photo libraries and other packages are left closed. Its Duplicate Pictures finds the same picture saved again at another size or in another format, which a byte match cannot see, and keeps the copy with the most pixels; it is not a similar-photos finder. Both move what you clean to the Trash and are rated Review before deleting. In the Mac App Store edition Desktop and Documents are searched once you allow them in Settings → Folders. The download edition also has Identical Copies in App Data, which turns spare copies inside app folders into APFS clones, so every file stays where its app expects it. For bursts and near-identical shots, Folio Refresh, a separate Caddora app, scans the Photos library and folders and moves nothing until you approve it.