Recommended tools for deduplicating files

I recommend these two tools for general deduplication of files:

Czkawa: GUI-based deduplication tool

This open source tool supports both hash-based deduplication (which finds byte-for-byte identical files) and additionally it supports similarity-based image deduplication with visual comparison. It is rather easy to use, so Czkawka is what I recommend everyone to start with - but nevertheless it supports many features that make deduplication efficient.

rmlint - command line deduplication

rmlint is a full-featured command line dedupliation tool which is extremely feature-rich and supports. As far as I know it doesn’t have a GUI, so it’s only for users familiar with the command line. I use it for deduplication on servers etc, and regularly use some of its features like tagged files so that duplicates from some folders will not be deleted:

rmlint -k folder_where_to_remove_files // original_folder