Deduplication Libraries

This package provides functionality to make use of hashing algorithms that are particularly good at finding exact duplicates as well as convolutional neural networks which are also adept at finding n…

rmlint finds space waste and other broken things on your filesystem and offers to remove it. Finds… It runs and compiles under most Unices, including Linux, FreeBSD and Darwin. The main target is Lin…

Upgrade pip In the following usage example the first two and the fourth instance of VideoHash class are computing the hash for the same video(not same as in checksum) and the third one is a different…

In the first phase the elasticsearch backend is checked against the current filesystem state, cleaning up database entries of files that no longer exist. This will speed up queries made later on. Alt…

Tool for managing data-deduplication within extant compressed archive files, with a heavy focus on Manga/Comic-book archive files. It maintains a database of hashes for all files it scans, and actual…

There's gonna be a lot of duplicates in almost any arbitrary collection ofimages, and it can actually be surprising how many. pHash lib, which is the core of the tool, easily detects cropped and reto…