Hi All, Due to some awful archiving. I have around 1500 digital still images that have been duplicated triplicated or more in a number of folders,C/D's and DVD's on a number of drives. I would like to know if there is a way I can dump ALL the directories into one folder then run a "find dupes" type program that would ensure that I did not delete any of my original photos. Regards John
I had a script to do this a while back; used find, md5sum and sort, produced a list of all the files where there was more than one with an identical md5sum, and optionally deleted all but one automatically. That was always a bit of a risky option! Also never got around to making it the oldest one, or the one with the most sensible-looking filename or anything. It was pretty random about which one it chose to keep. I can't find the script now but I could probably hack something up again if nobody knows of any better tools. On 8 March 2010 20:23, John <jaytee(a)clear.net.nz> wrote:
Hi All, Due to some awful archiving. I have around 1500 digital still images that have been duplicated triplicated or more in a number of folders,C/D's and DVD's on a number of drives. I would like to know if there is a way I can dump ALL the directories into one folder then run a "find dupes" type program that would ensure that I did not delete any of my original photos. Regards John
_______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
-- Disclaimer: By sending an email to any of my addresses you are agreeing that: 1. I am by definition, "the intended recipient" 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. 3. I may take the contents as representing the views of your company. 4. This overrides any disclaimer or statement of confidentiality that may be included on your message.
All the photos were taken on the same camera and the common thread is the date and time although some were resized (for email) the originals could be identified by there size. John On Mon, 2010-03-08 at 20:33 +1300, Bruce Kingsbury wrote:
I had a script to do this a while back; used find, md5sum and sort, produced a list of all the files where there was more than one with an identical md5sum, and optionally deleted all but one automatically. That was always a bit of a risky option! Also never got around to making it the oldest one, or the one with the most sensible-looking filename or anything. It was pretty random about which one it chose to keep.
I can't find the script now but I could probably hack something up again if nobody knows of any better tools.
All the photos were taken on the same camera and the common thread is the date and time although some were resized (for email) the originals could be identified by there size.
The following threads list a few programs that do the job of finding duplicates (or similar ones): http://ubuntuforums.org/showthread.php?t=570559 http://www.linuxquestions.org/questions/linux-newbie-8/i-need-a-command-that... Cheers, Peter -- Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ http://www.cs.waikato.ac.nz/~fracpete/ Ph. +64 (7) 858-5174
doesn't cp have an overwrite-if-larger option? On Mon, Mar 8, 2010 at 9:31 PM, Peter Reutemann <fracpete(a)waikato.ac.nz>wrote:
All the photos were taken on the same camera and the common thread is the date and time although some were resized (for email) the originals could be identified by there size.
The following threads list a few programs that do the job of finding duplicates (or similar ones): http://ubuntuforums.org/showthread.php?t=570559
http://www.linuxquestions.org/questions/linux-newbie-8/i-need-a-command-that...
Cheers, Peter -- Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ http://www.cs.waikato.ac.nz/~fracpete/ Ph. +64 (7) 858-5174 _______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
Thank you Peter...
The following threads list a few programs that do the job of finding duplicates (or similar ones): http://ubuntuforums.org/showthread.php?t=570559 http://www.linuxquestions.org/questions/linux-newbie-8/i-need-a-command-that...
Cheers, Peter
GQView has a very good duplicate checking and deletion feature and afaik it is installed by default in ubuntu It has options to check via checksum, size, date and also image similarity and can check them in place by simply adding all images into a single collection Regards Jason John wrote:
Hi All, Due to some awful archiving. I have around 1500 digital still images that have been duplicated triplicated or more in a number of folders,C/D's and DVD's on a number of drives. I would like to know if there is a way I can dump ALL the directories into one folder then run a "find dupes" type program that would ensure that I did not delete any of my original photos. Regards John
_______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
It was not a default on my system but I have now installed it thanks. Regards John On Mon, 2010-03-08 at 23:17 +1300, Jason Drake wrote:
GQView has a very good duplicate checking and deletion feature and afaik it is installed by default in ubuntu ___________________________________________
wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
participants (5)
-
Bruce Kingsbury -
Darryn Brooking -
Jason Drake -
John -
Peter Reutemann