Export doesn't deal with different files with same filename

Bug #648424 reported by Michael Hendry
34
This bug affects 6 people
Affects Status Importance Assigned to Milestone
One Hundred Papercuts
New
Undecided
Unassigned
Shotwell
New
Unknown
shotwell (Ubuntu)
Triaged
Low
Unassigned

Bug Description

Binary package hint: shotwell

I store scanned 35mm film images as IMG001.jpg, IMG002.jpg etc, where the digits indicate the negative's number. Each film has its own directory (e.g 1960-1, 1960-2 etc.

I'm collecting together a presentation for a camera club by tagging the images I want, and then exporting them, but some of these have identical filenames, for example, I have at least two files called IMG008.jpg. The outcome is that only one of the files is exported, and there is no warning nor offer of systematic renaming of duplicate files on export.

This is not just going to affect me (because of my method of naming scanned negatives), but will also affect those who choose to have their digital cameras start the numbering of their images at 1 again after all the images for a session have been removed from the camera.

This is going to make the preparation of this presentation (in three days time!) very tiresome, because I can't rely on having all the images I've tagged exported to the directory I'm using for the presentation manager.

In the short term I need a workaround short of renaming all my files. I think I'll have to spot the missing files by comparison of the export directory with the display of all the files tagged "CamClub", and to tag these CamClub2, export these to a separate export2 directory, and so on with Camclub3 and export 3 until the files are done. I can then modify the exported filenames from export2, export3 etc, before collecting all the files into one directory.

Revision history for this message
Jim Nelson (yorba-jim) wrote :

Yes, I can see how this would be a problem with your workflow. I've ticketed the immediate problem here: http://trac.yorba.org/ticket/2608

Changed in shotwell:
status: Unknown → New
Revision history for this message
Michael Hendry (hendry-michael) wrote :

One way of dealing with the problem of duplicate filenames would be to give the exported files new names, e.g. Export001.jpg, Export002.jpg etc.

As you'll see above, I'm preparing a presentation from my (recently imported) archive of 21000 images. Once I've exported the images I use an add-in to OpenOffice Impress called PhotoAlbum to import the images into a slide show. The images are imported in alphabetical order of filename in one simple operation, with dimensions adjusted to fill the slide. I then have to sort the images into the appropriate order for the slideshow.

Inevitably, I find I want to insert some other images into the show, and these have to be done "by hand", with tedious hand-resizing of the new images, one-by-one.

Ideally, I'd like to be able to collect and sort the images within Shotwell, and then export them with new filenames which would preserve this order when they're imported into Impress.

Let's say I've tagged ImageA.jpg, ImageB.jpg and ImageC,jpg for export, but I want them to be in the slideshow in reverse order. In my proposed new feature for Shotwell, I'd drag and drop them on screen into the order ImageC.jpg, ImageB.jpg and ImageA.jpg and then export them as Export001.jpg, Export002.jpg and Export003.jpg respectively.

The existing Slideshow feature within Shotwell goes part of the way, but doesn't allow for changes in order, and the images have to be linked together as an event or with a tag.

Revision history for this message
Rodrigo Virote Kassick (kassick) wrote :

I needed to export some photos, so I made a patch for this one.

It renames files to it's timestamp (ditized time), with a sequential number if two photos have the same timestamp.

Revision history for this message
Jim Nelson (yorba-jim) wrote :

Rodrigo,

Can you update to trunk and re-submit the patch? It looks like this patch was made against 0.7.2.

You can pull from trunk using this command:

$ svn co svn://svn.yorba.org/shotwell/trunk shotwell

Thanks.

Omer Akram (om26er)
Changed in shotwell (Ubuntu):
importance: Undecided → Low
status: New → Triaged
tags: added: patch
Revision history for this message
Michael Hendry (hendry-michael) wrote : Re: [Bug 648424] Re: Export doesn't deal with different files with same filename

On Wed, 2010-11-10 at 17:39 +0000, Brian Murray wrote:
> ** Tags added: patch
>

Thanks, Brian.

I'm not sure if this means that you've created a patch or that you plan
to do so - please clarify, and if it's been created tell me where to
find it.

Michael

Revision history for this message
Jim Nelson (yorba-jim) wrote :

I think Brian was merely grooming the ticket and marking that a patch was available (from Rodrigo). However, that patch is against an older version of Shotwell and I'd need it to be update to work with trunk to review it.

Also, I'm not sure the patch's strategy is what we want -- it might generate unique filenames, but it also loses the original's filename in the process.

Revision history for this message
Rodrigo Virote Kassick (kassick) wrote :

Em 10-11-2010 20:52, Jim Nelson escreveu:
> I think Brian was merely grooming the ticket and marking that a patch
> was available (from Rodrigo). However, that patch is against an older
> version of Shotwell and I'd need it to be update to work with trunk to
> review it.
>
> Also, I'm not sure the patch's strategy is what we want -- it might
> generate unique filenames, but it also loses the original's filename in
> the process.
>

Indeed, I had that issue in mind. But then what would be a good file
name in this case? Photos imported by shotwell and f-spot are named as a
sequential number.

Exporting a selection of photos (that may or may not be under a same tag
in shotwell) will create a directory with a bunch of files whose numbers
are mostly meaningless numbers.

Sugestions?

Revision history for this message
Michael Hendry (hendry-michael) wrote :

On Thu, 2010-11-11 at 00:01 +0000, Rodrigo Virote Kassick wrote:
> Em 10-11-2010 20:52, Jim Nelson escreveu:
> > I think Brian was merely grooming the ticket and marking that a patch
> > was available (from Rodrigo). However, that patch is against an older
> > version of Shotwell and I'd need it to be update to work with trunk to
> > review it.
> >
> > Also, I'm not sure the patch's strategy is what we want -- it might
> > generate unique filenames, but it also loses the original's filename in
> > the process.
> >
>
> Indeed, I had that issue in mind. But then what would be a good file
> name in this case? Photos imported by shotwell and f-spot are named as a
> sequential number.
>
> Exporting a selection of photos (that may or may not be under a same tag
> in shotwell) will create a directory with a bunch of files whose numbers
> are mostly meaningless numbers.
>
> Sugestions?
>

Suggestion 1:

Number the files sequentially, in the order in which they appear on the
screen:

Export0001.jpg
Export0002.jpg
Export0003.jpg
...

The number of leading zeroes could be adjusted according to the number
of files that are to be exported, the idea being that the exported files
will appear in a directory listing in the same order, and can be
imported into a presentation manager in this order too.

Suggestion 2:

Number the files as above, but append the original filename:

Export0001-IMAGE006.jpg
Export0002-DSC0999.jpg
Export0003-Jane's Wedding 6.jpg
...

This has the advantage of making it easier to track back to the original
image while keeping the chosen sort order.

Suggestion 3:

As above, but insert the file creation date and time:

Export0001-2004_10_06_12_03-IMAGE006.jpg
Export0002-1966_05_04_16_23-DSC0999.jpg
...

This would mean that a relative to whom I'd sent a selection of
(compressed) images of a family event could ask for a hi-resolution
print of a particular image using an unambiguous filename.

I'd like to be able to override the default filename prefix (e.g.
replace "Export" with "Xmas2007" or "Jim's60th") - indeed the default
filename prefix could be the tag by which the export list has been
selected, if this is the case, although I can see that this might cause
problems if future developments in Shotwell allow selection by some
logical combination of tags!

I'd also like to be able (as indicated in an earlier post) to
drag-and-drop the images into the order in which I want them to be
numbered on export.

Michael

Revision history for this message
Jim Nelson (yorba-jim) wrote :

I think there's any number of ways we can go about doing this. Michael's ideas are great, but they involve a broader idea of exporting photos -- being able to customize names and ordering and numbering, almost like generating a report. That's a scheme far larger than this ticket.

The problem here is when you export two photos with the same filename, the second one clobbers the first. That's obviously not right. What if we did something super-simple: When going to export the file, if the filename exists, append an underscore and a numeral and see if that file exists. If so, keep appending numbers until one is found.

With this scheme you would get files like this:

IMG_0123.JPG
IMG_0123_1.JPG

Not as powerful as Michael's idea, but it will work, and preserves the original filename (which is sometimes human-readable).

As it turns out, we have code in Shotwell that does exactly this (we use it for import). In file_util.vala is a function generate_unique_file(). The function returns a File object with a unique name (using the original if possible).

This, I think, is the easiest way to go to solve this ticket. Michael, I think your ideas are interesting, as always. If you'd like to propose a new feature, I encourage you to file a ticket fleshing this out on our Trac server: http://trac.yorba.org.

-- Jim

Revision history for this message
Michael Hendry (hendry-michael) wrote :

On Fri, 2010-11-12 at 20:31 +0000, Jim Nelson wrote:
> I think there's any number of ways we can go about doing this.
> Michael's ideas are great, but they involve a broader idea of exporting
> photos -- being able to customize names and ordering and numbering,
> almost like generating a report. That's a scheme far larger than this
> ticket.
>
> The problem here is when you export two photos with the same filename,
> the second one clobbers the first. That's obviously not right. What if
> we did something super-simple: When going to export the file, if the
> filename exists, append an underscore and a numeral and see if that file
> exists. If so, keep appending numbers until one is found.
>
> With this scheme you would get files like this:
>
> IMG_0123.JPG
> IMG_0123_1.JPG
>
> Not as powerful as Michael's idea, but it will work, and preserves the
> original filename (which is sometimes human-readable).
>
> As it turns out, we have code in Shotwell that does exactly this (we use
> it for import). In file_util.vala is a function generate_unique_file().
> The function returns a File object with a unique name (using the
> original if possible).

Thanks, Jim. Suffixing a serial number in the event of a duplicate
filename would certainly have solved my initial problem in a way I could
live with.

>
> This, I think, is the easiest way to go to solve this ticket. Michael,
> I think your ideas are interesting, as always. If you'd like to propose
> a new feature, I encourage you to file a ticket fleshing this out on our
> Trac server: http://trac.yorba.org.

As a recent convert to Linux from Windows, I find the idea of being able
to have a direct effect on the development of a product is very strange
and not a little frightening! I'll certainly have a look down this
avenue.

Michael

tags: added: patch-needswork
removed: patch
Revision history for this message
Michael Hendry (hendry-michael) wrote :

On Tue, 2011-04-05 at 12:00 +0000, Sebastien Bacher wrote:
> ** Tags added: patch-needswork
> ** Tags removed: patch
>

I'm not sure what this change of status means - I can confirm that this
bug is still present in Shotwell 0.9.0.

Michael

Revision history for this message
maksymov.vlad (maksymov.vlad) wrote :

same bug 0.12.2

Revision history for this message
Michael Hendry (hendry-michael) wrote :

And again in 0.12.3.

I rarely use the export feature, and I'd forgotten about this bug until it bit me again today, as I collected together a slideshow of winning entries from our Rotary Club's camera club for a presentation.

The image for each competition are stored in a single directory, and because the filenames are constructed from the members' surnames, with places in the competition indicated by a prefixed digit, there is a lot of duplication of filenames. Modesty forbids me from mentioning my own successes, but one of our members is an Associate of the Royal Photographic Society, and he has a lot of "1bowen.jpg" entries!

I didn't file a request-for-change ticket last time - if I'm not side-tracked again I'll do it this time.

Revision history for this message
Jim Nelson (yorba-jim) wrote :

Michael, I've marked this as a candiate for 0.14. We have a lot of requests for changes, so no promises, but I hope we can get this in when we start work on the next release.

Changed in hundredpapercuts:
milestone: none → papercuts-s-shotwell
Revision history for this message
Ian Young (youngian) wrote :

Still present in 0.18.1. I discovered that the "Send To..." option actually handles this correctly, so a workaround is choosing that option, closing the mail window and pulling the generated files out of `/tmp`. Still, would be much, much nicer if Export worked the same way instead of silently losing photos.

Revision history for this message
Vinod (vinod-kurup) wrote :

Thanks for that workaround Ian. It was helpful for me.

Revision history for this message
Roger Greenwood (rogerggbr) wrote :

This one just bit me as well - would appreciate a fix. Exported >700 pictures but only found about 600 without any warning. Noted the workaround from Ian Young (thanks!) so trying that next. Version 0.22.0 for info.

Revision history for this message
mirak (mirak-mirak) wrote :

Same issue for me.

Revision history for this message
Simon Naunton (snaunton) wrote :

I have the same issue.

I wrote a bash script as a workaround (and convert videos to MP4, date them and archive the original- we have a lot of AVIs from ~10 years ago), posting here in case it is helpful to anyone.

https://github.com/snaunton/shotwell-export

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.