[Upstream] FILEOPEN Word 97-2003 DOC structure as exported by apple pages is not recognized

Problem description:

Steps to reproduce:
Try to open a "Pages" text document that has been exported as .doc-file.

Current behavior:
LibreOffice doesn't recognize it; instead, a window "ASCII Filter Optionen" appears, asking me abaut the font, language etc. of the file.
Expected behavior:

Until August 19 (or 20) - i.e. before I installed the last programm update – LibreOffice had no problem to open doc files exported from Pages.

Platform (if different from the browser):

Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:14.0) Gecko/20100101 Firefox/14.0.1

Thank you for attending for the problem.
Volker

Created attachment 67372
DOC file that can not be viewed in LO

I am attaching problematic DOC file, possibly related by this issue.
PDF with the same content (also attached) states that it was generated using Apple Pages. I tried contacting document author to confirm that DOC was also created using Pages, but I did not receive any reply.

LibreOffice fails to open this file no matter what filter I choose. It just display dozens of pages with rubbish characters. I have tried LO 3.6.1 on Linux and Windows and 3.5.6 on Linux.
MS Word 2007 opens it flawlessly :( .

Created attachment 67373
PDF file with the same content as DOC file, as reference

This issue only started occurring in August 2012. Before then documents could open in LibreOffice.
Same problem with documents being opened from icloud.com files are downloaded as a .doc format but do not open in LibreOffice. Instead they are scrambled with ASCII Filter Options which wreck the entire Document.
This problem also occurs for documents being downloaded as a .xls format from numbers on icloud.com. It does not occur with PowerPoint ppt files.

Ubuntu Linux 12.10 beta 1 64-bit LibreOffice build version 3.6.1.2 build ID: 360m1(Build:2))
ljelly

Confirming that the DOC file opens fine in LO 3.5.4.2 on Linux 32bit

does not open in Libreoffice:
Build Info
tinderbox: administrator: <email address hidden>
tinderbox: buildname: Linux-x86_64_11-Release-Configuration
tinderbox: tree: libreoffice-3-6
tinderbox: pull time 2012-10-09 20:29:29
tinderbox: git sha1s
core:d126dcfe359782a6229156f7cbdb20e642b63a5e

Opening first attachment:
[correctly opens] 3.4.2, 3.5.0 beta3, 3.5.1.2 on Windows XP 32 bit
[wrong opens] 3.5.7, 3.6.3 on RFR 64 bit, 3.6.3 on Windows XP

according to my measurements, it is regression

File does not open on LO 3.6.3.2 (Build ID: 58f22d5) on Linux 64-bit. I can confirm that this same behavior has been happening for me with LO for quite a while on 32 bit also. I am curious that someone got it to read in 3.5, as my Linux copies of LO have been unable to work with my exported Pages docs for about a year.

File opens fine on NeoOffice (Mac) 3.3 patch 2.

File opens fine in Apple Pages 2009.

I didn't send this particular attached file to Google docs, but I sent one I exported from Pages as a Word doc to GD earlier this week and GD can read the exported Pages and iCloud Word files where LO cannot. BTW, downloading one of these files back from GD seems to clean whatever offending content bars opening the original. I can open my GD-downloaded files in LO.

Retracting my "about a year" -- I don't remember LO on Linux ever working for me with these exported docs. I used OO before LO and my recollection is that on Linux it had problems, also.

I've "laundered" the docs using NeoOffice before. My Google docs discovery was very recent.

Hi, I would like to confirm this problem.
Appearently Apple iWork creates DOC files in the 97-2003 binary Word format.
This is the output of the 'file' command:

factuur 20120829.doc: Composite Document File V2 Document, Little Endian, Os: Windows, Version 5.0, Code page: 0

I'm running Libreoffice on Gentoo Linux (I compile it myself).
6 weeks ago I could successfully open this type of documents, but now I can't anymore.

Below is the recent upgrade-history on my desktop machine:

     Fri Aug 31 22:06:02 2012 >>> app-office/libreoffice-3.5.5.3
     Fri Aug 31 22:06:23 2012 >>> app-office/libreoffice-l10n-3.5.5
     Sat Oct 13 17:49:50 2012 >>> app-office/libreoffice-3.5.6.2
     Sat Oct 13 17:50:10 2012 >>> app-office/libreoffice-l10n-3.5.6
     Sat Oct 27 20:02:22 2012 >>> app-office/libreoffice-l10n-3.5.7
     Wed Nov 14 16:19:30 2012 >>> app-office/libreoffice-3.5.7.2

Version 3.5.5.3 could certainly open the 'pages'.
I'm not sure whether I ever opened such a file with version 3.5.6.2
Version 3.5.7.2 doesn't work. When opening a binary 97-2003 Word file the ASCII filter dialog box is presented.

So it appears to be a regression indeed.
I hope this helps.
Regards,
Toon.

Setting version back to the one apparently causing this regression.

Document is still garbled when opened with LOdev version 4.0.0.0.alpha0+ (Build ID: 67fca3), pulled 19 November 2012.

Yesterday over at the #libreoffice irc channel I promised kdjfn to bisect this issue. It now appears that my Ubuntu install (version 10.4) is too old to run the 3.6 bisect. Sorry about that...

*** Bug 57331 has been marked as a duplicate of this bug. ***

Structured storage is not a component where one can allow regressions.

Definitely a regression; I can confirm (for the Mac OS X versions of LibO), that the file still openes correcly with
* 3.5.0
* 3.5.1rc2
* ...
* 3.5.5.3

But not with

* 3.5.6.2
* 3.5.7.1
* 3.6.0.4

So this bug was introduced somewhere in the 3.6 development process (because it already appears in 3.6.0[.4]) and then backported to the 3.5 branch, between the releases of 3.5.5.3 and 3.5.6.2. This is a fairly limited commit range to check.

-> Set the Version picker to the number of the 1st version which is known to contain the bug (3.5.6.2), as usual.

About the Importance field:
I am sorry to say so but AFAIK this kind of bug is not a “blocker” (we use that severity value very very rarely, only if a build does not work at all, etc.), just “critical”.

But this is definitely a good candidate for our “Most annoying bugs” lists, and theses lists are much more important than the Importance field, so I will add this bug there.

This is a regression from http://cgit.freedesktop.org/libreoffice/core/commit/?id=f95762beb3b5849bfaccd39523a11fe15b191d89

Caolan McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ef9f9b750eac2966634c385a2021716c4de0e4cd

Resolves: fdo#53909 STG_FREE sector locations rejected

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.

Caolan McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=e2036d4ec1133c337d1f47334a1a2d558b61dc70

Related: fdo#53909 add regression test

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.

fix submitted for review for 3-6, 3-6-4. 3-5 is a dead tree now

Thanks for so quick fixing this bug

Caolan McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-3-6":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d99c7edff18fb84312928589d3639d0005c23bbe&g=libreoffice-3-6

Resolves: fdo#53909 STG_FREE sector locations rejected

It will be available in LibreOffice 3.6.5.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.

@ Caolán:
Thank you, too, for fixing this annoying issue so fast!

(In reply to comment #18)
> fix submitted for review for 3-6, 3-6-4. 3-5 is a dead tree now

Is 3.5 really completely dead? There are still important bug fixes submitted to that branch, and this one is an inportant bug fix, of course ... If in doubt, please ask Michael Meeks who has, IIRC, advocated more than once submitting important bug fixes to the 3.5 branch. (But correct me if the wind has changed again, and even Michael Meeks does no longer suggest to commit anything to the 3.5 branch!)

Caolan McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-3-6-4":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d3a46db1103fcec615fc9dcb60b13b09c90614a4&g=libreoffice-3-6-4

Resolves: fdo#53909 STG_FREE sector locations rejected

It will be available already in LibreOffice 3.6.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.

Caolan McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-3-5":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4a27bce9c49539c942af27b180ba22f83a12d8e1&g=libreoffice-3-5

Resolves: fdo#53909 STG_FREE sector locations rejected

It will be available in LibreOffice 3.5.8.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.

*** Bug 57535 has been marked as a duplicate of this bug. ***

*** Bug 57671 has been marked as a duplicate of this bug. ***

@ Caolán:

(In reply to comment #23)
> Caolan McNamara committed a patch related to this issue.
> It has been pushed to "libreoffice-3-5": [...]
> It will be available in LibreOffice 3.5.8.

Thank you for pushing this to the 3.5 branch, too!

*** Bug 57689 has been marked as a duplicate of this bug. ***

*** Bug 57787 has been marked as a duplicate of this bug. ***

*** Bug 58071 has been marked as a duplicate of this bug. ***

*** Bug 54985 has been marked as a duplicate of this bug. ***

VERIFIED as FIXED both with

* LibreOffice 3.6.4.3 (Build ID: 2ef5aff)
* Version 4.0.0.0.beta1 (Build ID: 87906242e87d3ddb2ba9827818f2d1416d80cc7)
  TinderBox: MacOSX TDF Release, Branch:libreoffice-4-0,
  Time: 2012-12-05_22:13:37

on Mac OS X (10.6.8, Intel). The sample .doc file now opens correctly.

@ Caolán:
Thank you very much for fixing this bug!

*** Bug 58063 has been marked as a duplicate of this bug. ***

*** Bug 54985 has been marked as a duplicate of this bug. ***