IndexError on startup after external changes to @shadow files

On Tue, Apr 20, 2010 at 8:43 AM, Antti Kaihola
<email address hidden> wrote:
> Public bug reported:
>
> I raised this issue earlier today on the mailing list as well.
>
> I created and edited a number of @shadow entries in my .leo files. After
> closing Leo, I committed some changes into my Git repository and rolled
> back some whitespace-only changes. The next time I tried to open Leo
> with the .leo file I got this:

Thanks for this report. I'll look into immediately.

Edward

I had another file do the same and noticed the external changes had appended a space without a newline to the file. Here's a hexdump of the tail of the file:

000011e0 74 69 66 69 65 72 27 29 0a 20 20 20 20 20 20 20 |tifier'). |
000011f0 20 0a 0a 0a 20 | ... |

After truncating the space away from the file, Leo opens fine.

I'm not sure if the first file that caused this problem had similar characteristics.

On Wed, Apr 21, 2010 at 1:31 AM, Antti Kaihola
<email address hidden> wrote:
> I had another file do the same and noticed the external changes had
> appended a space without a newline to the file.

Let's try a simple-minded guard. Change sourcereader.get to:

def get (self):

    trace = False and not g.unitTesting

    if self.i < len(self.lines):
        result = self.lines[self.i]
        self.i+=1
    else:
        result = ''

    if trace: g.trace(repr(result))
    return result

Does this work for you?

Edward

Yes, now I get past the error, but the file appears corrupted in Leo.

The file I currently have this problem with has 5 sub-nodes. After saving the file, I had closed Leo and reverted Leo's removal of whitespace on blank lines. When re-opening:
- the first node is ok
- the second node contains contents of original nodes 2 & 3
- the third node contains original nodes 4 & 5 plus three extra blank lines and the last two lines of node 2 (or node 3 -- the nodes are almost identical)
- the fourth node is ok (contains original node 4)
- the fifth node is ok (contains original node 5)

I also get a dump of the imported file on the console with this message:

    @shadow did not pick up the external changes correctly

The file ends with a single newline (0x0a) both before and after saving with Leo, so it seems that missing final newlines are not the (only) cause for this symptom. Instead, the addition of whitespace on blank lines seems to cause trouble.

Might the similarity of nodes 2 & 3 confuse Leo in this case? They both contain 17 lines, and only their two initial lines differ (ignoring whitespace on blank lines).

Unfortunately I'm not in a position to force whitespace clean-up on tools used by other members of the team. To ease collaboration and creating clean patches, I'd really like Leo to *not* clean up whitespace, but I haven't found such settings.

I can work around Leo's clean-up by automatically excluding whitespace-only changes from my git commits. But I can't think of an easy way to re-apply them on my working tree after the commit, so fixing this bug would help me a lot. I can try to create further minimal test cases if that helps.

I really like the flexibility Leo gives me and I hope I can contribute to its development!

On Tue, Apr 27, 2010 at 2:23 AM, Antti Kaihola
<email address hidden> wrote:
> Yes, now I get past the error, but the file appears corrupted in Leo.

Could you send me a small .leo file and the corresponding
public/private files illustrating the problem? Thanks.

> I really like the flexibility Leo gives me and I hope I can contribute to its development!

Helping me fix this bug will be a good start :-)

Edward

Yikes. This bug has been around for 6 months. My apologies for the delay in fixing this important bug. I'll fix this next.

The fix is on the trunk at rev 3537.

This wasn't really a critical bug after the trivial index error crasher was fixed.

From the checkin log:

QQQQQ
- Made **no** further changes to the fundamental algorithm.

- Made several changes to the import checks:

1. Ignore trailing ws on every line, and trailing blank lines.

2. Use difflib.Differ to report problems.

Admittedly, ignoring trailing whitespace differences isn't "perfect",
but if this isn't good enough then you should use @auto or @thin.
QQQQQ

I'll mark this bug closed (fix released), but feel free to reopen it if you find errors that corrupt files in any way excluding whitespace.

Edward