[git] file blocks duplicated at the end of the file

Another file got corrupted. dmesg shows this:

[55148.424563] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
[55148.424568] ecryptfs_readpage: Error decrypting page; rc = [-4]
[55148.424579] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
[55148.424583] ecryptfs_readpage: Error decrypting page; rc = [-4]

For ways to reproduce see Bug #490005

I found out howto reproduce this on my system:

Doing a "git status -uno" and aborting (ctrl+c) give you a high chance of a corrupt file afterwards.

Hi I upgraded to 11.10 and the bug is still somewhat there:

iroli@iroli-thinkpad:~/projects/c3pro/repo/foo_git/trunk$ git status -uno
# On branch matcher_gen_casm
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: app/compiler/src/misc/some_stuff.cpp
# modified: app/csim/check/exit_status/args.xml <!-- This one was not modified!
# modified: arch/aaa/mem_address_writeback_instructions.xml
# modified: arch/aaa/minstr_set_register_interface.xml
# modified: arch/aaa/minstr_set_stale.xml
#
no changes added to commit (use "git add" and/or "git commit -a")

iroli@iroli-thinkpad:~/projects/c3pro/repo/foo_git/trunk$ git diff app/csim/check/exit_status/args.xml
error: short read No such file or directory
error: app/csim/check/exit_status/args.xml: failed to insert into database
fatal: cannot hash app/csim/check/exit_status/args.xml

iroli@iroli-thinkpad:~/projects/c3pro/repo/foo_git/trunk$ git status -uno
# On branch matcher_gen_casm
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: app/compiler/src/misc/some_stuff.cpp
# modified: arch/aaa/mem_address_writeback_instructions.xml <!-- no the file is not marked as modified!
# modified: arch/aaa/minstr_set_register_interface.xml
# modified: arch/aaa/minstr_set_stale.xml
#
no changes added to commit (use "git add" and/or "git commit -a")

iroli@iroli-thinkpad:~/projects/c3pro/repo/foo_git/trunk$

Tyler, sending this bug your way, hoping you can add some comments.

just ran into the short read again

Thanks - I can reproduce this and have tracked down the problem. We're missing an eCryptfs metadata read, in the ecryptfs_getattr() path, after a metadata read in the lookup() path fails. This results in a stat() call returning an incorrect inode size and then subsequent read()'s not adding up to the correct file size.

I've started on a fix, but will need a bit more time, as this code is fragile and needs improvements outside of this bug.

Confirmed with current 3.2-rc7 kernel.

Unfortunately, I'm not able to consistently reproduce this anymore. I can trigger it once every so often, but not consistent enough to give me much confidence. I've developed a proposed patch and pushed it to a temporary location here:

http://git.kernel.org/?p=linux/kernel/git/tyhicks/ecryptfs.git;a=commitdiff;h=4b08d9cf379be83641b8d7e0eea56e14a8e20cc3

I'm in the process of building test kernels for Precise and 11.10 in hopes that I can get some testing from iroli and others affected before I push the fix upstream. I'll provide links to the test kernels when they're done building.

The patch I linked to in comment 10 breaks plaintext passthrough support. The logic around handling plaintext passthrough and the metadata-in-xattrs feature is very delicate in this part of the code. I'd love to deprecate/remove those two "features" in future kernels, as they're rarely used by anyone, but we need a fix that is backportable.

I'm going to think this through a little more and I hope to have a new patch this later this evening.

Plaintext passthrough has been broken by design for a very long time,
and practically impossible to support.

I'm in favor of deprecating the current implementation and hopefully
replacing it with a better approach eventually.

Proposed fix against the current upstream kernel (3.4-rc0).

I'm still not able to consistently reproduce this issue anymore. I've built amd64 test kernels for Oneiric and Precise. For those affected, please give the kernels a try and leave a comment as to whether or not the bug is fixed.

http://people.canonical.com/~tyhicks/ecryptfs/842647/

The attachment "[PATCH] eCryptfs: Ensure i_size is set in ecryptfs_getattr()" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

I still get this [1] running 3.10rc7 without using git. Is there a way to find out which files are/got corrupted ?

[1] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
 ecryptfs_readpage: Error decrypting page; rc = [-4]

3.11.0-13-generic (i386), same problem.

I can confirm that the following error message still happens on 3.11.0-19-generic (x86_64)

[66675.577535] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
[66675.577540] ecryptfs_readpage: Error decrypting page; rc = [-4]

My workload is mostly git based

It looks like the same problem is present also in Ubuntu 14.04 LTS. After stating a 190k files with `find` and `grep`, the kernel log is full of

    ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
    ecryptfs_readpage: Error decrypting page; rc = [-4]

Couldn't these printk from ecryptfs be augmented with some additional information? At least to make sure that we are all hitting the same bug.

Any news? The oldest bug insofar.

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release