Data corruption with eglibc 2.17

Bug #1179281 reported by Nikolaus Rath
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
s3ql (Debian)
Fix Released
Unknown
s3ql (Ubuntu)
Fix Released
Undecided
Unassigned
Raring
Fix Released
High
Unassigned

Bug Description

[Impact]

The S3QL package in raring destroys user data.

There is a bug in src/s3ql/_deltadump.pyx, which repositions a file descriptor associated with a FILE* stream without first calling
fflush(). When using eglibc 2.17, this means that s3ql will store malformed metadata that cannot be loaded again (with prior eglibc versions the bug does not appear, which is probably why it hasn't been found until now).

The problem is made worse because this affects only the metadata uploaded to the remote server, but not the locally cached copy. So unless the user explicitly tests reloading the metadata from the server, the corruption will not show up until it's too late.

The attached patch fixes the problem by adding the missing fflush() call.

[Test Case]

 * The bug is detected by S3QL unit tests, so running "setup.py test" will fail with

$ ./setup.py test
[...]
======================================================================
ERROR: test_1_vals_1 (__main__.DumpTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "t1_dump.py", line 36, in test_1_vals_1
    self.compare_tables(self.src, self.dst)
  File "t1_dump.py", line 155, in compare_tables
    (id2, buf2) = i2.next()
  File "/usr/lib/s3ql/s3ql/database.py", line 250, in next
    for col in self.cur.next() ]
StopIteration

The unit tests are run automatically by debian/rules. This bug only made it into the release because (presumably) S3QL was build before eglibc was updated to 2.17.

[Regression Potential]

 * The patch effects the writing and reading of data from a binary dump file into an SQLite database. Any regression would thus be restricted to the initial reading, or final dumping of the data. While this makes testing easier, it also means that any regressions will probably make the entire program unusable.

[Other Info]

 * The same patch has already been applied to the S3QL 1.11 debian package.

Nikolaus Rath (nikratio)
description: updated
description: updated
Revision history for this message
Nikolaus Rath (nikratio) wrote :
Nikolaus Rath (nikratio)
description: updated
Julian Taylor (jtaylor)
Changed in s3ql (Ubuntu):
status: New → Fix Released
Changed in s3ql (Ubuntu Raring):
assignee: nobody → Julian Taylor (jtaylor)
Revision history for this message
Julian Taylor (jtaylor) wrote :

I have uploaded the fix, thanks for the patch

Changed in s3ql (Ubuntu Raring):
assignee: Julian Taylor (jtaylor) → nobody
Revision history for this message
Nikolaus Rath (nikratio) wrote :

Thanks Julian!

Relevant upstream bug is http://code.google.com/p/s3ql/issues/detail?id=399

Changed in s3ql (Debian):
status: Unknown → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Nikolaus, or anyone else affected,

Accepted s3ql into raring-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/s3ql/1.12-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in s3ql (Ubuntu Raring):
status: New → Fix Committed
tags: added: verification-needed
Changed in s3ql (Ubuntu Raring):
importance: Undecided → High
Revision history for this message
Nikolaus Rath (nikratio) wrote :

Tested s3ql_1.12-1ubuntu0.1 on amd64

tags: added: verification-done
removed: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s3ql - 1.12-1ubuntu0.1

---------------
s3ql (1.12-1ubuntu0.1) raring-proposed; urgency=low

  * Call fflush() on FILE stream before repositioning underlying
    file descriptor. LP: #1179281.
 -- Nikolaus Rath <email address hidden> Sun, 24 Feb 2013 18:51:42 -0800

Changed in s3ql (Ubuntu Raring):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.