EC: etag of range request is not consistent

Bug #1496234 reported by clayg
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Low
Mahati Chamarthy

Bug Description

I think the etag on some range requests is coming off the backend headers directly, maybe not being translated in the proxies "fixup backend ec headers" dance.

I added an assertion to an existing functest [1] and it works on repl but fails on ec [2].

1. https://gist.github.com/clayg/318ca1602a00e2ee33cd
2. i.e SWIFT_TEST_POLICY=[repl|ec] nosetests swift/test/functional/tests.py:TestFile.testRangedGets -vsx

Bill Huber (wbhuber)
Changed in swift:
status: New → Confirmed
Revision history for this message
Mahati Chamarthy (mahati-chamarthy) wrote :

reversing the parameters in the assert: https://gist.github.com/MahatiC/912eb17858a0b1505ef0 seems to be working for the test. Maybe it has to do with conn.response.read happening before?

Revision history for this message
clayg (clay-gerrard) wrote :

uploading a sufficiently small object name "foo" into an ec storage policy named "test" can demonstrate the error on 416 responses with curl:

curl ${OS_STORAGE_URL}/test/foo -v -H "x-auth-token: ${OS_AUTH_TOKEN}" -H 'Range: bytes=200000-300000'
...
< HTTP/1.1 416 Requested Range Not Satisfiable
< Content-Length: 97
< Content-Type: application/octet-stream
< Accept-Ranges: bytes
< Last-Modified: Wed, 24 Feb 2016 04:50:46 GMT
< Connection: close
< Etag: "3b318e854dcebd5b842fac53917969b2"
< X-Timestamp: 1456289445.50746
< Date: Wed, 24 Feb 2016 04:54:27 GMT
< X-Object-Meta-Mtime: 1428171700.000000
< X-Trans-Id: txaebc390e31104775be961-0056cd3783
...

the Etag of the object is not "3b318e854dcebd5b842fac53917969b2" it's a79dd7430be6111124e20ba304e11798

Changed in swift:
assignee: nobody → Mahati (mahati-chamarthy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/290369

Changed in swift:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on swift (master)

Change abandoned by Mahati Chamarthy (<email address hidden>) on branch: master
Review: https://review.openstack.org/290369
Reason: Abaonding because https://review.openstack.org/#/c/293633 works as an alternative.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/293633
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=12dd408823df158359e99fb01716f2059140c5c9
Submitter: Jenkins
Branch: master

commit 12dd408823df158359e99fb01716f2059140c5c9
Author: Alistair Coles <email address hidden>
Date: Wed Mar 16 17:41:30 2016 +0000

    Put correct Etag and Accept-Ranges in EC 304 and 416 responses

    When using an EC policy, 304 responses to conditional GETs
    are missing the Accept-Ranges header and have the wrong ETag
    value. 412 responses also have the wrong etag.

    416 responses to ranged GETs also have the wrong ETag.

    This patch ensures behaviour with EC policy is consistent
    with replication policy:

      - 304 and 416 responses have correct etag and Accept-Ranges
      - 412 responses have correct Etag but no Accept-Ranges

    Co-Authored-By: Mahati Chamarthy <email address hidden>
    Co-Authored-By: Thiago da Silva <email address hidden>

    Closes-Bug: #1496234
    Closes-Bug: #1558197
    Closes-Bug: #1558193
    Change-Id: Ic21317b9e4f632f0751133a3383eb5487379e11f

Changed in swift:
status: In Progress → Fix Released
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/swift 2.7.0

This issue was fixed in the openstack/swift 2.7.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/crypto)

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/299944

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (feature/crypto)
Download full text (37.8 KiB)

Reviewed: https://review.openstack.org/299944
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=77c181161a029ba8cea5d1ce65f979fe4b23ff37
Submitter: Jenkins
Branch: feature/crypto

commit 59bbe27fb0a40236108f09c9b3349e8faef0a95c
Author: Nguyen Hung Phuong <email address hidden>
Date: Wed Mar 30 11:07:46 2016 +0700

    Fix typos in Swift files

    Change-Id: I34e0c9a888127704ac1910e73ddd14e27ebade13

commit 7be55acf1bc4aa07d81b30fd93e144700889898d
Author: Tim Burke <email address hidden>
Date: Thu Feb 11 16:00:38 2016 -0800

    Simplify policy-name validation slightly

    _validate_policy_name always either returns True or raises an exception.
    Simplify it to just being a callable that may raise an exception.

    Also, move the check for blank/None names into _validate_policy_name, so
    it will be applied in more cases.

    Change-Id: I7832a0c9c895cd75ba4c6d0e8b5568a3c8a0ea25

commit 5902015fa8495ec0ef3c1ab92ae9a34c5bda4334
Author: OpenStack Proposal Bot <email address hidden>
Date: Sat Mar 26 06:35:18 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I3b5d401649fa3dea6dc43654516f7075bb06ee0d

commit 2f7d0f4a2ad2da7e6a35e5b054a47a2fafe5ed01
Author: Anh Tran <email address hidden>
Date: Fri Mar 25 11:44:26 2016 +0700

    Removing some redundant words

    This patch removes some redundant words.

    Change-Id: Ia79717664b06ed9a41c3c5dcf1a25e9e49e21cf2

commit 925546ae8a211b50cf7fad6634d47fd1dbfeb58e
Author: OpenStack Proposal Bot <email address hidden>
Date: Fri Mar 25 06:36:40 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I6ba2f35913e6ae83607b5e268645432d455d587c

commit 3407d737c705a7afedeed0159588ab4433a601f3
Author: David Liu <email address hidden>
Date: Thu Mar 24 16:08:19 2016 +0800

    Handle tempurl Content-Disposition header missing from HEAD

    Content-Disposition headers should make no difference between
    GET and HEAD according to HTTP rfc.

    Closes-Bug: #1539805

    Change-Id: Ifa41a7cda2f321eb8e36420ede7912ed0a549712

commit 2f24fb9683a57b67348d65864d5af8c3a03dee67
Author: Alistair Coles <email address hidden>
Date: Wed Mar 23 20:49:50 2016 +0000

    Check marker params in SimpleClient full listing requests

    Follow up for change [1] to add some assertions to check that
    marker param is included in sequential GET requests sent during
    a full listing.

    Extract multiple FakeConn class definitions to single class at
    module level and share between all classes.

    Also, explicitly unpack the return values from base request calls
    made in the full listing section of base_request, and explicitly
    return a list to make more consistent with rest of the method.

    [1] Change-Id: I6892390d72f70f1bc519b482d4f72603e1570163

    Change-Id: Iad038709f46364b8324d25ac79be4317add79df...

tags: added: in-feature-crypto
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/hummingbird)

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/323599

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (feature/hummingbird)
Download full text (84.7 KiB)

Reviewed: https://review.openstack.org/323599
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0330478b70d0a699a0f9c21ef87c7e639d92564b
Submitter: Jenkins
Branch: feature/hummingbird

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <email address hidden>
Date: Tue May 31 16:25:50 2016 +0200

    Fixed typo

    Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <email address hidden>
Date: Wed May 25 11:21:25 2016 -0700

    Clarify overload best practices in admin guide

    Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <email address hidden>
Date: Wed May 25 09:53:31 2016 +0200

    Add dispersion --verbose example to admin guide

    Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <email address hidden>
Date: Wed May 25 14:35:54 2016 +1000

    Add ring-builder dispersion command to admin guide

    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.

    Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <email address hidden>
Date: Tue May 24 13:37:58 2016 -0700

    Add links to mitaka install guides

    Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <email address hidden>
Date: Fri Apr 29 12:12:00 2016 -0500

    Allow concurrent bulk deletes

    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.

    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.

    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.

    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:

       delete_concurrency = 1

    This may be done prior to upgrading Swift.

    UpgradeImpact
    Closes-Bug: 1524454
    Change-Id: I128374d74a4cef7a479b221fd15eec785cc4694a

commit 226557afc42c245e050d84162497f46341407ef7
Author: Tim Burke <email address hidden>
Date: Thu May 19 18:55:40 2016 -0700

    Turn on H703, so our translators don't punch us

    Change-Id: I4ce3068f79563e4d4296c6e1078bc12f0cf84c96
    Related-Bug: 1559431

commit 7b706926a8ed5bbcec3a678e868e301c9a6ed8f1
Author: Alistair Coles <email address hidden>
Date: Mon May ...

tags: added: in-feature-hummingbird
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/324392

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (stable/liberty)

Reviewed: https://review.openstack.org/324392
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=92abacfc5c93df8d08ba83c9ed4641945b32d008
Submitter: Jenkins
Branch: stable/liberty

commit 92abacfc5c93df8d08ba83c9ed4641945b32d008
Author: Alistair Coles <email address hidden>
Date: Wed Mar 16 17:41:30 2016 +0000

    Put correct Etag and Accept-Ranges in EC 304 and 416 responses

    When using an EC policy, 304 responses to conditional GETs
    are missing the Accept-Ranges header and have the wrong ETag
    value. 412 responses also have the wrong etag.

    416 responses to ranged GETs also have the wrong ETag.

    This patch ensures behaviour with EC policy is consistent
    with replication policy:

      - 304 and 416 responses have correct etag and Accept-Ranges
      - 412 responses have correct Etag but no Accept-Ranges

    Co-Authored-By: Mahati Chamarthy <email address hidden>
    Co-Authored-By: Thiago da Silva <email address hidden>

    Cherry-picked from commit 12dd408823df158359e99fb01716f2059140c5c9

    Closes-Bug: #1496234
    Closes-Bug: #1558197
    Closes-Bug: #1558193
    Change-Id: Ic21317b9e4f632f0751133a3383eb5487379e11f

tags: added: in-stable-liberty
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/swift 2.7.0

This issue was fixed in the openstack/swift 2.7.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.