OpenStack Object Storage (swift)

Large X-Delete-At values cause object-server to 500

Bug #1860149 reported by Tim Burke on 2020-01-17

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Object Storage (swift)	Fix Released	High	Unassigned

Bug Description

So you can create objects with X-Delete-At timestamps in the far future:

$ curl -v http://saio:8090/v1/AUTH_test/c/o -X PUT -d 1234 -H x-delete-at:1579753284000
* Trying 127.0.1.1...
* TCP_NODELAY set
* Connected to saio (127.0.1.1) port 8090 (#0)
> PUT /v1/AUTH_test/c/o HTTP/1.1
> Host: saio:8090
> User-Agent: curl/7.58.0
> Accept: */*
> x-delete-at:1579753284000
> Content-Length: 4
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 4 out of 4 bytes
< HTTP/1.1 201 Created
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0
< Etag: 81dc9bdb52d04dc20036dbd8313ed055
< Last-Modified: Fri, 17 Jan 2020 15:51:01 GMT
< X-Trans-Id: txb9beeddcc97142ed8f9d5-005e21d7e4
< X-Openstack-Request-Id: txb9beeddcc97142ed8f9d5-005e21d7e4
< Date: Fri, 17 Jan 2020 15:51:01 GMT
<
* Connection #0 to host saio left intact

This might happen if you've got a JavaScript client accidentally using milliseconds instead of seconds, for example. But then you can't GET the data!

$ curl -v http://saio:8090/v1/AUTH_test/c/o
* Trying 127.0.1.1...
* TCP_NODELAY set
* Connected to saio (127.0.1.1) port 8090 (#0)
> GET /v1/AUTH_test/c/o HTTP/1.1
> Host: saio:8090
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< Content-Type: text/html; charset=UTF-8
< Content-Length: 118
< X-Trans-Id: txfa97de0ed5664597adf42-005e21d7f0
< X-Openstack-Request-Id: txfa97de0ed5664597adf42-005e21d7f0
< Date: Fri, 17 Jan 2020 15:51:13 GMT
<
* Connection #0 to host saio left intact
<html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

Object-server logs show

Jan 17 15:56:20 saio object-6040: ERROR __call__ error with GET /sdb4/312/AUTH_test/c/o :
Traceback (most recent call last):
  File "/vagrant/swift/swift/obj/server.py", line 1318, in __call__
    res = getattr(self, req.method)(req)
  File "/vagrant/swift/swift/common/utils.py", line 1985, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/vagrant/swift/swift/obj/server.py", line 1087, in GET
    with disk_file.open(current_time=req_timestamp):
  File "/vagrant/swift/swift/obj/diskfile.py", line 2441, in open
    current_time=current_time, modernize=modernize, **file_info)
  File "/vagrant/swift/swift/obj/diskfile.py", line 2716, in _construct_from_data_file
    self._verify_data_file(data_file, fp, current_time)
  File "/vagrant/swift/swift/obj/diskfile.py", line 2583, in _verify_data_file
    if x_delete_at <= current_time and not self._open_expired:
  File "/usr/lib/python3.6/functools.py", line 112, in _ge_from_lt
    op_result = self.__lt__(other)
  File "/vagrant/swift/swift/common/utils.py", line 1369, in __lt__
    other = Timestamp(other)
  File "/vagrant/swift/swift/common/utils.py", line 1281, in __init__
    raise ValueError('timestamp too large')
ValueError: timestamp too large (txn: txbc10a8f255df48f78880c-005e21d924)

Similar things happen if you try to overwrite it with a PUT, update the expiration time with a POST, or even just DELETE the damn thing! (Exact tracebacks may vary, of course). Since we've seen this in the wild [0][1], the fix ought to have two parts:

1. Stop tripping exceptions when working with existing data that has a bad x-delete-at timestamp.

2. Stop allowing new PUTs and POSTs with such large x-delete-at timestamps; it almost certainly means there's a client bug.

[0] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift.2020-01-16.log.html#t2020-01-16T07:17:22
[1] https://bugs.launchpad.net/swift/+bug/1821204

Tags:

clayg (clay-gerrard) on 2020-01-22

Changed in swift:
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-24: Fix merged to swift (master)

Reviewed: https://review.opendev.org/702950
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=57ca3570e911bd290ce0e44bdebaa4b5b4ceca13
Submitter: Zuul
Branch: master

commit 57ca3570e911bd290ce0e44bdebaa4b5b4ceca13
Author: Tim Burke <email address hidden>
Date: Thu Jan 16 10:07:33 2020 -0800

Allow Timestamp comparisons against out-of-range values

Prior to the related change, clients may have written down X-Delete-At headers
that are outside of the Timestamp range, for example.

    Change-Id: Ib8ae7ebcbdb32e0aa58446bd1ef949e5e2f63e74
    Related-Change: I23666ec8a067d829eaf9bfe54bd086c320b3429e
    Related-Bug: 1821204
    Partial-Bug: 1860149

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-26:

Reviewed: https://review.opendev.org/703155
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=7bba3ad04dec35a6d12f950c862846a7377df190
Submitter: Zuul
Branch: master

commit 7bba3ad04dec35a6d12f950c862846a7377df190
Author: Tim Burke <email address hidden>
Date: Fri Jan 17 08:32:34 2020 -0800

Store normalized x-delete-at on PUT/POST

Change-Id: I81e5d49623e884def8b5f6cbbf82f9b0d667d64a
Closes-Bug: 1860149

Changed in swift:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-02-03: Fix included in openstack/swift 2.24.0

This issue was fixed in the openstack/swift 2.24.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-03-18: Fix proposed to swift (feature/losf)

Fix proposed to branch: feature/losf
Review: https://review.opendev.org/713632

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-03-18: Fix merged to swift (feature/losf)

Download full text (40.5 KiB)

Reviewed: https://review.opendev.org/713632
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=79bd2e59e5c15ee84814ec1c4f0893176ba79412
Submitter: Zuul
Branch: feature/losf

commit f2ffd900593db2829a39a073f0c033d82985c40f
Author: Clay Gerrard <email address hidden>
Date: Fri Feb 28 11:09:51 2020 -0600

Apply limit to list versioned containers

Change-Id: I28e062273d673c4f07cd3c5da088aa790b77a599
Closes-Bug: #1863841

commit dc40779307095b8d0b2761b77b9cb2904ec721ae
Author: Clay Gerrard <email address hidden>
Date: Fri Feb 28 10:00:25 2020 -0600

Use float consistently for proxy timeout settings

Change-Id: I433c97df99193ec31c863038b9b6fd20bb3705b8

commit 55049beda5b9d7038a3604a87f28312d7702ccb2
Author: Tim Burke <email address hidden>
Date: Fri Feb 28 18:59:32 2020 -0800

tests: Use timedelta to adjust dates, not string manipulations

Change-Id: I8f65ccd7f2a79d5b877bfbef0274fb7857e21391

commit 3b65a5998cc921d2763cf1a9ec1e40b88491262d
Author: Tim Burke <email address hidden>
Date: Wed Jan 10 06:16:41 2018 +0000

Fix up some Content-Type handling in account/container listings

Update content type on 204 (not just 200) to properly handle HEAD
requests from xml/txt listings.

    Add "Vary: Accept" header to listings, since otherwise, browsers may
    serve the wrong content type from cache (even though we *would have*
    sent the *right* type if it actually sent the request).

Change-Id: Iaa333aaca36a8dc2df65d38ef2173e3a6e2000ee

commit ecca23eb806e11cf6517f0456483da7a065350a8
Author: Clay Gerrard <email address hidden>
Date: Fri Feb 21 15:33:21 2020 -0600

Extend eventlet_debug logging to GreenAsyncPile

Change-Id: Ibd9fe5c9a1e75b86eb7d540594d5cf516758e17e

commit 0fb3371484f1d0f629d0b0e33f6aafbff0e43ee9
Author: Sam Morrison <email address hidden>
Date: Tue Feb 18 10:17:50 2020 +1100

Delay importing swiftclient until after monkey-patching

Commit message below partly copied from nova:

Eventlet monkey patching should be as early as possible

    We were seeing infinite recursion opening an ssl socket when running
    various combinations of python3, eventlet, and urllib3. It is not
    clear exactly what combination of versions are affected, but for
    background there is an example of this issue documented here:

https://github.com/eventlet/eventlet/issues/371

    The immediate cause in swift's case was that we were calling
    eventlet.monkey_patch() after importing swiftclient (which imported
    requests, which finally imported urllib3).

    We only use the imported function in one place, however; hold off on
    importing until we actually need it to ensure that monkey patching
    happens first. Note that we *don't* want to monkey-patch at import time,
    as we've previously had bugs related to import-time side-effects.

    Change-Id: I24f4bcc3d62dc37fd9559032bfd25f5b15f98745
    Closes-bug: #1863680
    Related-bug: #1380815

commit a5afe767581d2cb97cf3690067e6d626c7682c2c
Author: Tim Burke <email address hidden>
Date: Wed Feb 19 10:09:49 2020 -0800

Revert "Make roll...

Reviewed:  https://review.opendev.org/713632
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=79bd2e59e5c15ee84814ec1c4f0893176ba79412
Submitter: Zuul
Branch:    feature/losf

commit f2ffd900593db2829a39a073f0c033d82985c40f
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Feb 28 11:09:51 2020 -0600

Apply limit to list versioned containers
    
    Change-Id: I28e062273d673c4f07cd3c5da088aa790b77a599
    Closes-Bug: #1863841

commit dc40779307095b8d0b2761b77b9cb2904ec721ae
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Feb 28 10:00:25 2020 -0600

Use float consistently for proxy timeout settings
    
    Change-Id: I433c97df99193ec31c863038b9b6fd20bb3705b8

commit 55049beda5b9d7038a3604a87f28312d7702ccb2
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Feb 28 18:59:32 2020 -0800

tests: Use timedelta to adjust dates, not string manipulations
    
    Change-Id: I8f65ccd7f2a79d5b877bfbef0274fb7857e21391

commit 3b65a5998cc921d2763cf1a9ec1e40b88491262d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jan 10 06:16:41 2018 +0000

Fix up some Content-Type handling in account/container listings
    
    Update content type on 204 (not just 200) to properly handle HEAD
    requests from xml/txt listings.
    
    Add "Vary: Accept" header to listings, since otherwise, browsers may
    serve the wrong content type from cache (even though we *would have*
    sent the *right* type if it actually sent the request).
    
    Change-Id: Iaa333aaca36a8dc2df65d38ef2173e3a6e2000ee

commit ecca23eb806e11cf6517f0456483da7a065350a8
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Feb 21 15:33:21 2020 -0600

Extend eventlet_debug logging to GreenAsyncPile
    
    Change-Id: Ibd9fe5c9a1e75b86eb7d540594d5cf516758e17e

commit 0fb3371484f1d0f629d0b0e33f6aafbff0e43ee9
Author: Sam Morrison <sorrison@gmail.com>
Date:   Tue Feb 18 10:17:50 2020 +1100

Delay importing swiftclient until after monkey-patching
    
    Commit message below partly copied from nova:
    
    Eventlet monkey patching should be as early as possible
    
    We were seeing infinite recursion opening an ssl socket when running
    various combinations of python3, eventlet, and urllib3. It is not
    clear exactly what combination of versions are affected, but for
    background there is an example of this issue documented here:
    
    https://github.com/eventlet/eventlet/issues/371
    
    The immediate cause in swift's case was that we were calling
    eventlet.monkey_patch() after importing swiftclient (which imported
    requests, which finally imported urllib3).
    
    We only use the imported function in one place, however; hold off on
    importing until we actually need it to ensure that monkey patching
    happens first. Note that we *don't* want to monkey-patch at import time,
    as we've previously had bugs related to import-time side-effects.
    
    Change-Id: I24f4bcc3d62dc37fd9559032bfd25f5b15f98745
    Closes-bug: #1863680
    Related-bug: #1380815

commit a5afe767581d2cb97cf3690067e6d626c7682c2c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Feb 19 10:09:49 2020 -0800

Revert "Make rolling-upgrade job non-voting until virtualenv problem is solved"
    
    This reverts commit cd4ba90cc1a13c94e2a02f1af015e20d6882e05f.
    
    The virtualenv problem seems to have been resolved.
    
    Change-Id: Icae429b4eba796328e767317e204ca30cc105da8

commit cd4ba90cc1a13c94e2a02f1af015e20d6882e05f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Feb 13 22:02:26 2020 -0800

Make rolling-upgrade job non-voting until virtualenv problem is solved
    
    Change-Id: I4c8a3756afccb2b42acbd9aeac5908f3d4febcae

commit 2a8d47f00eb3553755aaf2329d6619bee57d9cd9
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jan 2 15:19:27 2020 -0800

middlewares: Clean up app iters better
    
    Previously, logs would often show 499s in places where some other status
    would be more appropriate.
    
    Change-Id: I68dbb8593101cd3b5b64a1a947c68e340e36ce02

commit 61bf5ee1c428410b70bfab0a6eae361df948d03e
Author: Charles Hsu <charles0126@gmail.com>
Date:   Thu Feb 6 15:35:11 2020 +0800

Ensure domain stored in memcached gets utf8 decoded on py2
    
    Change-Id: I73b5af9645f3f7349144384609bf18a79620e92f
    Closes-Bug: #1862115

commit baaa5c57e18da5e7d4baa79e1b5420621c7adf0b
Author: Sean McGinnis <sean.mcginnis@gmail.com>
Date:   Wed Feb 5 16:20:53 2020 -0600

Remove reno from test-requirements
    
    Reno is only needed for building release notes. Reno has now dropped py2
    support, so attempts to install all requirements for py27 jobs fail with
    the latest reno release.
    
    Reno is already correctly listed in doc/requirements.txt, so this just
    removes the unnecessary entry in test-requirements.txt.
    
    Need-by: https://review.opendev.org/705994
    
    Change-Id: Id1c54915ebe43f2b46f611a95109d2ab0d02d34d
    Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>

commit 09b7ed600bfda6f708db85cd850c0539c3448a0a
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Feb 5 08:36:24 2020 -0800

sharding: filter shards based on prefix param when listing
    
    Otherwise, we make a bunch of backend requests where we have no
    real expectation of finding data.
    
    Change-Id: I7eaa012ba938eaa7fc22837c32007d1b7ae99709

commit ff885d30e46f65042843480f58a6f07ce66211cc
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Feb 3 13:01:51 2020 -0800

py3: Fix up probe tests
    
    Change-Id: Ic0f54f393002e2170e7f1459625ee5a2b37df900

commit d358b9130d80f5fff492de03abf74676a06e61c3
Author: John Dickinson <me@not.mn>
Date:   Thu Dec 13 10:45:51 2018 -0800

added value and notes to a sample config file for s3token
    
    Change-Id: I18accffb2cf6ba6a3fff6fd5d95f06a424d1d919

commit 0606f404fe92a3c56014feba55069a206ddb1cc8
Author: John Dickinson <me@not.mn>
Date:   Thu May 2 10:53:44 2019 -0600

enforce account quotas on empty accounts
    
    Change-Id: Icade637b0d384dcc87aaa31da3861cd2646d4ccd
    Closes-Bug: #1827394

commit b8c16de0238054b7757c6c0f541af853232b54a9
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Thu Jan 23 19:15:05 2020 +1100

Return correct etag for raw manifest
    
    When client sends a '?multipart-manifest=get&format=raw' request
    middleware will change the manifest returned from object server.
    This patch makes sure the response etag is updated to reflect
    changes to manifest content
    
    Change-Id: I0ac6dd0808fb041ba7663f4a472a06ee3f1d9a71

commit 5cd854b783ce80356ec62e709f0817cbe699c8cd
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jan 27 16:11:55 2020 -0800

Authors/changelog for swift 2.24.0
    
    Change-Id: I60d4ad62c7d14ed15e114ed95102a2368c48f9f4

commit c7cb34ad61feb5b830ccb79437b32965916220ef
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Thu Jan 30 15:33:41 2020 +1100

Dockerhub description of saio image
    
    Change-Id: Ifcb5013a5728d93cf5491fbff81b2677450698e6

commit 4a776c4fbd69271b6197b2c20beaaae8a2b2e980
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Thu Jan 30 15:56:36 2020 +1100

Add new versioning flag to docker image
    
    Change-Id: If6a25d6573502eebff3e28aa503721eda73af556

commit e3ee6ce31aed8d24fefdd20900d20688a12bb55d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Oct 14 18:35:00 2019 -0700

py38: cgi lost some names
    
    Drive-by: always use quote=True; it basically never hurts.
    
    Change-Id: Id91fdd19f226e9ec0d9c702d40d041c385c52b88

commit 6097660f0c350e12c026d62fd2fbd437140b917b
Author: karen chan <karen@karen-chan.com>
Date:   Fri Jan 27 07:43:37 2017 -0800

s3api: Implement object versioning API
    
    Translate AWS S3 Object Versioning API requests to native Swift Object
    Versioning API, speficially:
    
     * bucket versioning status
     * bucket versioned objects listing params
     * object GETorHEAD & DELETE versionId
     * multi_delete versionId
    
    Change-Id: I8296681b61996e073b3ba12ad46f99042dc15c37
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>

commit ebaf154fc4d7f91ff704ca2367556c137ed52137
Author: Romain LE DISEZ <romain.le-disez@corp.ovh.com>
Date:   Tue Jan 28 13:51:05 2020 -0500

Add a note to the part power increase documentation
    
    Specify that the relink command must be run as the same user than the
    daemon processes to avoid creating files that are not manipulable by the
    server/replicator/...
    
    Change-Id: If23592855db5f5bb0ec1e7c679de15769fd86871

commit bd063b678992a710f423a86d392a34237a8379ba
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jan 27 12:09:28 2020 -0800

s3api: use native strings in s3api.auth_details access_key
    
    Change-Id: I331b6871e5b62f61809338a1abddafe1263e7f02

commit a84e358dcfc4422cb67b8a19fba4203c6424bc43
Author: Romain LE DISEZ <romain.le-disez@corp.ovh.com>
Date:   Thu Mar 21 18:04:58 2019 +0100

SSYNC: log body of errors in SSYNC subrequests
    
    It helps an operator to understand why an SSYNC replication is not
    progressing on a specific partition.
    
    Change-Id: I75ccc833d579a98eff22377a83b703fd7e9c4720

commit d8821c75bd79c2b95ab7635b58754d059be2079b
Author: Romain LE DISEZ <romain.le-disez@corp.ovh.com>
Date:   Mon Jan 20 10:03:27 2020 -0500

proxy-logging: add fields ttfb and pid
    
    Change-Id: I1611e34846e586703e9d3709fa64e8df41f2d685

commit 27fd97cef927f39a057f46cf09a6c38ed52c6aa9
Author: Romain LE DISEZ <romain.le-disez@corp.ovh.com>
Date:   Thu Dec 19 14:24:51 2019 -0500

Middleware that allows a user to have quoted Etags
    
    Users have complained for a while that Swift's ETags don't match the
    expected RFC formats. We've resisted fixing this for just as long,
    worrying that the fix would break innumerable clients that expect the
    value to be a hex-encoded MD5 digest and *nothing else*.
    
    But, users keep asking for it, and some consumers (including some CDNs)
    break if we *don't* have quoted etags -- so, let's make it an option.
    
    With this middleware, Swift users can set metadata per-account or even
    per-container to explicitly request RFC compliant etags or not. Swift
    operators also get an option to change the default behavior
    cluster-wide; it defaults to the old, non-compliant format.
    
    See also:
      - https://tools.ietf.org/html/rfc2616#section-3.11
      - https://tools.ietf.org/html/rfc7232#section-2.3
    
    Closes-Bug: 1099087
    Closes-Bug: 1424614
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    Change-Id: I380c6e34949d857158e11eb428b3eda9975d855d

commit 51c9b13082b70bde91e553c5803bb2f403657b34
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jan 27 08:47:23 2020 -0800

Add unittest2 back to lower-constraints.txt
    
    testtools (required by os-testr) still requires unittest2; until
    
      * testtools no longer requires unittest2 and
      * we bump up our minimum-supported version of testtools to a version
        that doesn't reference unittest2,
    
    we should continue listing unittest2 in lower-constraints.txt (just in
    case they ever have a new release).
    
    Admittedly, that seems unlikely at this point -- unittest2's latest
    release was back in 2015.
    
    Change-Id: I09301bf81d68357ec006a21dd37e72c56cefffe3

commit 2759d5d51c5e3c684081e066f9efeb495c141524
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Sep 13 12:25:24 2019 -0500

New Object Versioning mode
    
    This patch adds a new object versioning mode. This new mode provides
    a new set of APIs for users to interact with older versions of an
    object. It also changes the naming scheme of older versions and adds
    a version-id to each object.
    
    This new mode is not backwards compatible or interchangeable with the
    other two modes (i.e., stack and history), especially due to the changes
    in the namimg scheme of older versions. This new mode will also serve
    as a foundation for adding S3 versioning compatibility in the s3api
    middleware.
    
    Note that this does not (yet) support using a versioned container as
    a source in container-sync. Container sync should be enhanced to sync
    previous versions of objects.
    
    Change-Id: Ic7d39ba425ca324eeb4543a2ce8d03428e2225a1
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    Co-Authored-By: Thiago da Silva <thiagodasilva@gmail.com>

commit 26ff2eb1cbec8052797dc5e28af3a8049d5e5dfb
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Tue Dec 10 16:32:47 2019 +1100

container-sync: Sync static links similar to how we sync SLOs
    
    This allows static symlinks to be synced before their target. Dynamic
    symlinks could already be synced even if target object has not been
    synced, but static links previously required that target object existed
    before it can be PUT. Now, have container_sync middleware plumb in an
    override like it does for SLO.
    
    Change-Id: I3bfc62b77b247003adcee6bd4d374168bfd6707d

commit 2be4716bc9ebbdb1da69231d4618489dff8af85f
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Sep 16 13:06:15 2019 -0500

More multi-char-delimiter func test coverage
    
    Change-Id: I4cad872ef97313ec19a8e4ef042696565a56d4c5

commit 55cdb55075657c283ce786ca634f8a42ec786490
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jan 22 11:00:18 2020 -0800

Lower the log level for falling back to mkstemp
    
    We recently started trying to use O_TMPFILE without the kernel version
    check, we gracefully degrade when it isn't available, and we don't give
    operators any way to avoid the O_TMPFILE attempt -- logging at warning
    seems way too noisy.
    
    Change-Id: I8f42127fd6eb930282161930c65e8614f3e3175f
    Related-Change: I3599e2ab257bcd99467aee83b747939afac639d8

commit cd78ee983d0ae0cebdb27327b6ea7950308d0dbf
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jan 22 09:40:54 2020 -0800

Bump up ceph tests timeout
    
    Looking at the last few hundred runs, we've had ~8% TIMEOUTs, and the
    average time is already 21 minutes. Let's give it an extra 10 minutes,
    see if the TIMEOUT rate comes down.
    
    Change-Id: Ifd9fbac032825bae1c8f5edd88c5d692a0b2cef1

commit f68e22d43a1bfbbdf182b1cc6819bdc83d166d5d
Author: gillesbiannic <gilles.biannic@corp.ovh.com>
Date:   Fri Oct 25 20:11:30 2019 +0000

Stop retrying every deletes in container-sync
    
    Internal clients used to retry operations even on client errors, where
    we had no expectation that we would get a different response. Among
    other things, this would cause container-sync to needlessly retry
    deletes that would 404, even though we had code specifically in place to
    say that 404s are OK.
    
    While we're at it, also flag 409s as being OK (since the remote has
    newer data than us).
    
    Change-Id: I1833a8ff2ac2f4126b6274d6bba47e2b58a41745
    Closes-Bug: #1849841

commit 73a0e8e9cf7d26e3a06d3b48164d345478de4161
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Oct 25 21:56:01 2019 -0700

probe: Add test for syncing a delete when the remote 404s
    
    This works fine; we continue processing the other rows in the DB. But it
    *does* take longer than it really ought to require. See the related bug;
    we ought to be able to shave some 17s off the test time by not retrying
    on the 404.
    
    Change-Id: I9ca2511651e9b2bc0045894baa4062d20bc15369
    Related-Bug: #1849841

commit dad8fb7c3c0b4c1dbaecde341a5c07d02a0a578a
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jan 20 21:15:17 2020 -0800

packaging: Build universal wheels
    
    Apparently this had been done for us previously? See also:
    http://lists.openstack.org/pipermail/openstack-discuss/2020-January/012097.html
    
    Change-Id: I27288f9e503f26827a8274d46b07c1330a21de10

commit 7bba3ad04dec35a6d12f950c862846a7377df190
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jan 17 08:32:34 2020 -0800

Store normalized x-delete-at on PUT/POST
    
    Change-Id: I81e5d49623e884def8b5f6cbbf82f9b0d667d64a
    Closes-Bug: 1860149

commit ff0753fe19be3d7442aef3972e08adc48d25fe93
Author: Romain LE DISEZ <romain.le-disez@corp.ovh.com>
Date:   Thu Jan 9 09:08:25 2020 -0500

Allow bulk to fwd some headers at tar extraction
    
    Whitelisted headers include X-Delete-At/X-Delete-After and all Object
    metadata headers (X-Object-Meta-*)
    
    Closes-Bug: 1857546
    Change-Id: If5fb164693e395f89d57899fb8ab355f1e3f817c

commit 57ca3570e911bd290ce0e44bdebaa4b5b4ceca13
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jan 16 10:07:33 2020 -0800

Allow Timestamp comparisons against out-of-range values
    
    Prior to the related change, clients may have written down X-Delete-At headers
    that are outside of the Timestamp range, for example.
    
    Change-Id: Ib8ae7ebcbdb32e0aa58446bd1ef949e5e2f63e74
    Related-Change: I23666ec8a067d829eaf9bfe54bd086c320b3429e
    Related-Bug: 1821204
    Partial-Bug: 1860149

commit 0901464513bc371e5fff2c52004de364d56c271f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jan 15 18:32:19 2020 -0800

Clean up container-sync docs
    
    Change-Id: I98ebe15353d675ca00fee387003bf6572ac385e6

commit 9483630ae130c5b2616f8617da25bcb51b647208
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Jan 7 21:16:37 2020 -0800

py3: Fix formpost unicode filename issues
    
    Previously, we took the native string filename attribute and put it
    directly in the (WSGI string) PATH_INFO field. Now, we convert it to
    a WSGI string first.
    
    Change-Id: I30e3beb8707b88c36bd3cdc7a0887d069e943ba6
    Closes-Bug: #1858259

commit 5b26b749b58f93ee37bbca26ed3f6a13d9a31bc1
Author: Sean McGinnis <sean.mcginnis@gmail.com>
Date:   Sun Jan 12 03:10:25 2020 -0600

Drop use of unittest2
    
    unittest2 was needed for Python version <= 2.6, so it hasn't been needed
    for quite some time. See unittest2 note one:
    
    https://docs.python.org/2.7/library/unittest.html
    
    This drops unittest2 in favor of the standard unittest module.
    
    Change-Id: I2e787cfbf1709b7f9c889230a10c03689e032957
    Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>

commit 61401907911d09fc1d2232118717b048e77a6649
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jan 6 16:27:06 2020 -0600

Make FakeSwift query param order agnostic
    
    Apparently FakeSwift has always been persnickety about the order of
    query params of registered responses and the app making the calls.
    Since query params can often be converted to dictionaries the order of
    the encoded params should be able to change without effecting the apps
    dispatching of registered responses.
    
    Change-Id: Ied68c9334201a7663e9c85f3bdaa5b0643d4b6db

commit 47eb78897d60ee60a4e42953f5efc52a46d6cd18
Author: zhufl <zhu.fanglei@zte.com.cn>
Date:   Mon Jan 6 10:34:42 2020 +0800

Fix  the duplicated words issue like "the the "
    
    This is to fix the duplicated words issue, like:
      * "both of which which must be stripped off"
      * "In addition the the values set"
      and so on
    
    Change-Id: Id3d84281f15815b4185c76874575e91a3589981b

commit b1178b4a967ab621e66356a9e7bca08de5be5e2a
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Dec 20 11:47:13 2019 -0600

Allow reconciler to handle reserved names
    
    Change-Id: Ib918f10e95970b9f562b88e923c25608b826b83f

commit 4601548dabdec0a4dc89cefba11e963217255be3
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Dec 31 13:48:13 2019 -0600

Deprecate per-service auto_create_account_prefix
    
    If we move it to constraints it's more globally accessible in our code,
    but more importantly it's more obvious to ops that everything breaks if
    you try to mis-configure different values per-service.
    
    Change-Id: Ib8f7d08bc48da12be5671abe91a17ae2b49ecfee

commit 0f13db30b6425b9c973b4add984e22ae56ce84c2
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Fri Jan 3 18:34:11 2020 -0600

Import directly from the inside of the ring module
    
    An odd thing happened: when my code did only
    "from swift.common.ring import Ring", and nothing else,
    the interpreter looped on the CPU.
    
    The import from the top of the ring module is our standard
    protocol for Ring. It causes no problem in places like
    container updater or list_endpoints.py. It is a big
    mystery why it causes Python 3.7.3 to loop, and only
    in specific circumstances.
    
    But we noticed that the recursive import is unnecesasry
    in this case, so an obviously harmless fix exists.
    
    Change-Id: I7373bbb0a50d090d6becf141e5832f8ae98381a4

commit 3f8890701259e24ef81c93947faf2d4ccf223c5f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Dec 18 15:14:00 2019 -0800

sharding: Better-handle newlines in container names
    
    Previously, if you were on Python 2.7.10+ [0], such a newline would cause the
    sharder to fail, complaining about invalid header values when trying to create
    the shard containers. On older versions of Python, it would most likely cause a
    parsing error in the container-server that was trying to handle the PUT.
    
    Now, quote all places that we pass around container paths. This includes:
    
      * The X-Container-Sysmeta-Shard-(Quoted-)Root sent when creating the (empty)
        remote shards
      * The X-Container-Sysmeta-Shard-(Quoted-)Root included when initializing the
        local handoff for cleaving
      * The X-Backend-(Quoted-)Container-Path the proxy sends to the object-server
        for container updates
      * The Location header the container-server sends to the object-updater
    
    Note that a new header was required in requests so that servers would
    know whether the value should be unquoted or not. We can get away with
    reusing Location in responses by having clients opt-in to quoting with
    a new X-Backend-Accept-Quoted-Location header.
    
    During a rolling upgrade,
    
      * old object-servers servicing requests from new proxy-servers will
        not know about the container path override and so will try to update
        the root container,
      * in general, object updates are more likely to land in the root
        container; the sharder will deal with them as misplaced objects, and
      * shard containers created by new code on servers running old code
        will think they are root containers until the server is running new
        code, too; during this time they'll fail the sharder audit and report
        stats to their account, but both of these should get cleared up upon
        upgrade.
    
    Drive-by: fix a "conainer_name" typo that prevented us from testing that
    we can shard a container with unicode in its name. Also, add more UTF8
    probe tests.
    
    [0] See https://bugs.python.org/issue22928
    
    Change-Id: Ie08f36e31a448a547468dd85911c3a3bc30e89f1
    Closes-Bug: 1856894

commit 286082222dc70da18bbca2e3d18a4e0aeb22cb2c
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Dec 20 14:30:20 2019 -0600

Use less responses from handoffs
    
    Since we don't use 404s from handoffs anymore, we need to not let errors
    on handoffs overwhelm primary responses either
    
    Change-Id: I2624e113c9d945542f787e5f18f487bd7be3d32e
    Closes-Bug: #1857909

commit 55ab08a5fadda87ffc904d9fe0ba1d1e09612ae3
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jan 2 16:21:19 2020 -0800

account-server: Add test for leading delimiter
    
    Related-Change: I27701a31bfa22842c272b7781738e8c546b82cbc
    Related-Change: If912f71d8b0d03369680374e8233da85d8d38f85
    Change-Id: I0c850b8ae40d1ab477d0f5d18f92579c457da54e
    Related-Bug: 1475018

commit e8b654f318cfab485a772e19db33cb2fe4c3d858
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Dec 6 23:26:37 2019 +0000

Have slo tell the object-server that it wants whole manifests
    
    Otherwise, we waste a request on some 416/206 response that won't be
    helpful.
    
    To do this, add a new X-Backend-Ignore-Range-If-Metadata-Present header
    whose value is a comma-separated list of header names. Middlewares may
    include this header to tell object-servers to send the whole object
    (rather than a 206 or 416) if *any* of the metadata are present.
    
    Have dlo and symlink use it, too; it won't save us any round-trips, but
    it should clean up some object-server logging.
    
    Change-Id: I4ff2a178d0456e7e37d561109ef57dd0d92cbd4e

commit 40edf07ab347a9a73e944f4a18dd96b504f7e194
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Dec 20 21:36:07 2019 +0000

sharding: Let swift-manage-shard-ranges accept a relative path
    
    Previously, passing a relative path would confuse the ContainerBroker about
    which DB files are available, leading to an IndexError when none were found.
    
    Just call realpath() on whatever the user provided so we don't have to muck
    with any of the broker code.
    
    Change-Id: Icdf100dfcd006d975b49d151b99aa9272452d013

commit a06c0a47574639a27ea3e4742904c522dbee7d19
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Dec 20 06:54:13 2019 +0000

sharder: quote() more Swift paths when logging
    
    The lack of quoting gets extra troublesome with reserved names,
    where messages get truncated.
    
    Change-Id: I415901d3a8cd24cb3cedc72235292bb9d1705bbc

commit d246bf20ed30047f9b285dcc3378f9c324ed938e
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Dec 19 22:27:27 2019 -0800

sharding: Tolerate blank limits when listing
    
    Otherwise, we can 500 with
    
       ValueError: invalid literal for int() with base 10: ''
    
    Change-Id: I35614aa4b42e61d97929579dcb16f7dfc9fef96f

commit 54312b0b79ef2517f87a84a101507038d833104f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Dec 19 22:09:57 2019 -0800

py3: Clean up some needlessly-noisy tracebacks
    
    Change-Id: I5b391664b9260c762bd122a92387084c0582a362
    Related-Change: I5f3bc6416207cab2c7e3a77ee6689360b55990e7

commit c94f625fe463d808d4bae1137739a4dfcf6f4cd8
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Dec 17 15:11:36 2019 -0800

Forward-port stable-release changelog entries
    
    Change-Id: Ib2df3ec683c1670cadba72d1fc0d666975f64aab

commit 226e2c12012d492969ebce3e12343b007b054c77
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Dec 17 14:50:09 2019 -0800

Fix type on py3
    
    This shouldn't have any practical effect, since *any* False-y value for
    chunk will give us the correct behavior, but let's be consistent in the
    type used with that name.
    
    Change-Id: I1928883792e7a9945f3731ddeea992d14882d679

commit 1f7b97ec0f49b26b8a33d67cf924f5ab8345a614
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Nov 19 21:25:45 2019 -0800

Add normalize_etag() helper function
    
    ... and drive-by a import rename
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Change-Id: I1eaf075ff9855cfa03e7991bdf33375b0e4397e6

commit b65d8b10c5ea1c897bcb02c97943af82fd27d578
Author: Tim Burke <tim.burke@gmail.com>
Date:   Sat Dec 14 01:15:06 2019 +0000

Early-return on non-Swift get_info requests
    
    Change-Id: Iadc61a1c3bcbfbc47f65ec65df36d8da3694ee74

commit 6b33cf99f43892a22f1ab4124b10c49462b9cbc2
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed Dec 11 12:14:50 2019 -0600

WSGI server workers must drop_privledges
    
    ... just like they always have and server per port strategy still does.
    
    Related-Change-Id: I3e5229d2fb04be67e53533ff65b0870038accbb7
    Change-Id: I14e3ed201ceaceef0f8dbc44685395f350a0e7fc

commit 05e4bb30b64d223c024127335870fa7c267f08ae
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Wed Dec 11 20:30:38 2019 +1100

Update list of experimental upgrade tests
    
    Removed test of upgrade from queens and added train
    
    Change-Id: Iadab0f87d6fd66fe49c79ee200d175f9bed92baa

commit 8c0fd3f1386cc71c618a6711f03a9f220692bd07
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Dec 11 01:07:19 2019 +0000

py3: Make seamless reloads work
    
    Starting with Python 3.4, newly-created file descriptors are non-inheritable
    [0], which causes trouble when we try to use a pipe for IPC. Fortunately, the
    same PEP that implemented this change *also* provided a new API to mark file
    descriptors as being inheritable -- so just do that.
    
    While we're at it,
    
    * Fix up the probe tests to work on py3
    * Fix up the probe tests to work when policy-0 is erasure-coded
    * Decode the bytes read so py3 doesn't log a b'pid'
    * Log a warning if the read() is empty; something surely went wrong
      in the re-exec
    
    [0] https://www.python.org/dev/peps/pep-0446/
    
    Change-Id: I2a8a9f3dc78abb99bf9cbcf6b44c32ca644bb07b
    Related-Change: I3e5229d2fb04be67e53533ff65b0870038accbb7

commit da4e07164e09d099b8148df7669cb82862ffdcc1
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Dec 5 14:23:42 2019 -0800

Set swift_source more in s3api middleware
    
    Change-Id: I89f3a4b5a3a8c160afb298aad726acce09c65265

commit 51b7b110a104806f10c32ddb7c4bf1859b66f1ab
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Dec 5 14:33:02 2019 -0800

Set swift_source more in versioned_writes
    
    Change-Id: Id0b10cb5f1f3e2c39c8d134f55774a36dc1e75df

commit c35ab2e1d8da21f5942791c6fd9aac1e66978b74
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Dec 5 14:18:18 2019 -0800

Set swift_source in account_quotas middleware
    
    Change-Id: Ib54a65a920fc335c1e0a496cb91c0de3003e2cc5

commit f33c061ae92f33ea467c58749d3f15d2b1cc942c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Dec 5 09:42:33 2019 -0800

s3api: Better handle 498/429 responses
    
    Currently, they just 500 as an unexpected response status. Much better
    would be S3's '503 Slow Down' response.
    
    Of course, that's all dependent on where you place ratelimit in your
    pipeline -- and we haven't really given clear guidance on that. I'm not
    actually sure you *want* ratelimit to be after s3api and auth... but if
    you *do*, let's at least handle it gracefully.
    
    Change-Id: I36f0954fd9949d7d1404a0c381b917d1cfb17ec5
    Related-Bug: 1669888

commit 698717d886b2b55ea9b490719851c85c20b57240
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Sep 13 12:25:24 2019 -0500

Allow internal clients to use reserved namespace
    
    Reserve the namespace starting with the NULL byte for internal
    use-cases.  Backend services will allow path names to include the NULL
    byte in urls and validate names in the reserved namespace.  Database
    services will filter all names starting with the NULL byte from
    responses unless the request includes the header:
    
        X-Backend-Allow-Reserved-Names: true
    
    The proxy server will not allow path names to include the NULL byte in
    urls unless a middlware has set the X-Backend-Allow-Reserved-Names
    header.  Middlewares can use the reserved namespace to create objects
    and containers that can not be directly manipulated by clients.  Any
    objects and bytes created in the reserved namespace will be aggregated
    to the user's account totals.
    
    When deploying internal proxys developers and operators may configure
    the gatekeeper middleware to translate the X-Allow-Reserved-Names header
    to the Backend header so they can manipulate the reserved namespace
    directly through the normal API.
    
    UpgradeImpact: it's not safe to rollback from this change
    
    Change-Id: If912f71d8b0d03369680374e8233da85d8d38f85

commit a2aaf598520331874c8d49a3230f6d4892fb3207
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Nov 26 09:51:09 2019 -0600

Fix BadResponseLength error when creating symlink
    
    Closes-Bug: #1854074
    
    Change-Id: I587275e68083aaed3c04486049b67791b52562b7

commit e11c58ef892084f3dd1b08e0d0bfd97668705988
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Nov 22 17:38:54 2019 +0000

Turn off dns_compliant_bucket_names for ceph tests
    
    We get a handful more passing tests that way, following their move to boto3.
    
    Change-Id: I73e9c38bde00a7117cec97e98f013e86350aa5be

commit b5c9dc1c9f68afe8083b74eaf8080e17a9ef8f8b
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Nov 19 09:02:27 2019 -0800

py3: Fix s3api header casing
    
    Closes-Bug: #1853367
    
    Change-Id: Ifb15ce50fc3bcfda9532a2c3dec542c272ea4933

commit 249a302d0c43bf7d847b3691b795a7ba3c5b6c10
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Nov 19 14:57:02 2019 -0800

Remove a bunch of known-failures that moved from boto to boto3
    
    Change-Id: I775a03e0ba1e10982eb6f7ef52be773c8831b1ec

commit e5dad445b258b6fd823bddb7b1e3ca10064ba3e8
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Nov 19 10:16:28 2019 -0800

Switch py2 DSVM jobs to only run swift under py2
    
    Other services are fine to move on to py3. We still want to test py2,
    though.
    
    Change-Id: I488f88a54948663c682b35dc976b62db6e72eef7

commit 29019985ed63790851df7a2a0ed75593b11906b4
Author: Thierry Carrez <thierry@openstack.org>
Date:   Tue Nov 19 17:32:02 2019 +0100

Start README.rst with a better title
    
    Now that we are using gitea the contents of our README.rst are
    more prominently displayed. Starting it with a "Team and repository
    tags" title is a bit confusing. This change makes it start with the
    name of the project instead.
    
    Change-Id: Iaf6c162b26a32d16a083aa721aa084807af8cd60

commit cfb3ae6019d9a4437675ff9d6161d67071fe528d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Nov 15 16:09:21 2019 -0800

Update known-failures and config for up-rev'ed ceph/s3tests
    
    Change-Id: I3833843cd8d23c498a2afe6c68a3f0afe26343c0

commit d6db7360e751e12895a5ba7b9d82db2df01fa057
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Fri Nov 15 10:25:47 2019 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://docs.openstack.org/i18n/latest/reviewing-translation-import.html
    
    Change-Id: I395f78516b33b675eab4aa7397742bff90e0167a

commit da5b94914302128c20b8ae9e3a2752a02ff29e73
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Oct 18 15:15:08 2019 -0700

Add py38 unit test job
    
    Depends-On: https://review.opendev.org/#/c/688594/
    Change-Id: Idf6c492d554a3ac3d38bd9dfc9c89494cae82bb1

commit 3de330ead3d5f68bcd04844ed9616456ee3ebe73
Author: Adrien Pensart <adrien.pensart@corp.ovh.com>
Date:   Fri Nov 8 12:18:06 2019 +0800

print deleted flag in account and container info cli
    
    Change-Id: I37c76e0c58a3d12b2062d5a6dbda2bf074f8bc9a

commit 1107f24179c0c6fdcb58771f3a6e6c025352b5d3
Author: Darrell Bishop <darrell@swiftstack.com>
Date:   Fri Oct 25 12:48:36 2019 -0700

Seamlessly reload servers with SIGUSR1
    
    Swift servers can now be seamlessly reloaded by sending them a SIGUSR1
    (instead of a SIGHUP).  The server forks off a synchronized child to
    wait to close the old listen socket(s) until the new server has started
    up and bound its listen socket(s).  The new server is exec'ed from the
    old one so its PID doesn't change.  This makes Systemd happier, so a
    ReloadExec= stanza can now be used.
    
    The seamless part means that incoming connections will alwyas get
    accepted either by the old server or the new one.  This eliminates
    client-perceived "downtime" during server reloads, while allowing the
    server to fully reload, re-reading configuration, becoming a fresh
    Python interpreter instance, etc.  The SO_REUSEPORT socket option has
    already been getting used, so nothing had to change there.
    
    This patch also includes a non-invasive fix for a current eventlet bug;
    see https://github.com/eventlet/eventlet/pull/590
    That bug prevents a SIGHUP "reload" from properly servicing existing
    requests before old worker processes close sockets and exit.  The
    existing probtests missed this, but the new ones, in this patch, caught
    it.
    
    New probe tests cover both old SIGHUP "reload" behavior as well as the
    new SIGUSR1 seamless reload behavior.
    
    Change-Id: I3e5229d2fb04be67e53533ff65b0870038accbb7

commit 2f1111a4364858fe87696f748405e6a3ff8f5655
Author: Romain LE DISEZ <romain.le-disez@corp.ovh.com>
Date:   Fri Nov 1 17:23:31 2019 -0400

proxy: stop sending chunks to objects with a Queue
    
    During a PUT of an object, the proxy instanciates one Putter per
    object-server that will store data (either the full object or a
    fragment, depending on the storage policy). Each Putter is owning a
    Queue that will be used to bufferize data chunks before they are
    written to the socket connected to the object-server. The chunks are
    moved from the queue to the socket by a greenthread. There is one
    greenthread per Putter. If the client is uploading faster than the
    object-servers can manage, the Queue could grow and consume a lot of
    memory. To avoid that, the queue is bounded (default: 10). Having a
    bounded queue also allows to ensure that all object-servers will get
    the data at the same rate because if one queue is full, the
    greenthread reading from the client socket will block when trying to
    write to the queue. So the global rate is the one of the slowest
    object-server.
    
    The thing is, every operating system manages socket buffers for incoming
    and outgoing data. Concerning the send buffer, the behavior is such that
    if the buffer is full, a call to write() will block, otherwise the call
    will return immediately. It behaves a lot like the Putter's Queue,
    except that the size of the buffer is dynamic so it adapts itself to the
    speed of the receiver.
    
    Thus, managing a queue in addition to the socket send buffer is a
    duplicate queueing/buffering that provides no interest but is, as shown
    by profiling and benchmarks, very CPU costly.
    
    This patch removes the queuing mecanism. Instead, the greenthread
    reading data from the client will directly write to the socket. If an
    object-server is getting slow, the buffer will fulfill, blocking the
    reader greenthread. Benchmark shows a CPU consumption reduction of more
    than 30% will the observed rate for an upload is increasing by about
    45%.
    
    Change-Id: Icf8f800cb25096f93d3faa1e6ec091eb29500758

commit 7f6b09dc930706d63a943dc239dff84ddae52014
Author: SeongSoo Cho <ppiyakk2@printf.kr>
Date:   Thu Nov 7 15:32:21 2019 +0800

Add delete_account in InternalClient
    
    Currently, The internal client module is missing a delete_account method.
    I added a delete_account in InternalClient
    
    Change-Id: I97cb090820b4099944ba49eaeb5b02f9bbcf1d58
    Closes-Bug: #1680377

commit 30d99f514071a9272caa018d140c9f7a367a7b6f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Nov 7 00:15:25 2019 +0000

Get rid of _delete_db
    
    It was parctically the same between both, only the table name was
    different -- and we already have a way to accomodate that.
    
    Change-Id: I112bef1cf44c7d6eeaacd30cea27cb71fb2e2406

commit e7cd8df5e93a89fba6a62d2f2f4270a1ce789e93
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Oct 25 12:30:13 2019 -0500

Add option for debug query logging
    
    Change-Id: Ic16b505a37748f50dc155212671efb45e2c5051f

commit e58840c571b1a1fe9ae9017a07c6754a10cbf69b
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Oct 14 11:01:59 2019 -0700

swift-account-audit: work with encryption
    
    Change-Id: I26c5fe9d45a9765da0d30138ea2df16fd4e73d57

commit deba3a63052d639e26252afb9a7f71aa66c04c91
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Jun 27 09:10:13 2019 -0500

clarify s3api ACL test HEADs
    
    Change-Id: Ic67c6698fd98a38f6ddf3ba92057331e437a888a

commit 46bde0c0b2a99580584f34e4dd72c9ac8b611fac
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Dec 11 18:47:53 2018 +0000

s3token: Raise error on negative secret_cache_duration config
    
    Change-Id: I8a961302a8905f483433976f6105b138a1acd7ba

commit ec70cf9605752e3b8a5c33ee7ef145f97df7567f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Dec 10 19:58:18 2018 +0000

s3token: When caching is enabled, default auth_type to password
    
    Otherwise, if you just set secret_cache_duration and skip configuring
    auth_type, you get a not-very-obvious
    
        TypeError: sequence item 0: expected string, NoneType found
    
    bubbling up from stevedore. Now, you'd get something like
    
        MissingRequiredOptions: Auth plugin requires parameters which were
        not given: auth_url
    
    Change-Id: Ifddbb9df60207dfaaf0434877f4fe6af5c26bdf8

commit 814a76689f69957f0d092546e07c0c1ce9c3d38b
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Sat Sep 15 06:49:18 2018 +0900

Follow up punch_hole patch
    
    Add missing tests to get more coverage and reduce a line.
    
    Change-Id: I34d8063ee82323c9751b4c965bee01ab584c5eb5

tags:

added: in-feature-losf

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

python-roundup #22928
[2:3] Edit
auto-github-eventlet-eventlet #371
[open importance-bug platform-python3] Edit

Bug watches keep track of this bug in other bug trackers.