Bug #1521339 “change default ports for servers” : Bugs : OpenStack Object Storage (swift)

Shashi (shashirekha-j-gundur) on 2016-01-29

Changed in swift:
assignee:	nobody → Shashi (shashirekha-j-gundur)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-02-01: Fix proposed to swift (master)

#1

Fix proposed to branch: master
Review: https://review.openstack.org/274840

Changed in swift:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-29: Fix merged to swift (master)

#2

Reviewed: https://review.openstack.org/274840
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=cf48e75c259350daa667155e3ffd38b17946bf44
Submitter: Jenkins
Branch: master

commit cf48e75c259350daa667155e3ffd38b17946bf44
Author: Shashirekha Gundur <email address hidden>
Date: Mon Feb 1 18:06:54 2016 +0000

change default ports for servers

    Changing the recommended ports for Swift services
    from ports 6000-6002 to unused ports 6200-6202;
    so they do not conflict with X-Windows or other services.

Updated SAIO docs.

    DocImpact
    Closes-Bug: #1521339
    Change-Id: Ie1c778b159792c8e259e2a54cb86051686ac9d18

Changed in swift:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-12: Fix proposed to swift (feature/crypto)

#3

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/315554

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-12: Fix merged to swift (feature/crypto)

#4

Download full text (14.4 KiB)

Reviewed: https://review.openstack.org/315554
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=3d934360d66fe768c5c6dadf29b38136d863c550
Submitter: Jenkins
Branch: feature/crypto

commit 46d61a4dcd9a5d9157625c06d6fe7d916e80c3d2
Author: Prashanth Pai <email address hidden>
Date: Wed Feb 18 11:59:31 2015 +0530

Refactor server side copy as middleware

    Rewrite server side copy and 'object post as copy' feature as middleware to
    simplify the PUT method in the object controller code. COPY is no longer
    a verb implemented as public method in Proxy application.

    The server side copy middleware is inserted to the left of dlo, slo and
    versioned_writes middlewares in the proxy server pipeline. As a result,
    dlo and slo copy_hooks are no longer required. SLO manifests are now
    validated when copied so when copying a manifest to another account the
    referenced segments must be readable in that account for the manifest
    copy to succeed (previously this validation was not made, meaning the
    manifest was copied but could be unusable if the segments were not
    readable).

    With this change, there should be no change in functionality or existing
    behavior. This is asserted with (almost) no changes required to existing
    functional tests.

    Some notes (for operators):
    * Middleware required to be auto-inserted before slo and dlo and
      versioned_writes
    * Turning off server side copy is not configurable.
    * object_post_as_copy is no longer a configurable option of proxy server
      but of this middleware. However, for smooth upgrade, config option set
      in proxy server app is also read.

DocImpact: Introducing server side copy as middleware

Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>

    Change-Id: Ic96a92e938589a2f6add35a40741fd062f1c29eb
    Signed-off-by: Prashanth Pai <email address hidden>
    Signed-off-by: Thiago da Silva <email address hidden>

commit cb8cda6588735411c542a1e4d78315ee047a357b
Author: Paul Dardeau <email address hidden>
Date: Mon May 9 20:28:42 2016 +0000

resurrect gholt blog posts on building consistent hashing ring

    These are blog posts authored by Greg Holt (gholt) and used with
    permission to add here. Content was only reformatted as rst and
    wrap prose lines at 70 characters.

Change-Id: I7aa47c24b5019aa598ee005e01612a49514da25f

commit ba1a568f815a8a40bf75c0beb96b789bd07adcb6
Author: Alistair Coles <email address hidden>
Date: Wed Mar 9 10:48:54 2016 +0000

Rename hash_cleanup_listdir tests

    hash_cleanup_listdir was removed in [1], this patch
    renames all references to it in test_diskfile to refer to
    the cleanup_ondisk_files method that is now tested directly.

Also remove the final references to the now non-existent
function in a few comments.

[1] I0b96dfde32b4c666eebda6e88228516dd693ef92

Change-Id: I1e151799fc2774de9a1af092afff875af24a630c
Related-Bug: #1550569

commit 6827affe62771949a041d958db17ba6cd29232ab
Author: John Dickinson <email address hidden>
Date: Tue Apr 5 ...

Reviewed:  https://review.openstack.org/315554
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=3d934360d66fe768c5c6dadf29b38136d863c550
Submitter: Jenkins
Branch:    feature/crypto

commit 46d61a4dcd9a5d9157625c06d6fe7d916e80c3d2
Author: Prashanth Pai <ppai@redhat.com>
Date:   Wed Feb 18 11:59:31 2015 +0530

Refactor server side copy as middleware
    
    Rewrite server side copy and 'object post as copy' feature as middleware to
    simplify the PUT method in the object controller code. COPY is no longer
    a verb implemented as public method in Proxy application.
    
    The server side copy middleware is inserted to the left of dlo, slo and
    versioned_writes middlewares in the proxy server pipeline. As a result,
    dlo and slo copy_hooks are no longer required. SLO manifests are now
    validated when copied so when copying a manifest to another account the
    referenced segments must be readable in that account for the manifest
    copy to succeed (previously this validation was not made, meaning the
    manifest was copied but could be unusable if the segments were not
    readable).
    
    With this change, there should be no change in functionality or existing
    behavior. This is asserted with (almost) no changes required to existing
    functional tests.
    
    Some notes (for operators):
    * Middleware required to be auto-inserted before slo and dlo and
      versioned_writes
    * Turning off server side copy is not configurable.
    * object_post_as_copy is no longer a configurable option of proxy server
      but of this middleware. However, for smooth upgrade, config option set
      in proxy server app is also read.
    
    DocImpact: Introducing server side copy as middleware
    
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    Co-Authored-By: Thiago da Silva <thiago@redhat.com>
    
    Change-Id: Ic96a92e938589a2f6add35a40741fd062f1c29eb
    Signed-off-by: Prashanth Pai <ppai@redhat.com>
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit cb8cda6588735411c542a1e4d78315ee047a357b
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Mon May 9 20:28:42 2016 +0000

resurrect gholt blog posts on building consistent hashing ring
    
    These are blog posts authored by Greg Holt (gholt) and used with
    permission to add here. Content was only reformatted as rst and
    wrap prose lines at 70 characters.
    
    Change-Id: I7aa47c24b5019aa598ee005e01612a49514da25f

commit ba1a568f815a8a40bf75c0beb96b789bd07adcb6
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 9 10:48:54 2016 +0000

Rename hash_cleanup_listdir tests
    
    hash_cleanup_listdir was removed in [1], this patch
    renames all references to it in test_diskfile to refer to
    the cleanup_ondisk_files method that is now tested directly.
    
    Also remove the final references to the now non-existent
    function in a few comments.
    
    [1] I0b96dfde32b4c666eebda6e88228516dd693ef92
    
    Change-Id: I1e151799fc2774de9a1af092afff875af24a630c
    Related-Bug: #1550569

commit 6827affe62771949a041d958db17ba6cd29232ab
Author: John Dickinson <me@not.mn>
Date:   Tue Apr 5 11:35:22 2016 -0700

Rework the contributor docs
    
    This started as a new "new_contributor" doc. But we've already got
    at least 3 different docs like that.
    
    Change-Id: Ia2303ab55eeea01cc71acbccaeab55dad0ef5ff9

commit deaef2f9d62bb8ae7f4e8538e9fd9ddffbb8ebb2
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed May 4 16:48:28 2016 -0700

Remove unneeded setting of SO_REUSEADDR.
    
    This has been in eventlet.listen() since version 0.15.
    
    Change-Id: Ib77b28231a2180f1ea082f356c4687c39681a6f7

commit b3dd6a5df1ac4a5265b330787abcd52c7e116208
Author: Christopher Bartz <bartz@dkrz.de>
Date:   Tue May 3 14:33:05 2016 +0200

Improves log message in swift.obj.replicator
    
    Before this commit, when a local device has not found been found
    in a object-replication run, the policy was not mentioned in the
    error log. But it is of interest to know the policy, for example for
    error searching, when no local device has been found.
    
    Change-Id: Icb9f9f1d4aec5c4a70dd8abdf5483d4816720418

commit 99305b93000f28530731c05b34c69e0c7ce0463f
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon May 2 17:29:32 2016 -0700

Fix probe tests from commit cf48e75
    
    Commit cf48e75 changed the default account/container/object ports in a
    lot of places, including the probetests. However, it didn't change
    them in doc/saio/bin/remakerings, and since the probe tests must match
    the rings, they started failing.
    
    This commit just backs out the changes to the test/probe directory so
    that remakerings and the probe tests match again.
    
    Change-Id: I316a09e6ee1a911f37ce9df3d641644739f88eeb

commit cf48e75c259350daa667155e3ffd38b17946bf44
Author: Shashirekha Gundur <shashirekha.j.gundur@intel.com>
Date:   Mon Feb 1 18:06:54 2016 +0000

change default ports for servers
    
    Changing the recommended ports for Swift services
    from ports 6000-6002 to unused ports 6200-6202;
    so they do not conflict with X-Windows or other services.
    
    Updated SAIO docs.
    
    DocImpact
    Closes-Bug: #1521339
    Change-Id: Ie1c778b159792c8e259e2a54cb86051686ac9d18

commit 4c11833a9cbff499725365e535e217f3eae3c442
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Apr 28 11:53:51 2016 -0500

Remove ThreadPool class
    
    With the removement of threads_per_disk there is no longer a need to use
    run_in_thread() at all; it was just calling the function itself when
    running with 0 threads.
    Similar to force_run_in_thread() - with 0 threads it was basically doing
    the same like in tpool_reraise(), therefore replacing the call and
    finally removing the complete ThreadPool class.
    
    Note that this might break external consumers that are inheriting
    BaseDiskFileManager; in this case you need to adopt this change in your
    codebase then.
    
    Change-Id: I39489dd660935bdbfbc26b92af86814369369fb5

commit 33fdd0a356d83f12f63b9a94ff423ca815c96066
Author: Bryan Keller <kellerbr@us.ibm.com>
Date:   Fri Apr 22 10:03:23 2016 -0700

SwiftLogFormatter will log transaction IDs on INFO level
    
    Previously SwiftLogFormatter would make two checks. One to see if the
    transaction id was already in the message field and another check to
    make sure the log level wasn't set to info. If either of these was
    true, then it would not log the transaction ID in the transaction ID
    field.
    
    This commit removes the check for the info log. Now transaction IDs
    will be recorded in all cases that have them.
    
    Change-Id: Ic06538ab55a75d298169ae1745671573ee9c09e8
    Closes-Bug: #1504344

commit c6ac69c552f87d07cd98b6e3f54258913bdc90db
Author: ZhiQiang Fan <aji.zqfan@gmail.com>
Date:   Fri Apr 29 20:39:02 2016 +0800

[Trivial] Remove unnecessary executable privilege
    
    swift/common/storage_policy.py is not required to be executable.
    
    Change-Id: I733d70a88be25b32d8caf590dee4c53d14757fef

commit 96c1838a15645639154f1048285d53a891809937
Author: Brian Cline <bcline@us.ibm.com>
Date:   Fri Apr 29 05:00:36 2016 -0500

Update mailmap, resolve identity crisis
    
    Change-Id: Ia8e30a8c0f81068c259d5d08adad4f1a98460561

commit a67d91987dff4859924aeaa3ca8fe30190b4c210
Author: Brian Cline <bcline@us.ibm.com>
Date:   Fri Apr 29 04:01:39 2016 -0500

Fix minor typos in review guidelines
    
    Change-Id: I3acb3daeacb784c4038bf54d4103fab66c52f41a

commit 9d6a055b31f267e2380d0470021cac44cdf36203
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Apr 28 12:06:24 2016 -0500

Remove threads_per_disk setting
    
    This patch removes the threads_per_disk setting. It was already a deprecated
    setting and by default set to 0, which effectively meant to not use a per-disk
    thread pool at all. Users are encouraged to use servers_per_port instead.
    
    DocImpact
    
    Change-Id: Ie76be5c8a74d60a1330627caace19e06d1b9383c

commit a1ed76f1cbb3dcd5b4253920278b8bc4f32978a2
Author: ZhiQiang Fan <aji.zqfan@gmail.com>
Date:   Thu Apr 28 06:29:41 2016 +0800

Fix doc build if git is absent
    
    When building packages if git is absent, then we should not set
    html_last_updated_fmt. It can still be set via the -D switch
    when building with sphinx-build.
    
    Change-Id: I5d0b6cc87f27a052d6d0265546c0d347f00c4bb8
    Closes-Bug: #1552251

commit 00dd89fe6936e4c2da667b496a9996638c26196b
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Mar 3 06:08:12 2016 +0000

adding review guidelines
    
    Change-Id: I61304856a4ecccbbf3aa06c30822494592a3b3d5

commit 29544a9e175b1ec9e0dbfd5288edc00a1402d5ca
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Apr 27 16:59:00 2016 -0500

Use smaller quorum size in proxy for even numbers of replicas
    
    Requiring 2/2 backends for PUT requests means that the cluster can't
    tolerate a single failure. Likewise, if you have 4 replicas in 2
    regions, requiring 3/4 on a POST request means you cannot POST with
    your inter-region link down or congested.
    
    This changes the (replication) quorum size in the proxy to be at least
    half the nodes instead of a majority of the nodes.
    
    Daemons that were looking for a majority remain unchanged. The
    container reconciler, replicator, and updater still require majorities
    so their functioning is unchanged.
    
    Odd numbers of replicas are unaffected by this commit.
    
    Change-Id: I3b07ff0222aba6293ad7d60afe1747acafbe6ce4

commit 5e420efc728b9f02a3f15eb312ebfa29e2e221ff
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jan 7 15:46:17 2016 -0800

Fix versioned_writes functional test skipping
    
    Previously, if object versioning was enabled via the old-style
    allow_versions container-server setting rather than the new-style
    allow_versioned_writes proxy-server setting,
    TestCrossPolicyObjectVersioning would skip tests while
    TestObjectVersioning and TestObjectVersioningUTF8 would run them.
    
    Additionally, if versioned_writes was explicitly included in the
    proxy-server's pipeline and allow_versioned_writes was disabled,
    the functional tests would fail with a 412.
    
    Now, all three will use the same logic to check whether versioning is
    enabled. Specifically, they will all try to set an X-Versions-Location
    header and skip if it doesn't stick.
    
    Additionally, the TestCrossPolicyObjectVersioningEnv will now properly
    clean up after itself.
    
    Change-Id: I4c788a0e18587ff17d3c6e346fd22b881495f06d

commit 1a18079a15c2630e602c7c156a30e10e5f884d4b
Author: Denis V. Meltsaykin <dmeltsaykin@mirantis.com>
Date:   Thu Apr 7 23:07:13 2016 +0300

Change pids' type to set
    
    Previously, the pids type was list(), which raised a ValueError in case if
    pid was not in pids during the pids.remove(pid) call. The fix changes
    the type of pids to set() and substitutes remove() with discard(), which
    doesn't raise an Exception if a value is not in the set.
    
    Change-Id: I8d41cb2a8ec14d29e5c7411ddfe48ae7a41deb45
    Closes-Bug: #1567638

commit 0da9da513101c172d8923469e40f9c15b3518e73
Author: Andy McCrae <andy.mccrae@gmail.com>
Date:   Thu Mar 3 11:14:39 2016 +0000

Allow fallocate_reserve to be a percentage
    
    Add the ability to set the fallocate_reserve value as a percentage.
    This happens automatically when adding the '%' at the end of the value.
    Having the ability to set a % of free space rather than a byte value is
    useful especially when drive sizes are heterogenous.
    
    The default for fallocate_reserve has been adjusted to 1%, having the
    fallocate_reserve set seems sensible for all deploys and percentages are
    far safer to default than byte values (across drives of any size).
    
    Tests added for using fallocate_reserve as a percentage.
    
    Duplicate tests for fallocate_reserve have been removed.
    
    Docs updated to reflect the fallocate_reserve change.
    
    Change-Id: I4aea613a708205c917e81d6b2861396655e73238

commit 91f980314f305c536e6e7c8607b8b52693c865f5
Author: John Dickinson <me@not.mn>
Date:   Wed Apr 13 11:57:45 2016 -0500

fix fallocate_reserve traceback
    
    Previously, fallocate_reserve could result in a traceback. The
    OSError being raised didn't have the proper errno set. This patch
    sets the errno to ENOSPC.
    
    Change-Id: I017b0584972ca8832f3b160bbcdff335ae9a1aa6

commit e97c4f794db7292a91135adb3fc3e1d6cce4dc84
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Sat Jan 16 21:59:20 2016 -0600

swift-[account|container]-info when disk is full
    
    Extended the use of the DatabaseBroker "stale_reads_ok" flag to the
    AccountBroker and ContainerBroker.  Now checks for an sqlite3 error
    from the _commit_puts call that processes the pending files.
    
    If this error is raised, then the stale_reads_ok flag will be checked
    to determine how to proceed as opposed to simply raising.
    
    The first time that print_info is attempted, the flag will be
    false, but swift-[account|container]-info will check for the
    raised exception.  If it was raised, then a warning is reported
    that the data may be stale, and another attempt will be
    made using the stale_reads_ok=True flag.
    
    Change-Id: I761526eef62327888c865d87a9caafa3e7eabab6
    Closes-Bug: 1531302

commit 936881543e60fa437cd0f91f029f9874fd7437d9
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Jan 14 18:31:21 2016 +0000

Reclaim isolated .meta files
    
    It is possible for an object dir to contain only
    a .meta file*, and currently this file will never
    be deleted. This patch changes the diskfile on disk
    file processing to identify any isolated meta files
    and mark them as ready to be reclaimed, so that the
    cleanup will reclaim them if they are older than
    reclaim age.
    
    *An isolated .meta can occur as follows:
    
    Consider two replicas of the same object whose ondisk files
    have diverged due to failures:
    
      A has t2.ts
      B has t1.data, t4.meta
    
    (The DELETE at t2 did not make it to B. The POST at t4 was
    rejected by A.)
    
    After ssync replication the two ondisk file sets will be:
    
      A has t2.ts
      B has t2.ts, t4.meta
    
    Once t2 becomes older than (current time - reclaim age) we are
    left with:
    
      B has t4.meta
    
    Closes-Bug: #1534685
    Change-Id: I3a175fd948a331d08df1b28f9831d7529216c6f1

tags:

added: in-feature-crypto

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-31: Fix proposed to swift (feature/hummingbird)

#5

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/323599

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-31: Fix merged to swift (feature/hummingbird)

#6

Download full text (84.7 KiB)

Reviewed: https://review.openstack.org/323599
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0330478b70d0a699a0f9c21ef87c7e639d92564b
Submitter: Jenkins
Branch: feature/hummingbird

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <email address hidden>
Date: Tue May 31 16:25:50 2016 +0200

Fixed typo

Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <email address hidden>
Date: Wed May 25 11:21:25 2016 -0700

Clarify overload best practices in admin guide

Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <email address hidden>
Date: Wed May 25 09:53:31 2016 +0200

Add dispersion --verbose example to admin guide

Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <email address hidden>
Date: Wed May 25 14:35:54 2016 +1000

Add ring-builder dispersion command to admin guide

    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.

Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <email address hidden>
Date: Tue May 24 13:37:58 2016 -0700

Add links to mitaka install guides

Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <email address hidden>
Date: Fri Apr 29 12:12:00 2016 -0500

Allow concurrent bulk deletes

    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.

    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.

    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.

    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:

delete_concurrency = 1

This may be done prior to upgrading Swift.

    UpgradeImpact
    Closes-Bug: 1524454
    Change-Id: I128374d74a4cef7a479b221fd15eec785cc4694a

commit 226557afc42c245e050d84162497f46341407ef7
Author: Tim Burke <email address hidden>
Date: Thu May 19 18:55:40 2016 -0700

Turn on H703, so our translators don't punch us

Change-Id: I4ce3068f79563e4d4296c6e1078bc12f0cf84c96
Related-Bug: 1559431

commit 7b706926a8ed5bbcec3a678e868e301c9a6ed8f1
Author: Alistair Coles <email address hidden>
Date: Mon May ...

Reviewed:  https://review.openstack.org/323599
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0330478b70d0a699a0f9c21ef87c7e639d92564b
Submitter: Jenkins
Branch:    feature/hummingbird

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Tue May 31 16:25:50 2016 +0200

Fixed typo
    
    Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed May 25 11:21:25 2016 -0700

Clarify overload best practices in admin guide
    
    Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed May 25 09:53:31 2016 +0200

Add dispersion --verbose example to admin guide
    
    Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Wed May 25 14:35:54 2016 +1000

Add ring-builder dispersion command to admin guide
    
    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.
    
    Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue May 24 13:37:58 2016 -0700

Add links to mitaka install guides
    
    Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Apr 29 12:12:00 2016 -0500

Allow concurrent bulk deletes
    
    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.
    
    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.
    
    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.
    
    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:
    
       delete_concurrency = 1
    
    This may be done prior to upgrading Swift.
    
    UpgradeImpact
    Closes-Bug: 1524454
    Change-Id: I128374d74a4cef7a479b221fd15eec785cc4694a

commit 226557afc42c245e050d84162497f46341407ef7
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu May 19 18:55:40 2016 -0700

Turn on H703, so our translators don't punch us
    
    Change-Id: I4ce3068f79563e4d4296c6e1078bc12f0cf84c96
    Related-Bug: 1559431

commit 7b706926a8ed5bbcec3a678e868e301c9a6ed8f1
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Mon May 23 15:20:06 2016 +0100

Fix setup of manifest responses in SLO tests
    
    The swift_bytes param is removed from the content-type
    in the proxy object controller, so the SLO unit tests should
    not be registering GET responses with FakeSwift that have
    swift_bytes appended to the content-type.
    
    Nor should submanifest segment dicts have swift_bytes appended to
    their content-type values.
    
    Also adds a test for the object controller and container server
    handling of SLO swift_bytes.
    
    Change-Id: Icf9bd87eee25002c8d9728b16e60c8347060f320

commit d0ec1adb78b26f5c24312090796c69912e9e3da9
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu May 19 19:58:56 2016 +0100

Make SLO manifest copies retain correct content-type
    
    When copying an SLO manifest with multipart-manifest=get the actual
    manifest content-type should get copied to the destination, rather
    than the application/json value that is synthesised by SLO in a GET
    response. That way the result of a HEAD on the copied manifest is the
    same as a HEAD to the source, and the container listings for the two
    are consistent.
    
    This patch also un-skips a functional test and adds functional tests
    that verify this patch and also verify that etags and size also get
    correctly copied and updated in destination container (bug #1260446).
    
    Closes-Bug: #1260446
    Closes-Bug: #1583756
    
    Change-Id: Ie7fa82f70b3ec3ef568f5355c69f6bce460ba25d

commit 1b7001adeb935893017ff2428291d7edf9470b6b
Author: Brian Cline <bcline@us.ibm.com>
Date:   Sat May 21 03:54:33 2016 -0500

Improve linting for third-party middleware
    
    Currently any third-party CI doing linting of their own middlewares
    against the Swift codebase don't do so hot -- if they use swob's
    Response.status_int in any way, the linter will fail because it only
    gets set when status does.
    
    This patch defaults it to None in the class definition, seeing as
    a linter can't truly know that status_int will only be created lazily
    at runtime, and further, that its existence is tied to setting
    Response.status (via __init__ or setting the property directly).
    
    I couldn't actually find a code path within Swift where we could end
    up with a None when we access it; at init time, we default the status
    code to 200, and there don't seem to be any code paths that init with
    a None status code. (You might try, but you can't do that even today.)
    As a result, this seems to have no ill effect on anything, but is a
    huge help for linters.
    
    This doesn't seem to break any unit tests, func tests via v1, v2, and
    v3 auth, nor tempest tests via v2 or v3.
    
    Change-Id: Id28fb56bc7133b9eb59df7989a71b892fd919643

commit 14eb1803e931c5c501d99e612a6d59a57eccf464
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu May 12 23:57:49 2016 -0700

Fix probe failure and small things
    
    This is follow up for https://review.openstack.org/#/c/283351/.
    
    Probe fix:
    - The probe in the patch now fails (sometimes success luckily)
      because inbound X-Timestamp is deprecated at the change,
      f581fccf71034818d19062593eeb52a4347bb174, so we can not use
      X-Timestamp to make an object with arbitrary timestamp anymore
      from outside of Swift. This patch makes the probe to use internal
      client to put the objects to make the inconsistent situation.
    
    Small things:
    
    - Enable expirer split brain test even if we have just one policy.
    - FAIL rather than ERROR if the object was expired incorrectly
    - ObjectBrainSplitter now uses the policy set at instance variable in
      default instead of random choice of ENABLED_POLICIES.
    
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    
    Change-Id: I757dbb0f1906932ef5d508b48b4120f2794b3d07

commit 9ae3ee5a8b6aa169e1f4179cee748c721f480229
Author: Alfredo Moralejo <amoralej@redhat.com>
Date:   Thu May 19 15:56:15 2016 +0200

Fix locale directory in MANIFEST.in
    
    Actual location is swift/locale, not locale.
    
    This makes build to not include .mo files if they have been created
    previously with compile_catalog.
    
    Change-Id: I907e9bd6dde57cb9ed718656e56a313e82bad128
    Closes-Bug: 1583618

commit 876df35f847e607901e2ed8a344c71c607495705
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Tue May 17 10:48:25 2016 +1000

disable_fallocate also disables fallocate_reserve
    
    Currently when disable_fallocate is true it disables calling the
    fallocate syscall, but it doesn't disable fallocate_reserve. This
    patch fixes this.
    
    This problem has caused functional tests to fail in our SAIOs, since
    SAIOs have disable_fallocate set but the fallocate_reserve space free
    checking was still being run creating 507 responses. This is thanks
    to the change in fallocate_reserve default changing from 0 to 1%.
    
    Because fallocate_reserve and disable_fallocate causes SAIO functional
    tests to fail a section called 'Known Issues' has been added to the
    SAIO developer documentation which includes a warning about
    using fallocate_reserve on SAIOs.
    
    Change-Id: I727bfb0861ea26fe2f16ad55f4d36ae088864d8f

commit 2744492f30c758a4f076e2b30eaaf1e2e7fa586c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Apr 27 13:31:11 2016 -0500

Use the same key for memcache and env['swift.infocache']
    
    When we were caching directly to the WSGI environment, it made sense to
    have different keys for the different caches. Now that we have a
    separate data structure for the per-request cache, however, we ought to
    be consistent.
    
    Change-Id: I199cba6e5fc9ab4205bba369e6a2f34fc5ce22d4

commit 4248123169f88bf62dbc5413ae2c15d53010482e
Author: venkatamahesh <venkatamaheshkotha@gmail.com>
Date:   Sun May 15 21:27:46 2016 +0530

Remove the invalid project link
    
    swiftsync project is no longer maintained and so
    is removed from the list
    
    Change-Id: Ieae67c728da16e5516babd33d2e5937b14738d6a
    Closes-Bug: #1581975

commit 1adc6047f0dc56caffb3ab552de948c720b50682
Author: Brian Ober <bober@us.ibm.com>
Date:   Fri May 13 16:43:50 2016 -0500

Bypass Account Mgmt Functional Cases When Disabled
    
    The testPUT case is failing when keystone was enabled and
    allow_account_management is set to True.   There were a few issues
    needing addressed.
    
    First the case was renamed to call out what it was actually doing which
    is verifying an error scenario for which a PUT on a storage account was
    not allowed.
    
    Second the case was running even when allow_account_management is
    enabled, which is incorrect.   It "accidently" works with TempAuth
    because it requires a reseller permission, so the Keystone failure here
    has more to do with not requiring a reseller permission to do a PUT on a
    storage account for which a user has an operator role on.
    
    The common sense fix here is to not execute this test case when
    allow_account_management is enabled.
    
    Change-Id: Id29f5ca48f92cd139535be7064107b8a61b02856

commit 1c88d2cb818cec07d12ac17be5820cdd769aea5d
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Feb 11 15:51:45 2016 -0800

Fix up get_account_info and get_container_info
    
    get_account_info used to work like this:
    
      * make an account HEAD request
    
      * ignore the response
    
      * get the account info by digging around in the request environment,
        where it had been deposited by elves or something
    
    Not actually elves, but the proxy's GETorHEAD_base method would take
    the HEAD response and cache it in the response environment, which was
    the same object as the request environment, thus enabling
    get_account_info to find it.
    
    This was extraordinarily brittle. If a WSGI middleware were to
    shallow-copy the request environment, then any middlewares to its left
    could not use get_account_info, as the left middleware's request
    environment would no longer be identical to the response environment
    down in GETorHEAD_base.
    
    Now, get_account_info works like this:
    
      * make an account HEAD request.
    
      * if the account info is in the request environment, return it. This
        is an optimization to avoid a double-set in memcached.
    
      * else, compute the account info from the response headers, store it
        in caches, and return it.
    
    This is much easier to think about; get_account_info can get and cache
    account info all on its own; the cache check and cache set are right
    next to each other.
    
    All the above is true for get_container_info as well.
    
    get_info() is still around, but it's just a shim. It was trying to
    unify get_account_info and get_container_info to exploit the
    commonalities, but the number of times that "if container:" showed up
    in get_info and its helpers really indicated that something was
    wrong. I'd rather have two functions with some duplication than one
    function with no duplication but a bunch of "if container:" branches.
    
    Other things of note:
    
      * a HEAD request to a deleted account returns 410, but
        get_account_info would return 404 since the 410 came from the
        account controller *after* GETorHEAD_base ran. Now
        get_account_info returns 410 as well.
    
      * cache validity period (recheck_account_existence and
        recheck_container_existence) is now communicated to
        get_account_info via an X-Backend header. This way,
        get_account_info doesn't need a reference to the
        swift.proxy.server.Application object.
    
      * both logged swift_source values are now correct for
        get_container_info calls; before, on a cold cache,
        get_container_info would call get_account_info but not pass along
        swift_source, resulting in get_account_info logging "GET_INFO" as
        the source. Amusingly, there was a unit test asserting this bogus
        behavior.
    
      * callers that modify the return value of get_account_info or of
        get_container_info don't modify what's stored in swift.infocache.
    
      * get_account_info on an account that *can* be autocreated but has
        not been will return a 200, same as a HEAD request. The old
        behavior was a 404 from get_account_info but a 200 from
        HEAD. Callers can tell the difference by looking at
        info['account_really_exists'] if they need to know the difference
        (there is one call site that needs to know, in container
        PUT). Note: this is for all accounts when the proxy's
        "account_autocreate" setting is on.
    
    Change-Id: I5167714025ec7237f7e6dd4759c2c6eb959b3fca

commit ce90a1e79e16472ff75d205c8d907a5991888b3a
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Jan 21 13:19:30 2016 -0800

Make info caching work across subrequests
    
    Previously, if you called get_account_info, get_container_info, or
    get_object_info, then the results of that call would be cached in the
    WSGI environment as top-level keys. This is okay, except that if you,
    in middleware, copy the WSGI environment and then make a subrequest
    using the copy, information retrieved in the subrequest is cached
    only in the copy and not in the original. This can mean lots of extra
    trips to memcache for, say, SLO validation where the segments are in
    another container; the object HEAD ends up getting container info for
    the segment container, but then the next object HEAD gets it again.
    
    This commit moves the cache for get_*_info into a dictionary at
    environ['swift.infocache']; this way, you can shallow-copy the request
    environment and still get the benefits from the cache.
    
    Change-Id: I3481b38b41c33cd1e39e19baab56193c5f9bf6ac

commit 30e39cc9fa6353be37cd3b5a515f09b42ceb3b20
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed May 4 12:02:07 2016 -0700

Skip SLO-reconciling probe test when SLO is off
    
    The probe test in question tries to make a manifest referencing
    unavailable objects and expects that to fail with a 400. If the SLO
    middleware is enabled, then it checks the segments, can't find some,
    and returns the 400. If it's disabled, the PUT succeeds and makes an
    object whose contents are some JSON blob. In the latter case, the
    probe test would fail because it expected a real SLO manifest but
    didn't find one.
    
    Now we skip the remainder of the test when we detect that SLO is not
    enabled.
    
    Change-Id: I3e7e8e98107608e675efc24156e703bc167458bb

commit 721c788b9a0ceb22ffac0b2c797890898096a9f4
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu May 12 18:57:31 2016 -0700

Import BrainSpliter directly in expirer probe
    
    Change-Id: Ib3cfe38d1b17c09c7248b5cbcfe4e2eadff7acd9

commit eda1b330f3cf7bf9dd643854d4a221b6f024884d
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Thu May 12 18:58:15 2016 +0000

fix docs word usage for large obj copy
    
    Change-Id: I2b6ca3f9277b344ff5b05a85304084a7edcbcb66

commit 6834547f668a4649f5a42b8a4edc43b844fd6bbe
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Fri Apr 22 14:55:35 2016 -0700

Clean up fallocate tests a little
    
    Change-Id: I01f1ad8ef0f8910718fd2fb30c9e8285358baf84

commit b430c384db38ea1153f52d0accf0561dac1378a6
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu May 12 14:57:56 2016 +0100

Put back missing sysmeta unit tests
    
    We lost some unit tests from test_sysmeta.py in [1].
    This patch restores them. Since the proxy no longer
    implements post-as-copy or COPY then we need to plumb
    in a copy middleware to the test app in test_sysmeta.
    
    It is a little odd perhaps to have these tests under
    test/unit/proxy but the alternative is to duplicate
    the test setup and base test code that already exists
    in test_sysmeta.py into test_copy.py.
    
    [1] commit 46d61a4dcd9a5d9157625c06d6fe7d916e80c3d2
        (server side copy middleware)
    
    Change-Id: Iec02387ccbddbe3841a417880389c707cd5c0346

commit ce022f4417e973215b563412b504a8a1b9ce240c
Author: Saverio Proto <saverio.proto@switch.ch>
Date:   Thu May 12 15:30:17 2016 +0200

README: fix broken link
    
    There is a broken link to CONTRIBUTING.rst
    introduced in commit a829bd59770681f9d6c1ef02a6e1d5e441587a23
    
    Change-Id: Iaedf5ff3995229cf292202793809080f9f2c7fed

commit 9729bc83eb6845e65f8244b4104b2c1678f8fe38
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Apr 11 11:56:07 2016 +0200

Don't delete misplaced dbs if not replicated
    
    If one uses only a single replica and a database file is placed on a
    wrong partition, it will be removed instead of replicated to the correct
    partition.
    
    There are two reasons for this:
    1. The list of nodes is empty when there is only a single replica
    2. all(responses) is True even if there is no response at all, and the
    latter is always True if there is no node to replicate to.
    
    This patch fixes this by adding a special case if used with only one
    replica to the node selection loop and ensures that the list of
    responses is not empty.  Also adds a test that fails on current master
    and passes with this change.
    
    Closes-Bug: 1568591
    
    Change-Id: I028ea8c1928e8c9a401db31fb266ff82606f8371

commit 92274d479010f691004b1885e354224dc9c3b61c
Author: Andreas Jaeger <aj@suse.com>
Date:   Mon Mar 28 18:21:22 2016 +0200

List system dependencies for running common tests
    
    Add an other-requirements.txt file containing a cross-platform list of
    dependencies needed for running included tox-based tests. Also include a
    tox environment for convenience calling the bindep[*] utility to list
    any missing system requirements.
    
    Document bindep and other-requirements usage.
    
    This change is self-testing.
    
    For other-requirements.txt see also
    http://docs.openstack.org/infra/manual/drivers.html#package-requirements
    
    [*] http://docs.openstack.org/infra/bindep/
    
    Change-Id: Iea6f5fecba3b7cb9f6dac7029c0f17fc31fc0e3c

commit ad16e2c77bb61bdf51a7d3b2c258daf69bfc74da
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed May 11 19:54:47 2016 -0700

Stop complaining about auditor_status files
    
    Following fd86d5a, the object-auditor would leave status files so it
    could resume where it left off if restarted. However, this would also
    cause the object-reconstructor to print warnings like:
    
      Unexpected entity in data dir: u'/srv/node4/sdb8/objects/auditor_status_ZBF.json'
    
    ...which isn't actually terribly useful or actionable. The auditor will
    clean it up (eventually); the operator doesn't have to do anything.
    
    Now, the reconstructor will specifically ignore those status files.
    
    Change-Id: I2f3d0bd2f1e242db6eb263c7755f1363d1430048