OpenStack Object Storage (swift)

testsuite config needs fixing along with test_wsgi.py

Bug #1249727 reported by IAN DELANEY on 2013-11-10

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Object Storage (swift)	Fix Released	Undecided	Unassigned

Bug Description

ho hum.

>>> Source compiled.
Unable to read test config /etc/swift/test.conf - file not found

Oh dear, we need to INSTALL swift before we run the tests. eer, why?
So we have a testsuite in 2 distinct parts; those that run from the source code without installing, those that need a system installed /etc/swift/test.conf to run. This is a high level design choice, and this choice is at odds with gentoo style installing or 'emerging'. The tests that require the /etc/swift/test.conf are skipped. More on them later.

On running the testsuite with sh .unittests we get 2 distinct fails.
a) sh: warning: setlocale: LC_ALL: cannot change locale (eo)
b) test_wsgi.py failures. I'll go for pasting those here, short enough to avoid uploading a log file

The test_locale.py I just don't see its point. It appears to assume a hardcoded LC_ALL eo whatever eo is. At this end I'm just removing it on it being just something to trip over.

Ran 1552 tests in 130.470s

FAILED (errors=6)

======================================================================
ERROR: test_appconfig_dir_ignores_hidden_files (test.unit.common.test_wsgi.TestWSGI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/test/unit/common/test_wsgi.py", line 417, in test_appconfig_dir_ignores_hidden_files
    conf = wsgi.appconfig(conf_dir)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 107, in wrapper
    return f(conf_uri, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 261, in appconfig
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 86, in _loadconfigdir
    return loader.get_context(object_type, name, global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    pkg_resources.require(self.spec)
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 728, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: xattr>=0.4

======================================================================
ERROR: test_init_request_processor (test.unit.common.test_wsgi.TestWSGI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/test/unit/common/test_wsgi.py", line 141, in test_init_request_processor
    conf_file, 'proxy-server')
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 358, in init_request_processor
    (conf, logger, log_name) = _initrp(conf_path, app_section, *args, **kwargs)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 326, in _initrp
    (conf_path, e))
ConfigFileError: Error trying to load config from /mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/temp/tmpdQVoNY/proxy-server.conf: xattr>=0.4

======================================================================
ERROR: test_init_request_processor_from_conf_dir (test.unit.common.test_wsgi.TestWSGI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/test/unit/common/test_wsgi.py", line 184, in test_init_request_processor_from_conf_dir
    conf_dir, 'proxy-server')
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 358, in init_request_processor
    (conf, logger, log_name) = _initrp(conf_path, app_section, *args, **kwargs)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 326, in _initrp
    (conf_path, e))
ConfigFileError: Error trying to load config from /mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/temp/tmp5ptmQ7/proxy-server.conf.d: xattr>=0.4

======================================================================
ERROR: test_run_server (test.unit.common.test_wsgi.TestWSGI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/test/unit/common/test_wsgi.py", line 332, in test_run_server
    conf = wsgi.appconfig(conf_file)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 107, in wrapper
    return f(conf_uri, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 261, in appconfig
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 450, in get_context
    global_additions=global_additions)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 559, in _pipeline_app_context
    APP, pipeline[-1], global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    pkg_resources.require(self.spec)
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 728, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: xattr>=0.4

======================================================================
ERROR: test_run_server_conf_dir (test.unit.common.test_wsgi.TestWSGI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/test/unit/common/test_wsgi.py", line 379, in test_run_server_conf_dir
    conf = wsgi.appconfig(conf_dir)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 107, in wrapper
    return f(conf_uri, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 261, in appconfig
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 86, in _loadconfigdir
    return loader.get_context(object_type, name, global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 450, in get_context
    global_additions=global_additions)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 559, in _pipeline_app_context
    APP, pipeline[-1], global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.10.0/work/swift-1.10.0/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/usr/lib64/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    pkg_resources.require(self.spec)
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 728, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: xattr>=0.4

or is this also subsequent to seeking the missing system installed test.conf??

So on cd to the source, let's run nosetests test. It begins with

Unable to read test config /etc/swift/test.conf - file not found
Unable to read test config /etc/swift/test.conf - file not found
SKIPPING FUNCTIONAL TESTS DUE TO NO CONFIG
Unable to read test config /etc/swift/test.conf - file not found
Unable to read test config /etc/swift/test.conf - file not found

telling us a whole stack of prepared tests are being skipped.

/path/tp/swift-1.10.0 $ nosetests test
yields
Ran 1715 tests in 103.341s
FAILED (SKIP=177, errors=24)
which could mean anything

/path/to/swift-1.10.0 $ nosetests ./test/unit/obj/test_diskfile.py
yields
----------------------------------------------------------------------
Ran 39 tests in 2.061s

At my reckoning, at minumum test_wsgi.py needs fixing

See original description

IAN DELANEY (johneed) on 2013-11-10

description:	updated
description:	updated

Revision history for this message

IAN DELANEY (johneed) wrote on 2013-11-10:

being from prior version swift -1.9.1 this may raise NO interest

======================================================================
FAIL: test_get_logger_sysloghandler_plumbing (test.unit.common.test_utils.TestUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/mnt/gen2/TmpDir/portage/sys-cluster/swift-1.9.1/work/swift-1.9.1/test/unit/common/test_utils.py", line 438, in test_get_logger_sysloghandler_plumbing
syslog_handler_args)
AssertionError: Lists differ: [((), {'facility': 19, 'addres... != [((), {'facility': 19, 'addres...

Second list contains 1 additional elements.
First extra element 1:
((), {'facility': 19})

- [((), {'address': '/dev/log', 'facility': 19})]
+ [((), {'address': '/dev/log', 'facility': 19}), ((), {'facility': 19})]
? ++++++++++++++++++++++++

Revision history for this message

Tim Burke (1-tim-z) wrote on 2017-07-13:

> So we have a testsuite in 2 distinct parts

There are actually *three* parts:

* Unit tests, which don't require swift to be running, but which *do* require some level of installation -- in particular, our unit tests include some validation that entrypoints have been properly configured
* Functional tests, which require a functional swift (or swift-like) endpoint, defined in /etc/swift/test.conf
* Probe tests, which require a local swift instance that looks more-or-less like the documented Swift-All-In-One [1] -- this allows us to test a cluster during various failure scenarios

Looking at the unit tests reported:

* The "Unable to read test config /etc/swift/test.conf - file not found" message is awkward but benign -- the unit tests will still run whether that file's present or not. I'm really questioning the wisdom having unit tests that take config [2], though.

* The locale test should be fixed [3].

* The DistributionNotFound errors are legit; swift requires xattr, and plumbing in pkg_resources will enforce that.

* The test_get_logger_sysloghandler_plumbing failure is fixed [4].

[1] https://docs.openstack.org/swift/latest/development_saio.html
[2] As was introduced in https://github.com/openstack/swift/commit/f7fdb9c
[3] https://github.com/openstack/swift/commit/62b87fc
[4] Most likely in https://github.com/openstack/swift/commit/64756f8

Revision history for this message

Tim Burke (1-tim-z) wrote on 2017-07-13:

Kicked out https://bugs.launchpad.net/swift/+bug/1704192 as an isolated issue -- I think the rest has been fixed?

Changed in swift:
status:	New → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.