rally verify configure-verifier --extend extra-options.conf converts options to lowercase

Bug #1877930 reported by Xing Zhang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Rally
Fix Released
Undecided
Xing Zhang

Bug Description

`rally verify configure-verifier --extend extra-options.conf` converts options to lower case which case some tests runs with unexpected result. Use `rally verify configure-verifier --show` to
see the extended config file.

Here is an example:

octavia-tempest-plugin has an option RBAC_test_type in load_balancer section:
[load_balancer]
RBAC_test_type = owner_or_admin

the option is defined here:
https://github.com/openstack/octavia-tempest-plugin/blob/a97deefcb3c21194ec1e2b4f11cc0fbd3c16b720/octavia_tempest_plugin/config.py#L99-L103

octavia-tempest-plugin gets this value by using oslo.conf:
https://github.com/openstack/octavia-tempest-plugin/blob/a97deefcb3c21194ec1e2b4f11cc0fbd3c16b720/octavia_tempest_plugin/tests/api/v2/test_availability_zone.py#L99

rally extend extra options by using configparser:
https://github.com/openstack/rally/blob/be72fbb2929c8a08b7a363dd94a6fd7a2c8aef1c/rally/cli/commands/verify.py#L335-L336

configparser converts options to lowercase:
https://docs.python.org/3/library/configparser.html

configparser dose not sensitive to lowercase or uppercase, but oslo.conf does:
$ python
Python 2.7.5 (default, Apr 2 2020, 13:16:51)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import configparser
>>> conf = configparser.ConfigParser()
>>> conf.read("extra-options.conf")
['extra-options.conf']
>>> conf['load_balancer'].keys()
[u'rbac_test_type']
>>> conf['load_balancer']['rbac_test_type']
u'owner_or_admin'
>>> conf['load_balancer']['RBAC_test_type']
u'owner_or_admin'
>>>

$ python
Python 2.7.5 (default, Apr 2 2020, 13:16:51)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from tempest import config
>>> CONF = config.CONF
>>> CONF.load_balancer.RBAC_test_type
'advanced'
>>> CONF.load_balancer.rbac_test_type
2020-05-11 14:20:32.483 2619 CRITICAL tempest [-] Unhandled error: NoSuchOptError: no such option rbac_test_type in group [load_balancer]
2020-05-11 14:20:32.483 2619 ERROR tempest Traceback (most recent call last):
2020-05-11 14:20:32.483 2619 ERROR tempest File "<stdin>", line 1, in <module>
2020-05-11 14:20:32.483 2619 ERROR tempest File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_config/cfg.py", line 3547, in __getattr__
2020-05-11 14:20:32.483 2619 ERROR tempest return self._conf._get(name, self._group)
2020-05-11 14:20:32.483 2619 ERROR tempest File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_config/cfg.py", line 3069, in _get
2020-05-11 14:20:32.483 2619 ERROR tempest value, loc = self._do_get(name, group, namespace)
2020-05-11 14:20:32.483 2619 ERROR tempest File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_config/cfg.py", line 3087, in _do_get
2020-05-11 14:20:32.483 2619 ERROR tempest info = self._get_opt_info(name, group)
2020-05-11 14:20:32.483 2619 ERROR tempest File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_config/cfg.py", line 3263, in _get_opt_info
2020-05-11 14:20:32.483 2619 ERROR tempest raise NoSuchOptError(opt_name, group)
2020-05-11 14:20:32.483 2619 ERROR tempest NoSuchOptError: no such option rbac_test_type in group [load_balancer]
2020-05-11 14:20:32.483 2619 ERROR tempest
>>>

Xing Zhang (xingzhang)
description: updated
Revision history for this message
Andriy Kurilin (andreykurilin) wrote :

Looks valid. `conf.optionxform = str ` should be used before reading the file to prevent case transformation.
Can you propose a fix?

Xing Zhang (xingzhang)
description: updated
Xing Zhang (xingzhang)
Changed in rally:
assignee: nobody → Xing Zhang (xingzhang)
Xing Zhang (xingzhang)
summary: - rally verify configure-verifier --extend extra-options.conf convert
- options and values to lowercase
+ rally verify configure-verifier --extend extra-options.conf converts
+ options to lowercase
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to rally (master)

Fix proposed to branch: master
Review: https://review.opendev.org/727096

Changed in rally:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to rally (master)

Reviewed: https://review.opendev.org/727096
Committed: https://git.openstack.org/cgit/openstack/rally/commit/?id=e1ad05c781fdabd013d81277b459caf6db1a4279
Submitter: Zuul
Branch: master

commit e1ad05c781fdabd013d81277b459caf6db1a4279
Author: Xing Zhang <email address hidden>
Date: Tue May 12 15:50:39 2020 +0800

    Make config parser case sensitivity in configure-verifier

    Options in config file should be case sensitive, some projects
    like octavia-tempest-plugin can't set some value as expected:
    [load_balancer]
    RBAC_test_type = owner_or_admin

    Use `conf.optionxform = str` to prevent case transformation[1].

    [1] https://docs.python.org/3/library/configparser.html

    Change-Id: I34d85b835aaa7a3737577b75ec9e6f2b02f90ead
    Closes-Bug: #1877930

Changed in rally:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.