Rally

[rally 1.5.0] rally deployment create --fromenv creates wrong spec for openstack platform

Bug #1829030 reported by Chuck Short
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Rally
Fix Released
High
Andriy Kurilin

Bug Description

019-05-10 22:30:15.113522 | primary | TASK [browbeat/rally : Check Rally deployment] *********************************
2019-05-10 22:30:15.128243 | primary | Friday 10 May 2019 22:30:15 +0000 (0:00:03.388) 0:03:50.427 ************
2019-05-10 22:30:18.197656 | primary | fatal: [undercloud]: FAILED! => {
2019-05-10 22:30:18.197860 | primary | "changed": true,
2019-05-10 22:30:18.198085 | primary | "cmd": ". /home/zuul/browbeat/.rally-venv/bin/activate; . /home/zuul/overcloudrc; rally deployment check",
2019-05-10 22:30:18.198181 | primary | "delta": "0:00:02.472996",
2019-05-10 22:30:18.198287 | primary | "end": "2019-05-10 22:30:18.176989",
2019-05-10 22:30:18.198358 | primary | "rc": 1,
2019-05-10 22:30:18.198469 | primary | "start": "2019-05-10 22:30:15.703993"
2019-05-10 22:30:18.198563 | primary | }
2019-05-10 22:30:18.198595 | primary |
2019-05-10 22:30:18.198632 | primary | STDOUT:
2019-05-10 22:30:18.198658 | primary |
2019-05-10 22:30:18.198813 | primary | --------------------------------------------------------------------------------
2019-05-10 22:30:18.198864 | primary | Platform openstack:
2019-05-10 22:30:18.198980 | primary | --------------------------------------------------------------------------------
2019-05-10 22:30:18.198997 | primary |
2019-05-10 22:30:18.199062 | primary | Error while checking admin credentials:
2019-05-10 22:30:18.199114 | primary | TypeError: Bad admin creds:
2019-05-10 22:30:18.199131 | primary | {
2019-05-10 22:30:18.199177 | primary | "api_info": {
2019-05-10 22:30:18.199221 | primary | "keystone": {
2019-05-10 22:30:18.199308 | primary | "service_type": "identityv3",
2019-05-10 22:30:18.199363 | primary | "version": 3
2019-05-10 22:30:18.199387 | primary | }
2019-05-10 22:30:18.199409 | primary | },
2019-05-10 22:30:18.199475 | primary | "auth_url": "http://10.0.0.5:5000",
2019-05-10 22:30:18.199581 | primary | "domain_name": null,
2019-05-10 22:30:18.199632 | primary | "endpoint_type": null,
2019-05-10 22:30:18.199675 | primary | "https_cacert": "",
2019-05-10 22:30:18.199716 | primary | "https_cert": "",
2019-05-10 22:30:18.199766 | primary | "https_insecure": false,
2019-05-10 22:30:18.199805 | primary | "https_key": "",
2019-05-10 22:30:18.199847 | primary | "password": "***",
2019-05-10 22:30:18.199899 | primary | "profiler_conn_str": null,
2019-05-10 22:30:18.199968 | primary | "profiler_hmac_key": null,
2019-05-10 22:30:18.200038 | primary | "project_domain_name": "Default",
2019-05-10 22:30:18.200106 | primary | "region_name": "regionOne",
2019-05-10 22:30:18.200193 | primary | "tenant_name": "admin",
2019-05-10 22:30:18.200262 | primary | "user_domain_name": "Default",
2019-05-10 22:30:18.200307 | primary | "username": "admin"
2019-05-10 22:30:18.200349 | primary | }
2019-05-10 22:30:18.200383 | primary |
2019-05-10 22:30:18.200406 | primary |
2019-05-10 22:30:18.200428 | primary | MSG:
2019-05-10 22:30:18.200444 | primary |
2019-05-10 22:30:18.200533 | primary | non-zero return code
2019-05-10 22:30:18.216117 | primary |

Root cause:

2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr [-] Plugin <email address hidden>() method is broken: TypeError: {u'https_key': u''}
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr Traceback (most recent call last):
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr File "/home/vagrant/venv/lib/python2.7/site-packages/rally/env/env_mgr.py", line 523, in get_info
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr info = p.info()
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr File "/home/vagrant/venv/lib/python2.7/site-packages/rally_openstack/platforms/existing.py", line 253, in info
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr for stype, name in osclients.Clients(active_user).services().items():
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr File "/home/vagrant/venv/lib/python2.7/site-packages/rally_openstack/osclients.py", line 942, in services
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr available_services = self.keystone.service_catalog.get_endpoints()
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr File "/home/vagrant/venv/lib/python2.7/site-packages/rally_openstack/osclients.py", line 899, in __getattr__
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr self.cache)
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr File "/home/vagrant/venv/lib/python2.7/site-packages/rally_openstack/osclients.py", line 113, in __init__
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr self.credential = oscred.OpenStackCredential(**self.credential)
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr File "/home/vagrant/venv/lib/python2.7/site-packages/rally_openstack/credential.py", line 34, in __init__
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr raise TypeError("%s" % kwargs)
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr TypeError: {u'https_key': u''}
2019-05-14 20:19:59.071 15912 ERROR rally.env.env_mgr

See original description
Chuck Short (zulcss)
summary: - Regression with rally 1.5.0
+ Regression with rally 1.5.0 checking deployment
Andriy Kurilin (andreykurilin)
Changed in rally:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Andrey Kurilin (andreykurilin)
Andriy Kurilin (andreykurilin)
Changed in rally:
status: Triaged → In Progress
Andriy Kurilin (andreykurilin)
summary: - Regression with rally 1.5.0 checking deployment
+ [rally 1.5.0] rally deployment create --fromenv creates wrong spec for
+ openstack platform
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to rally (master)

Reviewed: https://review.opendev.org/659197
Committed: https://git.openstack.org/cgit/openstack/rally/commit/?id=14a8b0f931424d42543c100c9deb409eaded86d3
Submitter: Zuul
Branch: master

commit 14a8b0f931424d42543c100c9deb409eaded86d3
Author: Andrey Kurilin <email address hidden>
Date: Tue May 14 18:06:57 2019 -0700

    Release 1.5.1 with a fix for os deployment create from env

    **rally deployment create* --fromenv* creates wrong spec for
    rally-openstack<=1.4.0

    Closes-Bug: #1829030
    Change-Id: I61588eaa070b10b780c92d54d1b9439e8fd3f6b6

Changed in rally:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/rally 1.5.1

This issue was fixed in the openstack/rally 1.5.1 release.

Revision history for this message
pini (pkomarov) wrote :

Hi, But without openstack-rally ver > 1.4.0 The fix will be working . so That's a dependency right ?
Tested with :

(rally) [stack@undercloud-0 rally]$ pip freeze|grep rally
rally==1.5.1
rally-openstack==1.4.0

TASK [Check Rally deployment]

[...]

"stdout_lines": ["-------------------------------------------------------------------------------", "Platform openstack:", "-------------------------------------------------------------------------------", "", "Error while checking admin credentials:", "\tTypeError: Bad admin creds: ", "{", " \"api_info\": {", " \"keystone\":
{", " \"service_type\": \"identityv3\", ", " \"version\": 3", " }

", " }, ", " \"auth_url\": \"https://10.0.0.101:13000/v3\", ", " \"domain_name\": null, ", " \"endpoint_type\": null, ", " \"https_cacert\": \"\", ", " \"https_cert\": \"\", ", " \"https_insecure\": true, ", " \"https_key\": \"\", ", " \"password\": \"***\", ", " \"profiler_conn_str\": null, ", " \"profiler_hmac_key\": null, ", " \"project_domain_name\": \"Default\", ", " \"region_name\": \"\", ", " \"tenant_name\": \"admin\", ", " \"user_domain_name\": \"Default\", ", " \"username\": \"admin\"", "}"]}

NO MORE HOSTS LEFT ****************************************************************************************************
to retry, use: --limit @/root/.infrared/plugins/rally/main.retry

PLAY RECAP ************************************************************************************************************
undercloud-0 : ok=27 changed=14 unreachable=0 failed=1

ERROR Playbook "/root/.infrared/plugins/rally/main.yml" failed!

Revision history for this message
Andriy Kurilin (andreykurilin) wrote :

@pini: Did you update the deployment config?

Revision history for this message
pini (pkomarov) wrote : Re: [Bug 1829030] Re: [rally 1.5.0] rally deployment create --fromenv creates wrong spec for openstack platform
Download full text (6.0 KiB)

can you post what you mean by updating the deployment config ..
what parameters and what command line ?
is it just adding the https_key var to overcloudrc ?

On Thu, May 16, 2019, 19:41 Andrey Kurilin <email address hidden> wrote:

> @pini: Did you update the deployment config?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1829030
>
> Title:
> [rally 1.5.0] rally deployment create --fromenv creates wrong spec for
> openstack platform
>
> Status in Rally:
> Fix Released
>
> Bug description:
> 019-05-10 22:30:15.113522 | primary | TASK [browbeat/rally : Check Rally
> deployment] *********************************
> 2019-05-10 22:30:15.128243 | primary | Friday 10 May 2019 22:30:15
> +0000 (0:00:03.388) 0:03:50.427 ************
> 2019-05-10 22:30:18.197656 | primary | fatal: [undercloud]: FAILED! => {
> 2019-05-10 22:30:18.197860 | primary | "changed": true,
> 2019-05-10 22:30:18.198085 | primary | "cmd": ".
> /home/zuul/browbeat/.rally-venv/bin/activate; . /home/zuul/overcloudrc;
> rally deployment check",
> 2019-05-10 22:30:18.198181 | primary | "delta": "0:00:02.472996",
> 2019-05-10 22:30:18.198287 | primary | "end": "2019-05-10
> 22:30:18.176989",
> 2019-05-10 22:30:18.198358 | primary | "rc": 1,
> 2019-05-10 22:30:18.198469 | primary | "start": "2019-05-10
> 22:30:15.703993"
> 2019-05-10 22:30:18.198563 | primary | }
> 2019-05-10 22:30:18.198595 | primary |
> 2019-05-10 22:30:18.198632 | primary | STDOUT:
> 2019-05-10 22:30:18.198658 | primary |
> 2019-05-10 22:30:18.198813 | primary |
> --------------------------------------------------------------------------------
> 2019-05-10 22:30:18.198864 | primary | Platform openstack:
> 2019-05-10 22:30:18.198980 | primary |
> --------------------------------------------------------------------------------
> 2019-05-10 22:30:18.198997 | primary |
> 2019-05-10 22:30:18.199062 | primary | Error while checking admin
> credentials:
> 2019-05-10 22:30:18.199114 | primary | TypeError: Bad admin creds:
> 2019-05-10 22:30:18.199131 | primary | {
> 2019-05-10 22:30:18.199177 | primary | "api_info": {
> 2019-05-10 22:30:18.199221 | primary | "keystone": {
> 2019-05-10 22:30:18.199308 | primary | "service_type":
> "identityv3",
> 2019-05-10 22:30:18.199363 | primary | "version": 3
> 2019-05-10 22:30:18.199387 | primary | }
> 2019-05-10 22:30:18.199409 | primary | },
> 2019-05-10 22:30:18.199475 | primary | "auth_url": "
> http://10.0.0.5:5000",
> 2019-05-10 22:30:18.199581 | primary | "domain_name": null,
> 2019-05-10 22:30:18.199632 | primary | "endpoint_type": null,
> 2019-05-10 22:30:18.199675 | primary | "https_cacert": "",
> 2019-05-10 22:30:18.199716 | primary | "https_cert": "",
> 2019-05-10 22:30:18.199766 | primary | "https_insecure": false,
> 2019-05-10 22:30:18.199805 | primary | "https_key": "",
> 2019-05-10 22:30:18.199847 | primary | "password": "***",
> 2019-05-10 22:30:18.199899 | primary | "profiler_conn_str": null,
> 2019-05-10 22:30:18.199968 | primary | ...

Read more...

Revision history for this message
Andriy Kurilin (andreykurilin) wrote :

Sorry, I meant creation. Does it just `rally deployment create --fromenv` ?

PS: the fix doesn't affect old deployments, only new ones.

Revision history for this message
pini (pkomarov) wrote :

yes rally was deployed on a fresh deployment

Revision history for this message
Waldemar Znoinski (wznoinsk) wrote :

hi @Andrey,
just to shed a bit of light on how we reproduce this problem:

1. we deploy a fresh installation of openstack (undercloud/overcloud if you like)
2. we get to testing of it that is:

a) Create Rally deployment
/home/stack/rally/bin/rally db ensure
/home/stack/rally/bin/rally deployment create --fromenv --name cloud_under_test

b) create rally tasks json
/home/stack/rally_repo/rally-jobs/mytest.json

c) check rally deployment
/home/stack/rally/bin/rally deployment check

and we're hitting this bug with rally-openstack 1.4.0 but we don't see the problem with rally-openstack 1.4.1

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.