test_check_glance_v1_notifications failed in grenade run

Bug #1351627 reported by Matt Riedemann
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Undecided
gordon chung

Bug Description

http://logs.openstack.org/28/111328/2/check/check-grenade-dsvm-partial-ncpu/8f8231a/console.html

2014-08-02 00:45:44.544 | tempest.api.telemetry.test_telemetry_notification_api.TelemetryNotificationAPITestJSON.test_check_glance_v1_notifications[gate,image,smoke]
2014-08-02 00:45:44.544 | -------------------------------------------------------------------------------------------------------------------------------------------
2014-08-02 00:45:44.544 |
2014-08-02 00:45:44.544 | Captured traceback:
2014-08-02 00:45:44.544 | ~~~~~~~~~~~~~~~~~~~
2014-08-02 00:45:44.544 | Traceback (most recent call last):
2014-08-02 00:45:44.545 | File "tempest/test.py", line 128, in wrapper
2014-08-02 00:45:44.545 | return f(self, *func_args, **func_kwargs)
2014-08-02 00:45:44.545 | File "tempest/api/telemetry/test_telemetry_notification_api.py", line 59, in test_check_glance_v1_notifications
2014-08-02 00:45:44.545 | self.await_samples(metric, query)
2014-08-02 00:45:44.545 | File "tempest/api/telemetry/base.py", line 111, in await_samples
2014-08-02 00:45:44.545 | CONF.compute.build_timeout))
2014-08-02 00:45:44.545 | TimeoutException: Request timed out
2014-08-02 00:45:44.545 | Details: Sample for metric:image.update with query:('resource', 'eq', u'a920c93a-7220-4611-9fe1-54a5a89e3375') has not been added to the database within 196 seconds

message:"Details: Sample for metric:image.update with query:('resource', 'eq'" AND message:"has not been added to the database within" AND tags:"console"

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiRGV0YWlsczogU2FtcGxlIGZvciBtZXRyaWM6aW1hZ2UudXBkYXRlIHdpdGggcXVlcnk6KCdyZXNvdXJjZScsICdlcSdcIiBBTkQgbWVzc2FnZTpcImhhcyBub3QgYmVlbiBhZGRlZCB0byB0aGUgZGF0YWJhc2Ugd2l0aGluXCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MDY5OTIwMTc5NjYsIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0=

74 hits in 7 days.

I'm not seeing any useful traces in the ceilometer logs on the new side of the grenade run.

Joe Gordon (jogo)
Changed in ceilometer:
status: New → Confirmed
Revision history for this message
Matt Riedemann (mriedem) wrote :

Proposal to disable the test is here: https://review.openstack.org/#/c/111852/

Revision history for this message
gordon chung (chungg) wrote :

this was caused by poor performance of sql backends and instability in message queue performance

this was addressed in ceilometer with:

https://review.openstack.org/#/c/113524/ and
https://review.openstack.org/#/c/94483/ and
https://review.openstack.org/#/c/110151/ and
https://review.openstack.org/#/c/111313/

Changed in ceilometer:
status: Confirmed → Fix Released
assignee: nobody → gordon chung (chungg)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.