OpenStack Heat

Circular dependency when using Ceilometer alarms

Bug #1315048 reported by Qiming Teng
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Invalid
Medium
Unassigned

Bug Description

There are two 'AWS::CloudWatch::Alarm' we can choose from by customizing /etc/heat/environment.d/default.yaml:

    #AWS::CloudWatch::Alarm": "file:///etc/heat/templates/AWS_CloudWatch_Alarm.yaml"
    "AWS::CloudWatch::Alarm": "OS::Heat::CWLiteAlarm"

If the template resource (former one) is chosen, we cannot create a stack using the following template:

https://review.openstack.org/#/c/83924/3/tempest/scenario/orchestration/test_ha.yaml

The error message is:

ERROR: Remote error: CircularDependencyException Circular Dependency Found: {AWS::CloudWatch::Alarm "ha_alarm": {Restarter "restarter"}, Restarter "restarter": {Server "instance1"}, Server "instance1": {AWS::CloudWatch::Alarm "ha_alarm"}}

It means instance1 depends on ha_alarm, which depends on the restarter, which depends on instance1 again.

We don't have this problem when using the 'OS::Heat::CWLiteAlarm' implementation.

Thomas Herve (therve)
summary: - Circular dependency when using Ceilometer alrams
+ Circular dependency when using Ceilometer alarms
Revision history for this message
Steven Hardy (shardy) wrote :

Can you please confirm if this still happens now that the fix for bug #1316842 has landed?

Steven Hardy (shardy)
Changed in heat:
status: New → Incomplete
Revision history for this message
Qiming Teng (tengqim) wrote :

Just checked, shardy. The bug is still there:

 heat stack-create -f test_ha_new.yaml -P "key_name=qmkey;image=F20-heat;flavor=m1.small;server_name=Server1" TestHA
ERROR: Remote error: CircularDependencyException Circular Dependency Found: {AWS::CloudWatch::Alarm "ha_alarm": {Restarter "restarter"}, Restarter "restarter": {Server "instance1"}, Server "instance1": {AWS::CloudWatch::Alarm "ha_alarm"}}

Changed in heat:
status: Incomplete → New
Revision history for this message
Rabi Mishra (rabi) wrote :

IMO, this is a valid error with circular dependency.

You don't see this issue with "OS::Heat::CWLiteAlarm" as 'strict_dependency = False' set for this resource.

https://github.com/openstack/heat/blob/master/heat/engine/resources/cloud_watch.py#L130

 If you remove that you'll see the same error. Interestingly this is the __only__ resource with that setting.

However, this toggle doesn't work with CeilometerAlarm. I 'll debug it further.

Revision history for this message
Rabi Mishra (rabi) wrote :

I further narrowed down the problem why 'strict_dependency' does not work with 'CeilometerAlarm'. In the parent stack, nested/provider stack resource setting for 'strict_dependency'(default resource setting) is being used for finding dependency. if you set 'strict_dependency = False' in 'stack_resource.py', you would be able to avoid this issue.

However, as mentioned earlier, this is a valid error.

Revision history for this message
Rabi Mishra (rabi) wrote :

Also, IMO, this template is trying a valid use-case that can be achieved with SoftwareConfig (which configures the heartbeat script).

Revision history for this message
Angus Salkeld (asalkeld) wrote :

I'll look at this, see if it is a bug in heat or that template.

Changed in heat:
importance: Undecided → Medium
Revision history for this message
Angus Salkeld (asalkeld) wrote :

This is an artifact of a poor initial implementation:(

AWS CW does not need the 'AlarmName' reference. Basically you need to remove the --watch option on non-clould-watch-lite
templates.

Some references:
https://github.com/openstack/heat-cfntools/blob/master/bin/cfn-push-stats#L251-L268
https://github.com/asalkeld/autoscaling-testing/blob/master/AWS_CW_cfn_push_stats/template.yaml#L108
https://github.com/openstack/heat/blob/master/heat/engine/watchrule.py#L269-L295

I'll mark this as invalid.

Changed in heat:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.