hieradata change doesn't reapply manifest

I had a chat with Dan and the options could be summarised as:

1) Similar to UpdateIdentifier, add an ApplyConfigIdentifier so that puppet-apply gets run when a unique value is set
2) digest all the hieradata deployments in the heat templates and feed that into the PostDeployment resource input_values so puppet-apply gets run whenever there is a hieradata change
3) replace the heat-config 55-heat-config with a dedicated orc script which builds the hieradata, digests it all, and decides whether to run puppet-apply.

Which one to do depends on what the actual use-cases are for deployers/operators choosing when to run puppet-apply. They might *want* to do a series of stack-updates which change hieradata, then do a puppet-apply update at the end. Or they might *never* want to do a manual update, and just want it to happen automatically always.

If it is the former, I'd recommend 1), otherwise I'd recommend 3).

2) might be quicker than 3) to implement, but it may result in harder to maintain templates.

The problem with (3) AFAICS is that even if we reapply the manifest when the digest changes, we have lost the ability to signal back to heat if that apply fails. Or are you suggesting that the Deployment creating the hieradata would fail as it cascades and applies all-the-manifests on update?

That's why I'm preferring (2) so we create an explicit dependency inside the templates, which should mean all deployment resources actually see an update when the hieradata changes, thus we enter the normal update cycle and potentially get more granular feedback about what actually failed in the event some manifest apply falls over with the new values?

I'm seeing more issues with 3) too.

I'll start playing with 2) as part of my work to involve puppet in package updates.

The challenge with 2) is that currently deployment resources do not expose any digestable attributes. Some possible solutions are:

a) In heat implement and backport a derived_config_id attribute on deployment resources
b) Replace os-apply-config config resources with a dedicated hieradata config resource, and use a custom orc script to do the hiera merging. This script signals back either the derived_config_id or a digest of all hieradata

Then 2) can be implemented digesting the required attribute from all the hiera deployment resources.

I think a) is worth trying, if we can justify it as a bugfix backport

OK, here is a plan, let me know what you think:

- write a heat-templates/hot/software-config/elements/heat-config-puppet/os-refresh-config/configure.d/45-heat-config-puppet-hiera which is very similar to 50-heat-config-docker-compose [1]. It will perform the same function as the hiera element (40-hiera-datafiles and oac hiera.yaml) for config resource which have group: puppet-hiera (instead of group: os-apply-config). It is responsible for ensuring that /etc/puppet/hiera.yaml and /etc/puppet/hieradata reflect exactly what has been deployed from heat every time os-refresh-config is run (and before puppet apply is triggered)

- write a heat-templates/hot/software-config/elements/heat-config-puppet/install.d/hook-puppet-hiera.py which does nothing except signal the output derived_config_id for all deployed group: puppet-hiera configs

- modify the templates to switch to group: puppet-hiera for all hiera configs

- modify the templates to build a digest of all derived_config_id values and feed that into the post deployment resource so that puppet apply is triggered whenever any hiera data changes

- once everything is switched over, delete the tripleo-puppet-elements hiera element

[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-docker-compose/os-refresh-config/configure.d/50-heat-config-docker-compose

This looks like it makes sense, and dprince was talking about wanting to rewrite the hiera element anyway - as a bonus this approach removes one of the last remaining dependencies on oac in the templates.

I'll try to dig into the details later and see how it works in practice.

Fix proposed to branch: master
Review: https://review.openstack.org/190282

Ok, so I spent some time looking in to the approach Steve suggests in comment #5 - that may still be the best long-term approach, but it started getting time-consuming, so I took the efficient/lazy engineer approach and started looking for a simpler way..

My alternate plan is a simplified variation on Steve's:

1. Modify 99-refresh-completed to know about a new group "orc-hiera-datafiles", and if we find that group, calculate a hash of the hieradata on the node and send it back via the deploy_stdout part of the signal payload (currently we're not passing any valid deployment signal data at all via 99-refresh-completed)

2. Update the templates to use get_attr to reference the deploy_stdout and switch the hiera SoftwareConfig groups to orc-hiera-datafiles

This to me seems like a quicker and less risky interim solution, I've already implemented (1), (2) is still todo (I've tested the get_attr with a simple minimal template on devstack though).

Fix proposed to branch: master
Review: https://review.openstack.org/191146