resolved always executes hooks

Bug #814987 reported by Clint Byrum
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
pyjuju
Fix Released
High
Kapil Thangavelu
juju (Ubuntu)
Fix Released
Undecided
Unassigned
Oneiric
Won't Fix
High
Unassigned

Bug Description

If I have uploaded a bad formula, I will need to fix my formula, and upgrade the formula.

I can't do that while the state is broken though. So the simplest way is to do an 'ensemble resolved' on the service units (though that would be kind of silly with 100 units, I think that is another bug entirely). However, when I do that, they re-run the config-changed hook, returning the state to broken.

config-changed should respect whether I ran resolved with --retry.

Related branches

Changed in ensemble:
milestone: none → dublin
importance: Undecided → Medium
Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

Was that an actual issue somehow?

It seems to me that this would be a nice default behavior. config-changed must necessarily be idempotent since it will be executed every time the configuration changes, which means reexecuting after a failure is a safe behavior.

I suggest we maintain this behavior, and reuse the same logic for all hooks we're sure are idempotent.

Revision history for this message
Clint Byrum (clint-fewbar) wrote : Re: [Bug 814987] Re: config-changed hook is retried on 'resolved' even when --retry is not passed

Excerpts from Gustavo Niemeyer's message of Sat Jul 23 21:15:41 UTC 2011:
> Was that an actual issue somehow?
>
> It seems to me that this would be a nice default behavior. config-
> changed must necessarily be idempotent since it will be executed every
> time the configuration changes, which means reexecuting after a failure
> is a safe behavior.
>
> I suggest we maintain this behavior, and reuse the same logic for all
> hooks we're sure are idempotent.

Yes, they should be idempotent, and I'd be in support of the default being
to retry, which I find myself almost always passing.

However, the only way to fix a broken config-changed hook with the
current behavior, without destroying the service, is:

* for each service unit
  * ssh in
    * manually change the config-changed hook so it will not error
  * run resolved
* upgrade-formula

So, for a sane way to fix broken formulas, we must either change the
default to match the draft doc here:

https://ensemble.ubuntu.com/docs/drafts/resolved.html

Which says

"After being resolved the unit would be in the started state. It important
to note that by default ensemble resolved does not fire hooks and the
start hook would not be invoked again as a result of the above."

*Or* change it, and provide a --no-retry. Or optionally, provide a
--ignore-error-state to upgrade-formula.

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote : Re: config-changed hook is retried on 'resolved' even when --retry is not passed

If I understand the issue correctly, the only thing that has to be changed is the draft doc so that it points out that
config-changed will retry by default because the hook is necessarily idempotent.

Revision history for this message
Clint Byrum (clint-fewbar) wrote : Re: [Bug 814987] Re: config-changed hook is retried on 'resolved' even when --retry is not passed

Excerpts from Gustavo Niemeyer's message of Mon Jul 25 17:43:07 UTC 2011:
> If I understand the issue correctly, the only thing that has to be changed is the draft doc so that it points out that
> config-changed will retry by default because the hook is necessarily idempotent.

Except that leaves us in a position where there is no way to fix a broken
hook except to destroy the service entirely, or go through the ssh and
manual intervention mess that I outlined before.

If the draft is changed, I'd suggest that a --no-retry must be added.

Changed in ensemble:
milestone: dublin → none
Revision history for this message
Kapil Thangavelu (hazmat) wrote : Re: config-changed hook is retried on 'resolved' even when --retry is not passed

hmm.. there's some missing discussion here that wasn't captured on a solution for this in the upgrade scenario.

Changed in ensemble:
milestone: none → eureka
Revision history for this message
Kapil Thangavelu (hazmat) wrote :

namely that we always execute config-changed after upgrade as well, afaicr, and we don't execute it when doing it without a retry.. and that we always execute it before starting.

Changed in ensemble:
importance: Medium → High
Changed in juju:
milestone: eureka → florence
Changed in juju:
assignee: nobody → Kapil Thangavelu (hazmat)
status: New → In Progress
Revision history for this message
Kapil Thangavelu (hazmat) wrote :

it looks like this discussion went down a side track. but in examining it was clear there was a more obvious problem, resolved was executing hooks, effectively due to a typo, it was checking for the hook retry value as a boolean value, when it was an enumerated one.

summary: - config-changed hook is retried on 'resolved' even when --retry is not
- passed
+ resolved always executes hooks
Changed in juju:
status: In Progress → Fix Released
Changed in juju (Ubuntu):
status: New → Fix Released
Changed in juju (Ubuntu Oneiric):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Rolf Leggewie (r0lf) wrote :

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".

Changed in juju (Ubuntu Oneiric):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.