HA failure when no IP address is bound to the VIP interface

Proxying from juju ML:

We've been working on setting up an Openstack cluster on Trusty for a few months now using Juju and MAAS, although we've yet to go into production. I had everything working fine, including HA deployments of Keystone, Glance, Percona etc.

The older versions of the charms supported HA using the config settings vip, vip_cidr and vip_iface. Without me making any modifications to these charms, I successfully deployed all of the above charms with the bog-standard hacluster charm.

Over the weekend I've been updating to Juno, and I naturally updated to the latest stable charms from the Charm store. Breaking changes have been introduced to these charms such that they no longer support my deployment. My Openstack cluster promptly broke in a nasty way. I'm *really* glad this isn't a production environment, but these kinds of non-backward compatible breakages do give me cause for concern going forward.

To explain how this broke, I'll first need to explain how our network was deployed:

    In order to not burn through many public IPs, we assign RFC1918 IPs to *every server* by DHCP.
    We run at least two instances of critical services
    Public IPs are assigned primarily by Pacemaker
    Public and Private subnets coexist on a single Layer-2 network.
    Nodes that do not directly participate in the Public subnet still have direct access (not via a router) to the Public IPs courtesy of the DHCP option (rfc3442-classless-static-routes). It turns out that a Linux hosts in different subnets can directly communicate with one-another on the same layer-2 network without the need of a router.

This set-up was highly efficient in terms of consumption of valuable public IP addresses, without forcing inter-subnet communications via an unnecessary hop. The only trick that we had to pull-off was getting the DHCP server to give out the rfc3442-classless-static-routes, which was simple.

The old OpenStack charms with their simple vip, vip_cidr and vip_iface options worked perfectly with this set-up. The new charms cannot support this at all, as they have become, in my view, "too clever". They now insist that the vip can only be bound to an interface that already has an IP in the same subnet.

If I have to bind public IPs to every server (IPs that they will never use) just in order to have Pacemaker assign the vip, I'll burn through a lot of IPs in the most pointless way imaginable.

I've modified the keystone and openstack-dashboard charms to re-introduce the old functionality in a way that doesn't break the new multiple-IP functionality. I'll paste my keystone patch below to give you an idea what I think is needed. This hasn't been thoroughly tested, but it seems to work. Pacemaker can at least set the public IP address again.

If there is some other (better) way to achieve the same level of IP address allocation efficiency and performance without patching the Openstack charms, please point me in the right direction.

Thanks,
John