This was so impossibly annoying that I have to share the solution I found. In my case, it ended up being that nova.conf was not owned by nova. This is absurd, because the mode was 644, so it shouldn't have mattered in the slightest. Nonetheless, after over a week debugging this (going so far as to add custom debug code, strace'ing relevant processes, doing an md5sum comparison of a working host and a non-working one; I pulled out all the stops!), this simple change fixed it for me instantly.
This was so impossibly annoying that I have to share the solution I found. In my case, it ended up being that nova.conf was not owned by nova. This is absurd, because the mode was 644, so it shouldn't have mattered in the slightest. Nonetheless, after over a week debugging this (going so far as to add custom debug code, strace'ing relevant processes, doing an md5sum comparison of a working host and a non-working one; I pulled out all the stops!), this simple change fixed it for me instantly.