VM interface goes down because of short dhcp lease

I have the same issue, especially on windows guests, sometimes the DHCPREQUEST is sent too late and the DHCP server answers with a NAK because the lease is not found (lease is expired).

We are testing with a longer lease time (yes you have to overwrite the hard-coded value of 120 in linux_net.py).

Hopefully a longer lease time will fix this issue and the dhcp_release switch will be used to free up private IPs of terminated instances before the lease expired so we don't waste fixed ips.

I don't think we should work on a client-based solution as they are many OS involved, but a switch to overwrite the lease time would be effective and simple to implement.

FYI, dhcp_lease_type is configurable in essex, so you don't have to overwrite the hard coded value.

I guess you mean --dhcp_lease_time and not --dhcp_lease_type

I see the switch is actually referenced in diablo but simply not used for the dnsmasq command. If it's fixed in essex then all is well.

sorry, yes --dhcp_lease_time and it is actually used in essex. :)

Maybe this isn't a duplicate. The bug originally reported by PierreF is exactly what we experience. We raised the lease time slightly (so the --dhcp_lease_time does its job ) and still have issues with network interfaces going down on VMs under high load.
So the --dhcp_lease_time parameter does its job but doesn't solve the issue of why the VM don't receive a DHCPOFFER and then looses its interface. So ok we can raise the lease but the actual issue is probably a dnsmasq related bug.

Our syslog:
Jun 27 08:32:26 node1 dhclient: DHCPREQUEST of 192.168.1.13 on eth0 to 192.168.1.7 port 67
Jun 27 08:32:26 node1 dhclient: DHCPACK of 192.168.1.13 from 192.168.1.7
Jun 27 08:32:26 node1 dhclient: bound to 192.168.1.13 -- renewal in 407 seconds.
Jun 27 08:38:24 ---> from this time point we get a lot error lines of deamons that start complaining that network interface is down
-->only after only 3 min the VM requests a new IP but doesn't get offer back, We check the node running DNSmasq and it never seems to receive these last requests
Jun 27 08:41:15 pbsnode1 dhclient: DHCPREQUEST of 192.168.1.13 on eth0 to 192.168.1.7 port 67
Jun 27 08:41:27 pbsnode1 dhclient: DHCPREQUEST of 192.168.1.13 on eth0 to 192.168.1.7 port 67
Jun 27 08:44:53 pbsnode1 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67
Jun 27 08:44:53-->at this time the node with dns masq server receives the DHCPDISCOVER and send out a DHCPACK of 192.168.1.13 back, but this doesn't sam to reach the VM and it keeps sending out DHCPDISCOVER requests
Jun 27 08:53:38 pbsnode1 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Jun 27 08:53:46 pbsnode1 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 19
Jun 27 08:54:05 pbsnode1 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
Jun 27 08:54:09 pbsnode1 dhclient: No DHCPOFFERS received.

--> at this point we just give up and reboot the VM to get our network back

Tomiles, I think you are affected by the dnsmasq bug discussed in this email:
http://openstack.markmail.org/search/?q=Dhcp+lease+errors+in+vlan+mode#query:Dhcp%20lease%20errors%20in%20vlan%20mode+page:1+mid:7kjf4hljszpydsrx+state:results

The problems detailed in my comment above occurred despite the fact that we had already upgraded to 2.61.
We even start loosing our network interface under high load now despite having raised our lease time to 600000s.

can you check syslog for errors on the compute host? You might want to make sure you aren't losing packets due to conntrack tables being full etc. If you are seeing issues like that, it could be a bug in libvirt / qemu / virtio that needs to be reported upstream.

the compute hosts syslog doesn't contain any errors around the time-point of the crash.
We are beginning to think DHCP is not the source issue but just a consequence of the network interface going down.
But we are in the dark about any indication on what could possible trigger a VMs network interface to do down?
Usual suspect are glusterfs mounts in the VM but don't know if that can cause a network down.

I'm seeing something quite similar on my deployment (essex/precise). I have either network or VM stability problems (haven't nailed down which yet) when the network interface gets busy on the VM. The instance can be rebooted successfully and begins to work properly, modulo network load. The odd thing is that we only see this issue on machines with large amounts of memory. We don't see the same issue on hosts/instances with smaller memory footprints. It is unclear to me if this is an artifact of resource footprint or a difference between node hardware types.