Network issues with hardy guests/hosts, using default nic and virtio

Apparently solved by https://launchpad.net/%7Ekirkland/+archive/ppa/+files/kvm_62+dfsg-0ubuntu9~ppa1_i386.deb - doing more tests right now.

I think I have a handle on this problem. It involves dropping a bad patch. I have test packages in my PPA for Hardy and Intrepid. And this should be solved already in Jaunty.

Anyone who can give a hand testing--that would be great!

:-Dustin

The patch seems not to solve the issue I am experiencing with virtio. The problem with virtio seems to appear only on SMP guests though.

Yann, could you please provide a more detailed test case? Is this local traffic between guest and host or general net traffic? Can you reproduce it on different systems? Dragomir tells me he has tried but not managed to reproduce it. Thanks!

Hi Yann-

Can you please confirm that you cannot reproduce this at all with 1-cpu guests in Hardy?

In some conversations with upstream, it seems that kvm-62's SMP code was really immature, and should not be expected to work.

I'm going to try and get permission to backport kvm-84 to Hardy. I'll provide it in a PPA for testing.

:-Dustin

If you're running Hardy, could you please test this using the kvm-84 package in this PPA:
 * https://launchpad.net/~ubuntu-virt/+archive/ppa

Does this solve the issue, or is it reproducible?

:-Dustin

I tried and successfully reproduced the issue with single processor in Hardy 64-bit host and 32-bit guest. [1] Next I tried installation of the PPA, and couldn't been able to start the VM after that (screenshot attached). After removing the PPA packages [2] and installing the original Hardy packages back [3] guest host is able to boot and run. Is there something I need to do to be able to run the VM with PPA packages?

[1]
eth0 Link encap:Ethernet HWaddr 52:54:00:92:98:b0
          inet addr:192.168.1.173 Bcast:192.168.1.255 Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe92:98b0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:769793 errors:60 dropped:116 overruns:0 frame:0
          TX packets:715790 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:773865207 (738.0 MB) TX bytes:770534040 (734.8 MB)
          Interrupt:11 Base address:0xc100

[2] With PPA enabled removing the packages PPA upgrades:
apt-get remove kvm libvirt-bin libvirt0 python-libvirt

[3] With PPA disabled instaling the original Hardy packages:
apt-get install kvm libvirt-bin libvirt0 python-libvirt python-virtinst virt-manager virt-viewer

I worked with Dragomir in IRC...

Turns out he needed to upgrade his kvm (userspace) as well as kvm-source (kernel module) packages. Once he did this, he's able to boot his vm's fine.

Now, it seems, he back to testing this as a potential fix for this issue.

Cheers,
:-Dustin

After the installation of kernel module (apt-get install kvm-source) the VM boot successfully. The followed tests show no network issue with PPA installed KVM 84.

eth0 Link encap:Ethernet HWaddr 52:54:00:92:98:b0
          inet addr:192.168.1.173 Bcast:192.168.1.255 Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe92:98b0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:1539018 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1442132 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1546810177 (1.4 GB) TX bytes:1541910564 (1.4 GB)
          Interrupt:11 Base address:0xc100

The test was made twice: with the single and with dual CPU. Same result for the dual CPU - problem is gone:

eth0 Link encap:Ethernet HWaddr 52:54:00:92:98:b0
          inet addr:192.168.1.173 Bcast:192.168.1.255 Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe92:98b0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:1540407 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1481222 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1546120498 (1.4 GB) TX bytes:1544483216 (1.4 GB)
          Interrupt:11 Base address:0xc100

Okay, I've nominated this bug for release in hardy. I'm starting to collect bugs that would support a backport of kvm-84 to hardy.

Dragomir-

I need you to do one more thing, please... Can you please test this same setup on a Jaunty host? I do *not* expect this problem to be present there. In which case, I'd like to close the main bug, and leave open the bug against hardy....

:-Dustin

Done. The same Hardy image was used in Jaunty host, and everything is working as expected.

Marking fix released for Jaunty. Still in progress for Hardy.

:-Dustin

This bug should be fixed in the kvm-84 backport to hardy and intrepid. If you are suffering from this bug on those platforms, please try enabling the backports repository and installing that package.

I'm having a (similar?) problem running KVMs on a Hardy server. Packets are intermittently dropped somewhere between layers 2 and 3, i.e. they are captured arriving at the network interface by tcpdump but they don't get recv'd by socket code or logged with iptables. Is this problem related somehow to Bug #332156? The KVMs have been upgraded to Karmic btw -- I don't know if that makes any difference. Details are below:

Physical Server:
Linux oghma 2.6.29 #3 SMP Tue Apr 14 18:17:50 EDT 2009 x86_64 GNU/Linux

[Ubuntu-Server 8.04.1 _Hardy Heron_ - Release amd64 (20080701)]

Dual quad-core processors 0-7 w/ 8 GB ram
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz
stepping : 6
cpu MHz : 1994.655
cache size : 6144 KB
...

KVM Instances:
Linux airnet-1 2.6.31-22-server #63-Ubuntu SMP Thu Aug 19 01:25:01 UTC 2010 x86_64 GNU/Linux

Running Ubuntu-Server Karmic & iptables v1.4.4

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.9.1
stepping : 3
cpu MHz : 1994.655
cache size : 2048 KB
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx
 lm up rep_good pni hypervisor
bogomips : 3989.31
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: