Bridge network device showing lots of dropped packets

Bug #787055 reported by Joeman1
44
This bug affects 9 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Expired
Undecided
Unassigned

Bug Description

After upgrading to 10.10 server from 10.04.02, I started noticing dropped packets on my br0 interface. After upgrading to 11.04 from 10.10, the problem still exists.

Kernel version where problem started - -2.6.38-8-server
Still relevant in kernel version from kernel PPA - 2.6.39-0-server

Kernel version where problem is NOT happening - 2.6.35-28-server

Software:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=11.04
DISTRIB_CODENAME=natty
DISTRIB_DESCRIPTION="Ubuntu 11.04"

System summery:

White box server running only KVM software. Server hosts 3 web servers.

System Information
        Manufacturer: Gigabyte Technology Co., Ltd.
        Product Name: GA-880GMA-UD2H

AMD Phenom(tm) II X6 1055T Processor

             total used free shared buffers cached
Mem: 8061336 3711396 4349940 0 258832 280444
-/+ buffers/cache: 3172120 4889216
Swap: 16787836 0 16787836

Network Hardware

Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 05)

cat /proc/interrupts
           CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
  0: 128 44 2758 289929 100775180 346 IO-APIC-edge timer
  1: 0 0 0 2 2 0 IO-APIC-edge i8042
  7: 1 0 0 0 0 0 IO-APIC-edge
  8: 0 0 0 0 1 0 IO-APIC-edge rtc0
  9: 0 0 0 0 0 0 IO-APIC-fasteoi acpi
 17: 0 0 1 15 48295 0 IO-APIC-fasteoi ehci_hcd:usb1, ehci_hcd:usb2, ehci_hcd:usb3, pata_jmicron
 18: 0 0 0 0 3 0 IO-APIC-fasteoi ohci_hcd:usb4, ohci_hcd:usb5, ohci_hcd:usb6, ohci_hcd:usb7, radeon
 19: 0 0 0 0 18 0 IO-APIC-fasteoi hda_intel
 20: 0 0 17 4757 529962 4 IO-APIC-fasteoi eth0
 41: 1 10 133 28385 4054049 43 PCI-MSI-edge ahci
 42: 0 0 0 0 1 0 PCI-MSI-edge xhci_hcd
 43: 0 0 0 0 0 0 PCI-MSI-edge xhci_hcd
 44: 0 0 0 0 0 0 PCI-MSI-edge xhci_hcd
 45: 0 0 0 0 0 0 PCI-MSI-edge xhci_hcd
 46: 0 0 0 0 0 0 PCI-MSI-edge xhci_hcd
 47: 0 0 0 0 0 0 PCI-MSI-edge xhci_hcd
 48: 0 0 0 0 0 0 PCI-MSI-edge xhci_hcd
NMI: 0 0 0 0 0 0 Non-maskable interrupts
LOC: 37306453 38833463 20554815 9666635 625754 7237306 Local timer interrupts
SPU: 0 0 0 0 0 0 Spurious interrupts
PMI: 0 0 0 0 0 0 Performance monitoring interrupts
IWI: 0 0 0 0 0 0 IRQ work interrupts
RES: 3313411 2954176 2396793 1528554 1852748 1007937 Rescheduling interrupts
CAL: 60321 49239 42056 19256 70396 17751 Function call interrupts
TLB: 322724 325409 292683 165751 334534 184039 TLB shootdowns
TRM: 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0 Machine check exceptions
MCP: 786 786 786 786 786 786 Machine check polls
ERR: 1
MIS: 0

ifconfig (Soon after reboot):

br0 Link encap:Ethernet HWaddr 00:08:c7:aa:92:6c
          inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:68537 errors:0 dropped:2335 overruns:0 frame:0
          TX packets:22216 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8763373 (8.7 MB) TX bytes:5082699 (5.0 MB)

eth0 Link encap:Ethernet HWaddr 00:08:c7:aa:92:6c
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:268424 errors:0 dropped:0 overruns:0 frame:0
          TX packets:228706 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:91508949 (91.5 MB) TX bytes:95147428 (95.1 MB)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:66 errors:0 dropped:0 overruns:0 frame:0
          TX packets:66 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:32357 (32.3 KB) TX bytes:32357 (32.3 KB)

vnet0 Link encap:Ethernet HWaddr fe:54:00:61:15:1f
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:3148 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52965 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:408953 (408.9 KB) TX bytes:4509499 (4.5 MB)

vnet1 Link encap:Ethernet HWaddr fe:54:00:5a:8d:89
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:569447 errors:0 dropped:0 overruns:0 frame:0
          TX packets:621884 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:128272930 (128.2 MB) TX bytes:264737686 (264.7 MB)

vnet2 Link encap:Ethernet HWaddr fe:54:00:18:38:7a
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:450782 errors:0 dropped:0 overruns:0 frame:0
          TX packets:527257 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:279408844 (279.4 MB) TX bytes:143002988 (143.0 MB)

netstat -s
Ip:
    31690 total packets received
    0 forwarded
    0 incoming packets discarded
    25843 incoming packets delivered
    21277 requests sent out
Icmp:
    16 ICMP messages received
    0 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 10
        echo requests: 2
        echo replies: 4
    19 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 13
        echo request: 4
        echo replies: 2
IcmpMsg:
        InType0: 4
        InType3: 10
        InType8: 2
        OutType0: 2
        OutType3: 13
        OutType8: 4
Tcp:
    43 active connections openings
    426 passive connection openings
    7 failed connection attempts
    0 connection resets received
    1 connections established
    19759 segments received
    19286 segments send out
    49 segments retransmited
    0 bad segments received.
    1 resets sent
Udp:
    2147 packets received
    3 packets to unknown port received.
    0 packet receive errors
    2157 packets sent
UdpLite:
TcpExt:
    1 invalid SYN cookies received
    7 resets received for embryonic SYN_RECV sockets
    34 TCP sockets finished time wait in fast timer
    1569 delayed acks sent
    1 delayed acks further delayed because of locked socket
    Quick ack mode was activated 26 times
    704 packets directly queued to recvmsg prequeue.
    97 bytes directly received in process context from prequeue
    9106 packet headers predicted
    2470 acknowledgments not containing data payload received
    5959 predicted acknowledgments
    1 times recovered from packet loss by selective acknowledgements
    2 timeouts after SACK recovery
    1 fast retransmits
    38 other TCP timeouts
    26 DSACKs sent for old packets
    3 DSACKs received
    TCPDSACKIgnoredOld: 1
    TCPDSACKIgnoredNoUndo: 1
    TCPSackMerged: 2
    TCPSackShiftFallback: 7
IpExt:
    InMcastPkts: 2459
    InBcastPkts: 1459
    InOctets: 7020174
    OutOctets: 4782729
    InMcastOctets: 78688
    InBcastOctets: 283460

What Do I Expect To Happen?
I expect system to no drop network packets

What Happened Instead?
As you can see, server is dropping quite a few network packets.

Please let me know if you need any additional information.

NOTE: As a test, I installed 3 other machines (2 HP Server Class (DL380 and DL580) and One Laptop (Asus)) with similar installed software, but not from upgrade; fresh installed. All machines had same problem on br0 interface.
---
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.23.
AplayDevices: Error: [Errno 2] No such file or directory
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D3p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info: Error: [Errno 2] No such file or directory
Card0.Amixer.values: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 11.04
HibernationDevice: RESUME=UUID=a6c0a462-6473-4585-b9d3-ad317b1f5121
InstallationMedia: Ubuntu-Server 10.04.2 LTS "Lucid Lynx" - Release amd64 (20110211.1)
MachineType: Gigabyte Technology Co., Ltd. GA-880GMA-UD2H
Package: linux (not installed)
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-2.6.38-8-server root=UUID=682cb361-703d-43a1-95a9-2838e204d6ee ro ipv6.disable=1 quiet
ProcVersionSignature: Ubuntu 2.6.38-8.42-server 2.6.38.2
RelatedPackageVersions:
 linux-restricted-modules-2.6.38-8-server N/A
 linux-backports-modules-2.6.38-8-server N/A
 linux-firmware 1.52
RfKill:

Tags: natty
Uname: Linux 2.6.38-8-server x86_64
UnreportableReason: This is not a genuine Ubuntu package
UpgradeStatus: Upgraded to natty on 2011-05-20 (3 days ago)
UserGroups:

dmi.bios.date: 09/30/2010
dmi.bios.vendor: Award Software International, Inc.
dmi.bios.version: F5
dmi.board.name: GA-880GMA-UD2H
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.modalias: dmi:bvnAwardSoftwareInternational,Inc.:bvrF5:bd09/30/2010:svnGigabyteTechnologyCo.,Ltd.:pnGA-880GMA-UD2H:pvr:rvnGigabyteTechnologyCo.,Ltd.:rnGA-880GMA-UD2H:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvr:
dmi.product.name: GA-880GMA-UD2H
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

Revision history for this message
Fabio Marconi (fabiomarconi) wrote :

Hello
Can you please, from the affected kernel run
apport-collect -p linux 787055
Thanks
Fabio

affects: ubuntu → linux (Ubuntu)
Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Joeman1 (jgiles) wrote : AcpiTables.txt

apport information

tags: added: apport-collected natty
description: updated
Revision history for this message
Joeman1 (jgiles) wrote : AlsaDevices.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : BootDmesg.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : Card0.Codecs.codec.0.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : IwConfig.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : Lspci.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : Lsusb.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : PciMultimedia.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : ProcCpuinfo_.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : ProcModules.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : UdevDb.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : UdevLog.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote : WifiSyslog.txt

apport information

Revision history for this message
Joeman1 (jgiles) wrote :

Ok, uploaded the report.

Please let me know if you need any further information.

Thanks!
Joe

Changed in linux (Ubuntu):
status: Incomplete → New
Brad Figg (brad-figg)
Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Joeman1 (jgiles) wrote :

Thanks, Brad, for confirming the bug...

Is there any more information you need from my system, or, are you ok to start working on it? Have you been able to reproduce?

Thanks!
Joe

Revision history for this message
Dusty Baker (dmbake88) wrote :

Just thought I would chime in and letting you know that this is also affecting me. Wireless seems fine but wired packets are dropping inbound. I don't loose link and wireshark is still sending ping to the gateway ip with no response when this happens. It is inconsistent for me but when running a long ping I'm getting anywhere from 2% to 10% packet loss.

Revision history for this message
penalvch (penalvch) wrote :

Joeman1, thank you for reporting this and helping make Ubuntu better. This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu? ISO CD images are available from http://cdimage.ubuntu.com/releases/ .

If it remains an issue, could you run the following command from a Terminal (Applications->Accessories->Terminal). It will automatically gather and attach updated debug information to this report.

apport-collect -p linux <replace-with-bug-number>

Also, if you could test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

tags: added: needs-upstream-testing regression-release
removed: br0 dropped network packets
Changed in linux (Ubuntu):
status: Confirmed → Incomplete
tags: added: maverick
Revision history for this message
Joeman1 (jgiles) wrote :

Hi Chris,

Yes, well, this issue was reported a year ago and there has been no interest from Canonical on releasing a fix or even acknowledging that there was an issue at all despite other having the same problem and even reporting it here.

Until Canonical understands the needs of the enterprise community and started paying attention to issues that come up, its going to be a long time before they are a billion dollar company or even close to it.

That said, we have moved our infrastructure to RHEL where we get much snappier support; even from the community members.

Sorry it didn't work out; however, we still have a few desktops running Kubuntu, but seems like even that version doesn't mean too much to Canonical these days... Pitty...

You can close out this ticket - unless the others that reported the issue have moved on too. Might want to do your best to keep what existing customers you have left.

Good day!

Joe

Revision history for this message
Tamas Papp (tompos) wrote :

Still exists with 12.04.

Or 10.04 and backport kernel.

Please try to fix this.

Revision history for this message
penalvch (penalvch) wrote :

Tamas Papp, please execute the following at the Terminal and feel free to subscribe me to it:
ubuntu-bug linux

Thanks!

Revision history for this message
Tamas Papp (tompos) wrote :

it's done: #986043 .

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]

Changed in linux (Ubuntu):
status: Incomplete → Expired
Revision history for this message
Henry Scott (henryscott) wrote :

I am not understanding the process here. This was submitted as a bug, has not been fixed, but was closed. I am evaluating Ubuntu to use as a sniffer, but if I am understanding this process correctly, things don't get fixed in this distro. Am I wrong? Was this fixed somewhere and I don't see it? My implementation is also dropping packets. As it is, this is not useful.

Revision history for this message
penalvch (penalvch) wrote :

Henry Scott, this bug is not closed, but is Status Expired. For more on this, please see https://wiki.ubuntu.com/Bugs/Status .

Despite this, could you please file a new report by executing the following in a terminal:
ubuntu-bug linux

For more on this, please see the Ubuntu Bug Control and Ubuntu Bug Squad article:
https://wiki.ubuntu.com/Bugs/BestPractices#X.2BAC8-Reporting.Focus_on_One_Issue

and Ubuntu Community article:
https://help.ubuntu.com/community/ReportingBugs#Bug_Reporting_Etiquette

When opening up the new report, please feel free to subscribe me to it. Thank you for your understanding.

Helpful Bug Reporting Links:
https://help.ubuntu.com/community/ReportingBugs#A3._Make_sure_the_bug_hasn.27t_already_been_reported
https://help.ubuntu.com/community/ReportingBugs#Adding_Apport_Debug_Information_to_an_Existing_Launchpad_Bug
https://help.ubuntu.com/community/ReportingBugs#Adding_Additional_Attachments_to_an_Existing_Launchpad_Bug

Revision history for this message
David Favor (davidfavor) wrote :

This problem still persists as of quantal 12.10 with the symptom of slow packet loss.

Suggest someone update this ticket with specific instructions about how to...

1) determine the protocol where packets are dropping

2) isolate the process where packets are dropping

ifconfig simply shows dropped RX packets for the adapter. More specifics are required to
determine the packet drop source.

Thanks.

Revision history for this message
penalvch (penalvch) wrote :

David Favor, if you have a bug in Ubuntu, could you please file a new report by executing the following in a terminal:
ubuntu-bug linux

For more on this, please see the Ubuntu Kernel team article:
https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Filing_Kernel_Bug_reports

the Ubuntu Bug Control team and Ubuntu Bug Squad team article:
https://wiki.ubuntu.com/Bugs/BestPractices#X.2BAC8-Reporting.Focus_on_One_Issue

and Ubuntu Community article:
https://help.ubuntu.com/community/ReportingBugs#Bug_reporting_etiquette

When opening up the new report, please feel free to subscribe me to it.

Please note, not filing a new report may delay your problem being addressed as quickly as possible.

Thank you for your understanding.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.