RTL8102EL NIC driver oops during Hardy boot

We can't with this issue to boot the system to do the ODM testing in May. It's urgent for us.

And we actually also tried cut down the NIC chip from MB and installed 8.04 without any problem.

Steve - the dmesg that you have attached appears to be garbled. The first line makes no sense 'Linux version 2.6.2-1-generic'. It ought to be ' Linux version 2.6.24-16-generic'.

Are you using a serial device or something to capture the console? The stack trace is similarly confusing in that it references symbols that do not exist, e.g.,

[<f8abe6>] rtl86_init_one+0xc4/0xe9 [r86]

I cannot find that function anywhere. I suspect it is supposed to be rtl8169_init_one().

Steve - could you PCI identifiers for this NIC? Is the motherboard capable of using MSI?

Yes, it's rtl18169_init_one().
Actually I believe it's a kernel issue that we lost the 2nd digit of each 2 digits. That means if there is "2.6.24-16-generic" printed by printk, it will appear on the monitor as "2.6.2-1-generic". I duplicate this lost digit issue on our RTSed platform with AHCI on, but I can't find digit lose on the same system with ATA on. It's kind of strange.

And could you please give me more information about what is MSI? I don't know MSI.
And do you know in current Kernel code, does Ubunut use PCIBIOS or PCI direct configuration? I suspect that resource is not correct allocated for this NIC.

Also, by looking at r8169.c, the problem happens after rtl8169_init_one() calls pci_set_master.

Steve: I wanted the PCI IDs for the ethernet chipset.

MSI is described here: http://lwn.net/Articles/44139/

There are various flags you can pass at kernel boot time that allow you to control the PCI BIOS stuff. You can read about them in the kernel source code under Documentation/kernel-parameters.txt. Look for "pci=option". The interesting choices look like 'pci=bios' or 'pci=nobios'.

We have tried both pci=bios and pci=nobios before and can find this problem all the time.
We will vistit ODM tomorrow, and will get pci ids then. And we also try the beta driver from Realtek. For this beta driver, we need to check if it's OK to be integrate into upstream kernel.

Steve - I think somehow we are going to have to instrument and debug this issue. Somebody with kernel development abilities is going to have to get their hands on this hardware in order to figure out why the PCI code is crashing.

In the meantime, there are a couple of other PCI options you could try, e.g., nomsi and noacpi. I know its time consuming, but you can also play with these parameters in combination, e.g., a comma separated list such as pci=nobios,nomsi, etc.

Tim -, We tried all the PCI options but still get the same problem. And today, we tried the source code from RealTek, and it can successfully loaded in hardy. Here is the steps we take

1. Install Ubuntu 7.10. NIC is recogonized as rtl8169 and driver is loaded but not functional.
2. Upgrade to 8.04 by online update
3. Now you have 8.04 boot menu in Grub. But you can't boot from it due to the this issue
4. build Realtek driver under 7.10 by using 8.04 head files and modules
5. extract 8.04 initrd.img and replace r8169.ko with r8101.ko and pack initrd.img back
6. replace r8169.ko with r8101.ko in /lib/modules
7. Reboot with 8.04 and now we can successfully boot 8.04 and NIC can work normally.

Here is pci ids.
04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8101E PCI Express Fast Ethernet controller [10ec:8136] (rev 02)

I am checking if Realtek can provide source code and allow source code be integrated to upstream kernel driver.

SRU Justification:

Impact: r8169 driver crashes during boot.

Patch description: The PCI device pointer is initialized prior to be dereferenced.

Patch: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=commit;h=222947b1ea81041269639ddb2033f252a5e0ed25

TEST CASE: Certain motherboards with an r8169 ethernet device crash on boot.

http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=commit;h=222947b1ea81041269639ddb2033f252a5e0ed25

For the ubuntu8.04 install issue ,issue fixed with this solution. The steps is:
1. Put the following line to the file: /etc/apt/source.list.

deb http://ppa.launchpad.net/timg-tpi/ubuntu hardy main
deb-src http://ppa.launchpad.net/timg-tpi/ubuntu hardy main

2. run Synaptic with internet connection.
3. Search for kernel 2.6.24-17. And install related package:
linux-headers-2.6.24-17
linux-headers-2.6.24-17-386
linux-headers-2.6.24-17-generic
linux-image-2.6.24-17-386
linux-image-2.6.24-17-generic
linux-image-debug-2.6.24-17-386
linux-image-debug-2.6.24-17-generic

4. Then reboot system and press ESC
5. Choose the 17-generic item from list for booting
6, the system boot is OK.

Tim,

Thank you for the fix. We finally get the HW in Dell and Scott help to verify your fix on this issue with our prototype finally and proved to pass the NIC test. You can go on to officially publish it according to your process. Just one question to you, how will you provide the fix to us, by using NIC driver deb to fix both initrd.img and modules or just include it in kernel update?

Steve,

This will end up available in the 8.04.1 release to users later this July. If you will need a solution sooner, perhaps Tim can get you something for a driver disk or maybe another live disk.

Regards,

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Steve Zhang
Sent: Wednesday, May 14, 2008 9:00 AM
To: <email address hidden>
Subject: [Bug 223656] Re: RTL8102EL NIC chip cause Ubuntu 8.04 LiveCD boot upfail fail

Tim,

Thank you for the fix. We finally get the HW in Dell and Scott help to
verify your fix on this issue with our prototype finally and proved to
pass the NIC test. You can go on to officially publish it according to
your process. Just one question to you, how will you provide the fix to
us, by using NIC driver deb to fix both initrd.img and modules or just
include it in kernel update?

--
RTL8102EL NIC chip cause Ubuntu 8.04 LiveCD boot up fail fail
https://bugs.launchpad.net/bugs/223656
You received this bug notification because you are a member of The Dell
Team, which is a direct subscriber.

SRU Justification:

Impact: Some models of RTL8102EL NIC cause kernel oops during module load.

Patch Description: Perform PCI initialization earlier in load sequence.

Patch: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=commit;h=222947b1ea81041269639ddb2033f252a5e0ed25

TEST CASE: See bug description.

Accepted into -proposed, please test and give feedback here

Tim,

Just saw this bug, I've reported my "live cd" problem by updating Bug#32123 with [probably] too much detail. I wound up looking at my initramfs looping and should have come to you guys.

I encountered problems booting my new Vostro 200 [dual core Intel 2180]. Appears to be with DVD/hard drive controls failing to negotiate speed [I think]. [BUG#32123 has some details]. The only NIC on my Dell is wireless.

I'm new to ubuntu, but with speed of new hardware, do some 'items' need to be single threaded on stack? Just a thought. Regards, Bob

I'm also having problems on a Vostro 200. It worked OK for a while (several weeks, occasional usage) but has now become unbootable.

During the attempt, it drops to BusyBox but still keeps trying, producing lots of errors (but I'm not familiar with how to save them -- maybe a digital camera of the screen will help). It reports a failure in initramfs, with messages about ata1:00 and so on.

The system is standard from Dell -- 2 GB RAM, no floppy, media readers (SD cards etc). On-board NIC, no wireless or PCI cards.

I saw a suggestion to adjust the grub boot with "all_generic_ide" and this allows it to boot and run fine.

The alternate installer installs fine, but the reboot fails.

The standard installer fails.

The live-CD mode also fails.

This is not a very active system so I can try stuff on it if you wish.

Sounds like we bought the same system; and, I too plan to develop/play on this Vostro!

Your grub circumvent suggestion works fine, but I only rebooted 3 times! Let me know if you need help [I used text editor and have a terminal log of pertinent changes]. Otherwise, I'm using Linksys USB adapter fine and have Dell 926 [known as a Linux "paperweight"] .

dmesg can give [too] much info but I debug it from watching the initramfs loop while ? determines UDMA speed of what appears a combined DVD/hard drive controller [unplugging Dell USB printer w/card readers helped sometimes when that routine did not have to run]. See bug 32123 [where I MISTAKENLY added comment] for many more details.

I'll let you know if this fails to work again; but, I'm not sure we are commenting at the right place again!

We are currently building candidates for Ubuntu 8.04.1, which you can get from http://cdimage.ubuntu.com/hardy/daily-live/current (desktop) and http://cdimage.ubuntu.com/hardy/daily/current (alternates). They should fix this particular bug, too. Testing and verification appreciated!

Copied to hardy-updates. The new kernel was tested extensively by many people, who reported back in other bug reports. Due to lack of feedback, this particular bug was not confirmed to be tested, though. Please report back here if the bug still occurs for you with the new kernel packages, then we will reopen this bug.

Please take a look at Bug #240470. I'm having another issue that might be related to this one even using the -19.33 kernel; did any of you see anything similar?

The image from <http://cdimage.ubuntu.com/hardy/daily-live/current> I downloaded this morning does NOT solve the problem. I tried to boot as LiveCD and it arrived at busybox. This is a Dell Vostro 200 (SATA disk, SATA DVD, no floppy, Core 2 Duo)

I'll test live-cd later today and will let you know. thanks.

hello, i have intel little falls motherboard with the same NIC. I can do some tests for you, if i am not sure, what would help you. I try the current daily live CD. If it's OK?

As reported earlier [similar Dell Vostro], this liveCD fails. Note: 'all-generic-ide' works for me!

Great thing is, that thic cd boots, it doesn't hangup during initram. But the NIC doesn't work out of the box.

Jan

Reopening then, thanks for testing!

Please look at issue #240648, we find the updated R8169.c driver will fail to load after system go to S4 for 3 or 4 times. And eth0 will disappear.

I tested it with same platform I reported this issue(223656) and checked with latest latest 2.6.24-19-generic kernel. And still can find this issue.
By looking at dmesg, during the driver loading, IRQ is failed to allocate for this NIC card and cause the NIC initiate fail after system resume from S4.

I've no permission to look at bug 240648! But, 8.04 worked/installed on Dell/Vostro for several boots [until we hit run level 4 a few times!].
Go figure!

Sorry for the misleading. Actually I am not meaning RUN LEVEL 4. I mean when we go to hibernate for 3 or 4 times without reboot. r8169 will fail to load on the RTL8102EL NIC card.

Three or four times ! That's good. I think I might have had a successful restore once. Certainly not more.

I tried it yesterday with the all_generic_ide fix to see if that helped.

Nah. Reloaded all the pages and then froze at "Suspending console(s)"

This is the Dell Vostro 200, by the way.

Polvell, are you failing on the NIC driver with the old LiveCD or you fail on ide channel?

Hi Steve,

my failure on the Dell Vostro is with the ide drivers. Adding "all_generic_ide" in grub allows the system to run. Of course, the LiveCD won't boot.

Thanks.....Peter

Now i can report, tahat with newest 19 gneric kernel the NIC works well, i havent tried S3 or S4, but now it is runnig 24 hour well.

Thank you

Jan

Martin - The kernel oops reported in https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/223656/comments/1 has been fixed. This driver has other issues, but I'm marking this one as fix released. Steve has confirmed that the oops no longer happens.

Fixed upstream in 2.6.26

I can also confirm Intel D945GCLF with RTL8101E now works OK - Hardy 8.04 with 2.6.24.-19-generic. Thanks a lot!

This is a great fix !

I also have a D945GCLF with RTL8101E, but i am trying to netboot install hardy using these instructions:
http://ubuntu-tutorials.com/2007/10/11/how-to-configure-pxe-network-booting-on-ubuntu-for-network-based-installations/
The problem I have is that i see this oops above. I really need the config file for hardys kernel so i can update the kernel i am trying to netboot with the fix.

Does anyone know where i can find the config file for the netboot kernel for hardy ?

Thanks for any help on this.

I have found a more recent hardy iso:
http://cdimage.ubuntu.com/hardy/daily/20080619.3/
This seems to boot, but when it comes to configuring the network settings the problems start :(
I can not ping anything on my network when manually configured and it will not DHCP.
The netboot image is using kernel version : 2.6.24-19-generic.
using lsmod i can see r8169 is loaded and eth0 exists, it just doesnt work :(

I am only seeing these problems as i am netbooting ?

Thanks for any help.

after pulling the power cable out and then booting networking seems to spring into life, odd. Perhaps there is some dodgy reset circuitry on the board :)

Baconbits,

Can you check with lspci -vvnn to see if there is any error with your NIC PCI information?

I am having a similar problem. Live CD and install have been fine but during restart following a suspend or hibernate the ethernet does not come back.

Normal lspci -vvnn:
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller [10ec:8168] (rev 02)
 Subsystem: Dell Unknown device [1028:0278]
 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
 Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
 Latency: 0, Cache Line Size: 64 bytes
 Interrupt: pin A routed to IRQ 219
 Region 0: I/O ports at be00 [size=256]
 Region 2: Memory at fd8ff000 (64-bit, non-prefetchable) [size=4K]
 Region 4: Memory at fdef0000 (64-bit, prefetchable) [size=64K]
 [virtual] Expansion ROM at fde00000 [disabled] [size=128K]
 Capabilities: [40] Power Management version 3
  Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
  Status: D0 PME-Enable- DSel=0 DScale=0 PME-
 Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable+
  Address: 00000000fee0f00c Data: 413a
 Capabilities: [70] Express Endpoint IRQ 1
  Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag-
  Device: Latency L0s <512ns, L1 <8us
  Device: AtnBtn- AtnInd- PwrInd-
  Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
  Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
  Device: MaxPayload 128 bytes, MaxReadReq 512 bytes
  Link: Supported Speed 2.5Gb/s, Width x1, ASPM L0s L1, Port 0
  Link: Latency L0s <512ns, L1 <64us
  Link: ASPM Disabled RCB 64 bytes CommClk+ ExtSynch-
  Link: Speed 2.5Gb/s, Width x1
 Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
  Vector table: BAR=4 offset=00000000
  PBA: BAR=4 offset=00000800
 Capabilities: [d0] Vital Product Data

After restart from suspend:
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller [10ec:8168] (rev ff) (prog-if ff)
 !!! Unknown header type 7f

Needless to say eth0 fails

I'm seeing this issue with a clean install of 8.04.1 i386 Server Edition on an Intel "Little Falls" D945GCLF motherboard. The built in NIC (eth0) does not work at any time including when powered up from cold. Kernel is "2.6.24-19-server".

"dmesg" and "lspci -vvnn" output attached.

Wierdly, the NIC worked whilst in the installation process!

I've plugged a Kingston KNE100TX NIC into the PCI slot as my temporary workaround for the moment.

Typical state:

$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:1c:c0:46:41:65
          inet6 addr: fe80::21c:c0ff:fe46:4165/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:4294967291 overruns:0 frame:0
          TX packets:0 errors:0 dropped:46 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
          Interrupt:220 Base address:0x6000

No cable is plugged in yet it thinks the interface is up!

Has the fix not made it into the Server Edition yet?

cheers

Strange. In my case it works most of the time. It's working right now. I do turn off the power supply too before restarting it, did you try that? I noticed that the motherboard still has power even after a complete shut-down, and WOL might keep your network card in a weird state. (You can just unplug it after turning off if your power supply doesn't have a hardware switch.)

This is root caused as a kernel r8169 driver issue. A patch is ready but not sure when to release.

Is there more information/solution for the D945GCLF board, to get the network card working.

I'm using Moblin image 2.6.24 Kernel [Ubuntu mobile]

Same problem here - even with the latest Hardy kernel ... Also on the Intel D945GCLF Atom Board.

Same problem, also with D945GCLF and Realtek 8101E. I've posted a detailed description of the problem and my efforts here:

https://bugs.launchpad.net/ubuntu/+bug/215450

Steve,
What's the fix? How can we test it?
I have tried latest hardy-updates and hardy-proposed kernels and still have a driver problem.
Usage: Board D945GCLF, using NIC as PXE Boot with LTSP.

I have done quite a bit of testing on this issue. Here's what I found:
1) 2.6.24-16 kernel does not work with this board (kernel panic on NIC driver load).
2) 2.6.24-19 (old one in hardy-updates, I believe was -33 or -37) passes kernel panic, NIC driver loads, PXE is functional, , but has audio problem as alsa improperly configures to 16kHz and 1 channel. I researched this in bug report 240648 which is related and found there to be a reported PCI scan conflict with the NIC and Audio at the same time. Video was non-functional at the login manager at first, then after xorg settings adjustment, got it to work.
3) 2.6.24-19-41 (latest in hardy-updates) fixes the audio problem above, however, video does not work (the login manager, e.g. ldm is not visible- shows strange lines on the screen). Note on video problem: this kernel is similar to what we saw with the older kernels with respect to the video problem, however, some xorg settings made the video work with the .24-19-33/37 kernel whereas with the -41 kernel this can not be fixed.
4) 2.6.24-21 kernel (in hardy-proposed) is similar to 2.6.24-19-41 kernel in it seems the NIC and audio problems are fixed but the video problem is not fixable.

Note: I believe this to be a kernel issue, as using different kernels does affect the result, and when considering the number of kernel issues reported for this board.

You are talking about the driver issue for Video or you are talking about issue for NIC? I think Oops issue on NIC is fixed in 2.6.24.19.

Maybe you need a new bug open for Video issue.

Possibly... but this seems to vary by the kernel version, so I guess its a kernel driver and not xorg driver issue.
Do you think I should open a new bug?

The bug task for the somerville project has been removed by an automated script. This bug has been cloned on that project and is available here: https://bugs.launchpad.net/bugs/1306169