USB keyboard does not work after grub, and until after encrypted root volume is unlocked

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1619.0 Safari/537.36 SUSE/31.0.1619.0

Kernel 3.11 hangs immediately during boot upon reaching the point where it prompts for luks key phrase. There is not even an opportunity to enter the password. This is the build (3.11.0-2.1.g0a1c41f) available from:

http://download.opensuse.org/repositories/Kernel:/stable/standard/

Reproducible: Always

Steps to Reproduce:
1. Install kernel 3.11.0-2.1.g0a1c41f on encrypted root system
2. Boot system
3.
Actual Results:
The system hangs and does not allowed entering luks password.

Expected Results:
The system should allow luks key phrased to be entered and continue booting.

Upon further investigation, this issue seems to be related to USB keyboards only, in other words, the system does not hang, but rather it only recognizes non-USB keyboards. So, this could suggest the necessary module is not being loaded prior to prompting for luks key phrase. Non-USB keyboards work once the system is booted.

What keyboard are you using? What kernel (arch/flavor)?

On my system with a 3.11 kernel, I see usbhid in the initrd so it should work fine.

(In reply to comment #2)
> What keyboard are you using? What kernel (arch/flavor)?
>
> On my system with a 3.11 kernel, I see usbhid in the initrd so it should work
> fine.

First I should correct my previous statement that "Non-USB keyboards work once the system is booted." Instead, it should read, USB keyboards work once the system is booted.

I have tested a Logitech wireless keyboard and a generic/brandless USB keyboard. Both work on grub menu, but fail when prompted to enter luks key phrase. Only PS/2 keyboard works.

The kernel version/flavor is 3.11.0-2.g0a1c41f-desktop and the architecture is x64. Here is a similar bug report on this kernel release which mentions only wireless, but based on my testing, it is USB related: https://bugzilla.novell.com/show_bug.cgi?id=838864

The bug is still present in Kernel version 3.11.1-1.gcee6127-desktop x86_64.

I believe this to be a duplicate of a bug I filed on September 6th. The bug number is: 838864

If you run mkinitrd -v when the system is up, do you get a kernel error at the end of the "modules" statements? I think this might be a mkinitrd problem rather than actually being the kernel. Possibly all the extra characters at the end of the newer kernel names is causing problems?

(In reply to comment #5)
> I believe this to be a duplicate of a bug I filed on September 6th. The bug
> number is: 838864
>
> If you run mkinitrd -v when the system is up, do you get a kernel error at the
> end of the "modules" statements? I think this might be a mkinitrd problem
> rather than actually being the kernel. Possibly all the extra characters at the
> end of the newer kernel names is causing problems?

This is the output:

Kernel image: /boot/vmlinuz-3.11.1-1.gcee6127-desktop
Initrd image: /boot/initrd-3.11.1-1.gcee6127-desktop
KMS drivers: radeon
Root device: /dev/system/root (mounted on / as ext4)
Resume device: /dev/system/swap
enabling LUKS support for /dev/disk/by-id/ata-[redacted]-part2 (cr_ata-[redacted]-part2)
[BLOCK] /dev/sda -> ahci
[MODULES] 01-acpi.sh: thermal processor fan
[MODULES] 02-start.sh: pata_atiixp ata_generic
[MODULES] 02-start.sh:
[MODULES] 03-dm.sh: dm-mod
[MODULES] 03-dm.sh: dm-snapshot
[MODULES] 03-dm.sh: dm-crypt
[MODULES] 03-rtc.sh: rtc_cmos
[MODULES] 03-scsi_dh.sh: scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh_alua
[MODULES] 03-storage.sh:
[MODULES] 05-kms.sh: radeon
[MODULES] 11-block.sh: ahci
[MODULES] 11-usb.sh: usbcore
[MODULES] 11-usb.sh: ohci_hcd
[MODULES] 11-usb.sh: uhci-hcd
[MODULES] 11-usb.sh: ehci_hcd
[MODULES] 11-usb.sh: xhci-hcd
[MODULES] 11-usb.sh: usbhid
[MODULES] 11-usb.sh: hid-logitech-dj
[MODULES] 11-usb.sh: hid-generic
[MODULES] 11-usb.sh: hid-holtek-kbd
[MODULES] 11-usb.sh: hid-lenovo-tpkbd
[MODULES] 11-usb.sh: hid-logitech-dj
[MODULES] 11-usb.sh: hid-ortek
[MODULES] 11-usb.sh: hid-roccat-arvo
[MODULES] 11-usb.sh: hid-roccat-isku
[MODULES] 11-usb.sh: hid-samsung
[MODULES] 11-usb.sh: hid-apple
[MODULES] 11-usb.sh: hid-belkin
[MODULES] 11-usb.sh: hid-cherry
[MODULES] 11-usb.sh: hid-ezkey
[MODULES] 11-usb.sh: hid-microsoft
[MODULES] 61-lvm2.sh: linear
[MODULES] 71-luks.sh: dm-crypt
[MODULES] 71-luks.sh: sha256_generic sha256_generic cbc cryptomgr
[MODULES] 81-btrfs.sh: btrfs
[MODULES] Unsupported kernel (3.11.1-1.gcee6127-desktop)
Kernel Modules: thermal_sys thermal processor fan pata_atiixp ata_generic dm-mod dm-snapshot dm-crypt scsi_dh scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh_alua i2c-algo-bit drm drm_kms_helper ttm radeon xhci-hcd hid-logitech-dj hid-holtek-kbd hid-lenovo-tpkbd hid-ortek hid-roccat hid-roccat-common hid-roccat-arvo hid-roccat-isku hid-samsung linear sha256_generic cbc libcrc32c xor zlib_deflate raid6_pq btrfs crc32c-intel
Firmware: radeon/R520_cp.bin radeon/RS600_cp.bin radeon/RS690_cp.bin radeon/R420_cp.bin radeon/R300_cp.bin radeon/R200_cp.bin radeon/R100_cp.bin radeon/SUMO2_me.bin radeon/SUMO2_pfp.bin radeon/SUMO_me.bin radeon/SUMO_pfp.bin radeon/SUMO_rlc.bin radeon/PALM_me.bin radeon/PALM_pfp.bin radeon/CYPRESS_smc.bin radeon/CYPRESS_rlc.bin radeon/CYPRESS_me.bin radeon/CYPRESS_pfp.bin radeon/JUNIPER_smc.bin radeon/JUNIPER_rlc.bin radeon/JUNIPER_me.bin radeon/JUNIPER_pfp.bin radeon/REDWOOD_smc.bin radeon/REDWOOD_rlc.bin ...

Yes, It's almost identical to mine. I use the ATI fglrx proprietary driver and kernel-default. The problem appears to be the same. Under the "Kernel Modules" statement, it shows the correct modules to be included, as far as I can tell. But apparently those modules are not actually being included in the initrd when mkinitrd is building, or possibly they are included, but not built correctly because of the following error:

[MODULES] Unsupported kernel (3.11.1-1.gcee6127-desktop)

Now all we need is someone with enough knowledge to pin down the exact problem.

Since this seems to be an mkinitrd problem
I'm assigning to its maintainer

meanwhile, could you check output of
gzip -cd /boot/initrd | cpio -t | grep usb

if it lists usbhid.ko, usbcore.ko and 4 *hci-hdc.ko modules

*** Bug 841631 has been marked as a duplicate of this bug. ***

>gzip -cd /boot/initrd | cpio -t | grep usb

Here's the output:
  ---- start ----
lib/modules/3.11.0-27.g0a1c41f-desktop/kernel/drivers/usb
lib/modules/3.11.0-27.g0a1c41f-desktop/kernel/drivers/usb/host
lib/modules/3.11.0-27.g0a1c41f-desktop/kernel/drivers/usb/host/xhci-hcd.ko
usr/lib64/libusbmuxd.so.1.0.8
usr/lib64/libusbmuxd.so.2
usr/lib64/libusb-1.0.so.0.1.0
usr/lib64/libusb-1.0.so.0
usr/lib64/libusb-0.1.so.4.4.4
usr/lib64/libusb-0.1.so.4
usr/lib/udev/usb_modeswitch
config/usb.sh
48651 blocks
boot/11-usb.sh
 ---- end ---

I also checked the output with kernel 3.10.10, and it is almost identical
(except for the kernel version in the module path). Everything works
with the 3.10.10 kernel, but not with the 3.11.0 kernel.

My keyboard is a plain Dell USB keyboard. I have another computer, with the same keyboard model (but the computer itself is a lot newer). I do not have any problem on the newer computer.

I have the same problem, reported under bug: 838864

I am currently running kernel-default-3.11.1-4.1 and the problem still exists. I ran the command that was suggested and I came up with the correct modules:

---start---
gzip -cd /boot/initrd | cpio -t | grep usb
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/hid/usbhid
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/hid/usbhid/usbhid.ko
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/core
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/core/usbcore.ko
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/host
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/host/xhci-hcd.ko
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/host/ehci-hcd.ko
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/host/ohci-hcd.ko
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/host/uhci-hcd.ko
lib/modules/3.11.1-4.g2fa222d-default/kernel/drivers/usb/usb-common.ko
usr/lib64/libusb-1.0.so.0
usr/lib64/libusb-1.0.so.0.1.0
112595 blocks
boot/11-usb.sh
config/usb.sh
---end---

To me, that raises more questions than answers. Mine also shows hid-logitech-dj.ko in place. The reason mine is different from Neil's output is possibly because I manually entered the modules in the Yast>sysconfig editor. I did that in an attempt to force the modules, as I thought that they weren't being correctly included.

Now it seems that the modules are there, but for some reason the kernel can't make use of them.

Eureka. I've got it.

I added "ohci_pci" to the list of modules to be included in the initrd.

That seems to have solved the problem. Please check if you are having the same problem.

Details: On my tumbleweed system, while running a 3.10 kernel, I did a "cd" to "/lib/modules" directory for the current kernel, ran "ls -R" and put the output into a file. I did the same with the directory for the unusable 3.11 kernel on the same system.

I did a diff on the two files, to see what new modules were present for 3.11

I then booted 13.1Beta1 on the same box, and ran "lsmod". I checked which of the new modules were being used. I only found two (the other was "mii"), and the likely one seemed to be "ohci_pck".

Booting back into tumbleweed, 3.10 kernel, I added that the the list of modules to be included in initrd, and rebuild the initrd (I use the Yast sysconfig editor on "kernel" which allowed the editing and rebuilt the "initrd".

A final reboot, and I was successful with the 3.11 kernel.

It looks to me as if "ohci_pci" needs to be automatically included for 3.11 kernels. Whoever maintains "mkinitrd" should take care of that (I hope).

make that "ohci_pci" - I accidently had "pck" in place of "pci" in the 6th paragraph.

I can verify that your work-around does the job. My wireless keyboard is back to normal. I can also verify that it works in kernel-default-3.11.2 as well.

That was a fine piece of analytical work. ohci_pci needs to be in the initrd and it is not done automatically.

Thanks for all the work.

I spent a little time with google search yesterday, trying to find out more about this.

Here's a link that I found:

http://lists.linaro.org/pipermail/linaro-dev/2013-May/016112.html

I'm not quite sure what mailing list that is. But it looks like a discussion of breaking some of the USB code out of the kernel, and moving to ohci_pci. This is not code for keyboards and other devices. Rather, it is code for the USB host adapter that the keyboard is plugged into.

If I am understanding that correctly, then it becomes clear that ohci_pci needs to be in the "initrd", at least for hardware where it might be applicable.

Great job tracking this one down, guys!

The fix will be to add ohci_pci to the list of modules in /lib/mkinitrd/scripts/boot-usb.sh

You can do that now as a work around until the package is updated.

This change (adding ohci_pci) needs to be in 13.1RC1.

(In reply to comment #16)
> Great job tracking this one down, guys!
>
> The fix will be to add ohci_pci to the list of modules in
> /lib/mkinitrd/scripts/boot-usb.sh
>
> You can do that now as a work around until the package is updated.

The fix works I can confirm as well. Though, it is not permanent.

I added the module to /lib/mkinitrd/scripts/boot-usb.sh and it has remained for me. I have upgraded the kernel twice and have run mkinitrd -v several times manually since the edit, and mkinitrd still includes ohci_pci during build. The file has not changed since the edit. I removed the modules that I had added to /etc/sysconfig/kernel

>The fix works I can confirm as well. Though, it is not permanent.

For the time being, it is better to force ohci_pci in "/etc/sysconfig/kernel"

The trouble with putting it in that "mkinitrd" script, is that if the script is update, that will lose your changes. On one of my beta installs, I have seen that script updated twice. The bug we are discussing has still not been fixed in the updated script.

To whomever is responsible: it is important to get the fix in before RC1.

I had planned to install Beta1 in an encrypted LVM, but held off because of this problem. I will install RC1 in that encrypted LVM (on an affected computer). But, if the bug is not fixed by then, I will run into problems. I will have to go to rescue mode, edit in a fix, and "mkinitrd" from rescue mode, before I can boot the new system for the first time. This is not satisfactory. WE NEED THAT FIX NOW.

This is an autogenerated message for OBS integration:
This bug (839071) was mentioned in
https://build.opensuse.org/request/show/202960 Factory / mkinitrd

This is still a problem with RC1. It looks as if the fix in comment 21 did not make it in time.

I installed RC1 into an encrypted LVM. And, of course, I could not boot because I could not provide the encryption key. I went into rescue mode, forced "ohci_pci" into the "initrd" (vi "/etc/sysconfig/kernel"), and rebuilt the "initrd". Thereafter, I could boot and complete the final step of install.