ISST-LTE: unable to open rtc device

Bug #1410604 reported by bugproxy
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
Low
Canonical Kernel Team

Bug Description

== Comment: #0 - Chanh H. Nguyen <email address hidden> - 2015-01-09 17:30:08 ==
We use DVD to install Ubuntu 15.04 kernel 3.18.0-8-generic version on our PowerVM system. After install, I am seeing this error "/build/buildd/linux-3.18.0/drivers/rtc/hctosys.c: unable to open rtc device" on the dmesg log...

root@pole1lp3:~# uname -a
Linux pole1lp3 3.18.0-8-generic #9-Ubuntu SMP Mon Jan 5 22:52:15 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
root@pole1lp3:~# dmesg |grep rtc
[ 0.556763] /build/buildd/linux-3.18.0/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 3.296241] rtc-generic rtc-generic: rtc core: registered rtc-generic as rtc0
root@pole1lp3:~#

== Comment: #2 - David Heller <email address hidden> - 2015-01-13 20:53:50 ==
It must be an order of operations thing: something is trying to do the hctosys before the driver is loaded. A few seconds later the driver loads and it seems fine after that.

root@pole1lp3:~# dmesg | grep rtc
[ 0.551202] /build/buildd/linux-3.18.0/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 3.353719] rtc-generic rtc-generic: rtc core: registered rtc-generic as rtc0

root@pole1lp3:~# find /dev -name \*rtc\*
/dev/rtc
/dev/rtc0

root@pole1lp3:~# hwclock
Tue 13 Jan 2015 07:36:15 PM CST -0.779773 seconds

root@pole1lp3:~# hwclock --hctosys

root@pole1lp3:~# date
Tue Jan 13 19:38:05 CST 2015

bugproxy (bugproxy)
tags: added: architecture-ppc64le bugnameltc-120243 severity-medium targetmilestone-inin---
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1410604/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

tags: added: bot-comment
Revision history for this message
Dave Heller (hellerda) wrote :

Setting to package "init" on LP side, as it appears to be an init issue rather than a kernel issue.

affects: ubuntu → init-system-helpers (Ubuntu)
tags: added: vivid
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2015-01-14 15:15 EDT-------
The same problem is seen on 14.10 as well:

root@cellc10:~# uname -a
Linux cellc10 3.16.0-23-powerpc64-smp #31-Ubuntu SMP Tue Oct 21 18:12:52 UTC 2014 ppc64 ppc64 ppc64 GNU/Linux

root@cellc10:~# dmesg | grep rtc
[ 0.692983] /build/buildd/linux-3.16.0/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 4.349881] rtc-generic rtc-generic: rtc core: registered rtc-generic as rtc0

root@cellc10:~# find /dev -name \*rtc\*
/dev/rtc
/dev/rtc0

root@cellc10:~# hwclock
Wed 14 Jan 2015 10:11:47 AM EST -0.816985 seconds

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-01-20 10:51 EDT-------
*** Bug 120596 has been marked as a duplicate of this bug. ***

Revision history for this message
Dave Heller (hellerda) wrote :

Hi Canonical,

Even thought this is a benign error, it is seen on all archs on every release as far back as trusty (at least). Can you please set a target for resolution? Thanks.

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-11-18 11:00 EDT-------
*** Bug 128128 has been marked as a duplicate of this bug. ***

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-12-03 06:47 EDT-------
Hi Canonical,

Is there any update on this bug?
Do you need any other information in order debug/solve this bug?

Thanks for your support.!!

bugproxy (bugproxy)
tags: added: targetmilestone-inin1604
removed: targetmilestone-inin---
Revision history for this message
Martin Pitt (pitti) wrote :

The only place known to me that does that is /etc/init.d/hwclock.sh, which is from util-linux. Something similar is also done under upstart from /etc/init/hwclock.conf where hwclock is called with --systz. This quite plausibly opens rtc as well.

However, these two scripts aren't being run at all any more under systemd -- there it would be timedated which accesses /dev/rtc, but I don't see how this is being activated that early (it usually doesn't run at boot at all).

Can you please confirm whether this happens under upstart or systemd? In the latter case, can you please do

   sudo journalctl -b > /tmp/journal.txt

and attach /tmp/journal.txt here? Thanks!

affects: init-system-helpers (Ubuntu) → util-linux (Ubuntu)
Changed in util-linux (Ubuntu):
status: New → Incomplete
importance: Undecided → Low
Revision history for this message
bugproxy (bugproxy) wrote : journalctl -b file

------- Comment on attachment From <email address hidden> 2016-02-24 09:33 EDT-------

We still see this on 16.04 release.
root@pole2lp2:~# uname -r
4.4.0-6-generic
root@pole2lp2:~# dmesg |grep "rtc"
[ 25.057147] hctosys: unable to open rtc device (rtc0)
[ 484.267157] rtc-generic rtc-generic: rtc core: registered rtc-generic as rtc0
root@pole2lp2:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Xenial Xerus (development branch)
Release: 16.04
Codename: xenial

Revision history for this message
Martin Pitt (pitti) wrote :

Thanks for the journal. This shows that the error message happens way earlier than userspace/pid 1 starts. So it's either the kernel itself, or the initramfs. I can indeed see this error message in our ppc64el scalingstack instances as well.

So I went through the initramfs-tools hooks and scripts and found /usr/share/initramfs-tools/scripts/local-premount/fixrtc, which indeed does call "hwclock -s || true" (-s == --hctosys). However, while that's promising that isn't it: That code only runs if "fixrtc" is on the command line, which it isn't (at least for me). Adding a set -x there also shows that it runs after the error message:

[ 0.625781] evm: HMAC attrs: 0x1
[ 0.625920] hctosys: unable to open rtc device (rtc0)
[ 0.626595] Freeing unused kernel memory: 6144K (c000000000e80000 - c000000001480000)
Loading, please wait...
starting version 229
[ 0.637968] random: udevadm urandom read with 1 bits of entropy available
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... + PREREQ=
+ BROKEN_CLOCK=
+ ROOTDEV=
+ SYSTEMPART=
+ cat /proc/cmdline
+ value=UUID=9f6cb10b-904a-4b22-b9e1-80455f176cc2
+ ROOTDEV=/dev/disk/by-uuid/9f6cb10b-904a-4b22-b9e1-80455f176cc2
+ [ -z /dev/disk/by-uuid/9f6cb10b-904a-4b22-b9e1-80455f176cc2 -a -n ]
+ [ -n -a -n /dev/disk/by-uuid/9f6cb10b-904a-4b22-b9e1-80455f176cc2 ]
+ exit 0
done.
Warning: fsck not present, so skipping root file system
[ 2.608767] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[ 2.657889] systemd[1]: systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSET
UP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN)

affects: util-linux (Ubuntu) → initramfs-tools (Ubuntu)
Changed in initramfs-tools (Ubuntu):
status: Incomplete → Triaged
Revision history for this message
Martin Pitt (pitti) wrote :

Log from booting with "debug", i. e. set -x output from initramfs.

Revision history for this message
Martin Pitt (pitti) wrote :

Booting with "debug=y" works better, as then the initramfs debug goes right on the console, instead of /run/initramfs/initramfs.debug. This shows that even the initrd starts after the error message, so this must come from the kernel itself:

Loading Linux 4.4.0-8-generic ...
Loading initial ramdisk ...
OF stdout device is: /vdevice/vty@30000000
Preparing to boot Linux version 4.4.0-8-generic (buildd@bos01-ppc64el-008) (gcc version 5.3.1 20160222 (Ubuntu/IBM 5.3.1-9ubuntu3) ) #23-Ubuntu SMP Wed Feb 24 20:43:54 UTC 2016 (Ubuntu 4.4.0-8.23-generic 4.4.2)
Detected machine type: 0000000000000101
Max number of cores passed to firmware: 2048 (NR_CPUS = 2048)
Calling ibm,client-architecture-support... done
command line: BOOT_IMAGE=/boot/vmlinux-4.4.0-8-generic root=UUID=9f6cb10b-904a-4b22-b9e1-80455f176cc2 ro console=hvc0 earlyprintk debug=y
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 0000000005860000
  alloc_top : 0000000030000000
  alloc_top_hi : 0000000080000000
  rmo_top : 0000000030000000
  ram_top : 0000000080000000
instantiating rtas at 0x000000002fff0000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x0000000005870000 -> 0x0000000005870849
Device tree struct 0x0000000005880000 -> 0x0000000005890000
Quiescing Open Firmware ...
Booting Linux via __start() ...
[ 0.000000] Allocated 4980736 bytes for 2048 pacas at c00000000fb40000
[ 0.000000] Using pSeries machine description

[... only kernel messages here ...]

[ 0.633920] hctosys: unable to open rtc device (rtc0)
[ 0.634546] Freeing unused kernel memory: 6144K (c000000000e80000 - c000000001480000)
Loading, please wait...
+ [ -z ]
+ BOOT=local
+ [ -n ]
+ resume=
+ maybe_break top
+ export BOOT
+ run_scripts /scripts/init-top

affects: initramfs-tools (Ubuntu) → linux (Ubuntu)
Revision history for this message
Martin Pitt (pitti) wrote :

Indeed the code in http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/rtc/hctosys.c looks like it runs entirely "self-"triggered (late_initcall), not triggered by an userspace call. This function tries to open the device which simply isn't present (yet) at that time. So maybe this can be moved after the loading of the RTC modules somehow, or the error message quiesced, but I'd like to defer that to a kernel engineer.

tags: added: bot-stop-nagging xenial
Changed in linux (Ubuntu):
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-11-30 06:31 EDT-------
This bug is not being seen with Ubuntu 16.04.1 (kernel 4.4.0-47)

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial
$ uname -a
Linux p824l 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:38:24 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux
$ dmesg | grep rtc
[ 1.359605] rtc-opal opal-rtc: rtc core: registered rtc-opal as rtc0
[ 1.403682] rtc-opal opal-rtc: setting system clock to 2016-11-15 00:33:58 UTC (1479170038)
$

------- Comment From <email address hidden> 2016-11-30 06:32 EDT-------
Rejecting this bug as its not showing up with latest Ubuntu release.

Revision history for this message
Tim Gardner (timg-tpi) wrote :

Marking fixed release based on comment #14

Changed in linux (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.