check.py: infinite loop due wrong handling of symbolic links

Bug #755441 reported by Martina Theuerjahr
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
HPLIP
New
Undecided
Unassigned
Gentoo Linux
Fix Released
Medium

Bug Description

Version: hplip-3.11.3a

When check.py is executed, the first call of check_file() (with argument "crypto.h") leads to an infinite loop within the recursive function utils.walkFiles(). The reason is that this function follows symbolic links to directories.

On my system the directory X contains the symbolic link "Y -> ." because X/Z and X/Y/Z shall be accessible for all entries Z without having the Zs twice on the disk. This is necessary because one program assume Z at X/Z and another program at X/Y/Z.

Anyhow, it is not a good practice to follow symbolic links at a directory descent. Because often an infinite loop starts at it is the case for check.py.

Obviously other users already had the same problem, because there is the question https://answers.launchpad.net/hplip/+question/36267 and the source code for walkFiles() contains the following comment:

# if os.path.islink(fullname):
# fullname = os.path.realpath(os.readlink(fullname))

When the comment chars are removed from these lines check.py works perfectly again.

check_file()
/usr/share/hplip/installer/dcheck.py

walkFiles()
/usr/share/hplip/base/utils.py

description: updated
description: updated
summary: - check.py: infinite loop due wrong handling of to symbolic links
+ check.py: infinite loop due wrong handling of symbolic links
description: updated
Revision history for this message
Wayne E Robertz (wer6) wrote :

I found the same problem in my Linux distribution (Slackware 12.0), but when I uncommented the two lines it still did not work. Turns out that in /usr/include/seamonkey, there are 3 symbolic links:
    /usr/include/seamonkey/plugin -> .
    /usr/include/seamonkey/nss -> .
    /usr/include/seamonkey/xpcom -> .

 Instead, I added the following to walkFiles after the assignment of fullname:

        if os.path.islink(name):
            linkname = os.readlink(name);
# log.debug("link: %s -> %s" % (fullname,linkname))
            if linkname == ".":
                log.debug("skipping link: %s (.)" % fullname)
                continue

This effectively skips over symbolic links that point to the current root.

This works for me. I've attached a copy of my revised utils.py.

Revision history for this message
Wayne E Robertz (wer6) wrote :

Sorry, in my previous comment, I forgot to mention that this is for hplip-3.13.7

Revision history for this message
Wayne E Robertz (wer6) wrote :

Actually, in the lines above, it should be:
    if os.path.islink(fullname):
            linkname = os.readlink(fullname)

This the following symbolic link much better than the above:
    /usr/include/gphoto2/gphoto2 -> .

Tested in hplip-3.13.8

Revision history for this message
In , brian.gloyer (brian.gloyer-gentoo-bugs) wrote :
Download full text (9.0 KiB)

hp-doctor runs forever. It worked in a previous version (3.18.6 I think).

>hp-doctor -g
hp-doctor[13883]: debug: ['unknown', 'mepis', 'debian', 'suse', 'mandriva', 'fedora', 'redhat', 'rhel', 'slackware', 'gentoo', 'redflag', 'ubuntu', 'xandros', 'freebsd', 'linspire', 'ark', 'pclinuxos', 'centos', 'igos', 'linuxmint', 'linpus', 'gos', 'boss', 'lfs', 'manjarolinux']
hp-doctor[13883]: debug: HPLIP Description= Public version=3.19.12 Internal version = 3.19.12
hp-doctor[13883]: debug: Determining distro...
hp-doctor[13883]: debug: Able to detect distro
hp-doctor[13883]: debug: Distro name=gentoo
hp-doctor[13883]: debug: Distro version=2.6
hp-doctor[13883]: debug: distro=9, distro_version=2.6
hp-doctor[13883]: debug: Enable PPD install: True (False=drv)
hp-doctor[13883]: debug: Default UI toolkit: Qt4
hp-doctor[13883]: debug: Checking for dependency 'epm'...

hp-doctor[13883]: debug: Checking: epm --version (min ver=0.000000)
hp-doctor[13883]: debug: Not found!
hp-doctor[13883]: debug: have epm = False
hp-doctor[13883]: debug: Checking for dependency 'automake'...

hp-doctor[13883]: debug: Checking: automake --version (min ver=0.000000)
hp-doctor[13883]: debug: automake (GNU automake) 1.16.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Tom Tromey <email address hidden>
       and Alexandre Duret-Lutz <email address hidden>.

hp-doctor[13883]: debug: Found.
hp-doctor[13883]: debug: have automake = True
hp-doctor[13883]: debug: Checking for dependency 'libjpeg'...

hp-doctor[13883]: debug: Checking for library 'libjpeg'...
hp-doctor[13883]: debug: Found.
hp-doctor[13883]: debug: Searching for file 'jpeglib.h' in '/usr/include'...
[[at this point it pegs a cpu and runs forever]]

>emerge --info hplip
Portage 2.3.89 (python 3.6.10-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-9.2.0, glibc-2.30-r8, 5.2.11-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.2.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-8700_CPU_@_3.20GHz-with-gentoo-2.6
KiB Mem: 16287688 total, 7814812 free
KiB Swap: 125975548 total, 125975548 free
Timestamp of repository gentoo: Sun, 26 Apr 2020 17:30:01 +0000
Head commit of repository gentoo: 00bf0487dbb811dda18eaa7626e6c9e38b15bf6f
Timestamp of repository guru: Fri, 24 Apr 2020 15:39:39 +0000
Head commit of repository guru: 2abf7f72c6f0d8b6549228b00389e187a4a540de

Timestamp of repository jm-overlay: Sun, 19 Apr 2020 17:13:02 +0000
Head commit of repository jm-overlay: e6776212108b6007436cb83f2671c3aa0ed83da7

sh bash 4.4_p23-r1
ld GNU ld (Gentoo 2.33.1 p2) 2.33.1
app-shells/bash: 4.4_p23-r1::gentoo
dev-java/java-config: 2.2.0-r4::gentoo
dev-lang/perl: 5.30.1::gentoo
dev-lang/python: 2.7.18::gentoo, 3.6.10-r2::gentoo, 3.7.7-r2::gentoo, 3.8.2-r2::gentoo
dev-util/cmake: 3.16.5::gentoo
dev-util/pkgconfig: 0.29.2::g...

Read more...

Revision history for this message
In , mail (mail-gentoo-bugs) wrote :

Do you have a /usr/include/jpeglib.h file?
It should belong to media-libs/libjpeg-turbo.

See also: https://forums.gentoo.org/viewtopic-p-8306276.html#8306276

Revision history for this message
In , brian.gloyer (brian.gloyer-gentoo-bugs) wrote :

I do have /usr/include/jpeglib.h from media-libs/libjpeg-turbo.

I think I'm seeing the same hp-doctor problem mentioned in that forum post (that was on hplip 3.18.12). My printer is working fine so it is a problem with hp-doctor.

There are a few more posts out there with the same problem, these are from 2017:
https://forums.gentoo.org/viewtopic-t-1065638-highlight-hpdoctor.html
https://forums.gentoo.org/viewtopic-t-1071492-highlight-hpdoctor.html

this one from 2014 shows it working:
https://forums.gentoo.org/viewtopic-t-952040-highlight-hpdoctor.html

It worked for me in Feb 2019 so it doesn't seem to be tied to a particular version.

hp-check also has the same problem.

Revision history for this message
In , brian.gloyer (brian.gloyer-gentoo-bugs) wrote :

I think I found the cause. There is a symbolic link that points to its parent and hp-doctor goes into a loop expanding the tree.

$ ls -l /usr/include/node/src
lrwxrwxrwx 1 root root 1 Mar 27 00:06 /usr/include/node/src -> .

$ qfile /usr/include/node/src
net-libs/nodejs: /usr/include/node/src

Revision history for this message
Daniel Pielmeier (daniel-pielmeier) wrote :

Can someone please take a look at this? hplip-3.20.3 has problems with symlinks. This causes tools like hp-check or hp-doctor which are using walkFiles() to run infinite.

Revision history for this message
In , billie (billie-gentoo-bugs) wrote :

Pretty sure this problem existed before 3.18.6 as this has been reported upstream already for 3.11.3.

I think it now pops up again because net-libs/nodejs introduced this symlink in a recent version.

Changed in gentoo:
importance: Unknown → Medium
Revision history for this message
In , billie (billie-gentoo-bugs) wrote :

Created attachment 637524
Fix issues related to hp-check and hp-doctor

I uncommented the symlink check as recommended in the upstream bug. Afterwards hp-check and hp-doctor rune although slower as I remember.

There were also problems detecting pillow versions > 7.0.0 as well as PyQT5.

This patch tries to fix all this issues although I am not sure about the PyQt5 fix.

Revision history for this message
In , billie (billie-gentoo-bugs) wrote :

Should be fixed in Git with hplip-3.20.5.

Changed in gentoo:
status: Unknown → Fix Released
Revision history for this message
In , herrtimson (herrtimson-gentoo-bugs) wrote :

I'm sorry, but this came back with net-print/hplip-3.20.6-r1, where hp-check keeps hanging in a deadlock. Problem can be solved by removing net-libs/nodejs, where it seems to have found a symlink it happily follows until eternity.

Shall we reopen here and close my bug as duplicate?

Changed in gentoo:
status: Fix Released → Unknown
Revision history for this message
Daniel Pielmeier (daniel-pielmeier) wrote :

I took a look at it and I think I found a solution for this problem.

The walkfiles procedure is calling itself which in some cases causes an infinite loop when a symbolic link is pointing to a directory.

Before going into the recursion the patch checks if the object in question is a link and if the target of the link is a directory and then breaks the for loop. Omitting the links to directories should not be a problem as the directories itself are checked anyway during recursion.

Revision history for this message
Daniel Pielmeier (daniel-pielmeier) wrote :

Actually continue should be used instead of break to only omit the symbolic link and not everything else afterwards.

Changed in gentoo:
status: Unknown → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.