fwts PCIe ASPM tests are throwing Segmentation fault

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1452201/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

Is it possible to have someone from the FWTS team take a look at this bug? Thanks.

Looking at it right now

Can fwts be run as follows:

sudo strace fwts aspm - >& trace.log

and can the trace.log file be added to the bug report. Thanks!

Also, this particular tool is reading the PCI config spaces to sanity check the PCIe ASPM modes. Can I get a copy of these config spaces so I can debug the one that causing fwts to break: Please can you do the following:

sudo tar cvf - /sys/bus/pci/devices/*/config > fwts-pci-config.tar

and attach that to the bug report.

Hi, without the information from the machine, this is going to be inconveniently hard to debug and fix, so if I can get the data as requested, that would be useful.

------- Comment From <email address hidden> 2015-12-11 09:49 EDT-------
Pavaman,

Can you provide the requested data?

Thanks

------- Comment (attachment only) From <email address hidden> 2015-12-11 12:21 EDT-------

------- Comment (attachment only) From <email address hidden> 2015-12-11 12:22 EDT-------

It appears that the issue is because the fwts ACPI table loader was failing to load ACPI tables for this machine and was trying to load them from file (as a last resort) and unfortunately was accessing a path that was NULL, hence a null pointer deference. I've fixed this up and I think it should resolve the issue.

I've uploaded the fix into a PPA, namely: ppa:colin-king/fwts-fixes

can you please test this and let me know if it fixes the issue. To get the version from my ppa do:

sudo add-apt-repository ppa:colin-king/fwts-fixes
sudo apt-get update
sudo apt-get dist-upgrade

and re-run fwts

Please let me know if this resolves the issue. Thanks!

For reference, fix attached

------- Comment From <email address hidden> 2015-12-14 05:00 EDT-------
I have tried to install the proposed fwts package on the machine by using the steps as below:

root@lep8d:~# add-apt-repository ppa:colin-king/fwts-fixes

More info: https://launchpad.net/~colin-king/+archive/ubuntu/fwts-fixes
Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keyring `/tmp/tmpcebj3onq/secring.gpg' created
gpg: keyring `/tmp/tmpcebj3onq/pubring.gpg' created
gpg: requesting key B0F01786 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpcebj3onq/trustdb.gpg: trustdb created
gpg: key B0F01786: public key "Launchpad PPA for Colin King" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK

root@lep8d:~# apt-get update
Hit http://archive.ubuntu.com vivid InRelease
Hit http://ports.ubuntu.com vivid InRelease
Hit http://archive.ubuntu.com vivid-updates InRelease
Ign http://ppa.launchpad.net vivid InRelease
Hit http://archive.ubuntu.com vivid-backports InRelease
Hit http://ports.ubuntu.com vivid-updates InRelease
Hit http://archive.ubuntu.com vivid/main Sources
Ign http://ppa.launchpad.net vivid Release.gpg
Hit http://ports.ubuntu.com vivid-backports InRelease
Hit http://archive.ubuntu.com vivid/restricted Sources
Ign http://ppa.launchpad.net vivid Release
Hit http://archive.ubuntu.com vivid/universe Sources
Hit http://ports.ubuntu.com vivid-security InRelease
Hit http://archive.ubuntu.com vivid/multiverse Sources
Hit http://ports.ubuntu.com vivid/main ppc64el Packages
Hit http://archive.ubuntu.com vivid-updates/main Sources
Hit http://ports.ubuntu.com vivid/restricted ppc64el Packages
Hit http://archive.ubuntu.com vivid-updates/restricted Sources
Hit http://archive.ubuntu.com vivid-updates/universe Sources
Hit http://ports.ubuntu.com vivid/universe ppc64el Packages
Hit http://archive.ubuntu.com vivid-updates/multiverse Sources
Hit http://ports.ubuntu.com vivid/multiverse ppc64el Packages
Hit http://archive.ubuntu.com vivid-backports/main Sources
Hit http://archive.ubuntu.com vivid-backports/restricted Sources
Hit http://ports.ubuntu.com vivid/main Translation-en
Hit http://archive.ubuntu.com vivid-backports/universe Sources
Hit http://ports.ubuntu.com vivid/multiverse Translation-en
Hit http://archive.ubuntu.com vivid-backports/multiverse Sources
Hit http://ports.ubuntu.com vivid/restricted Translation-en
Hit http://ports.ubuntu.com vivid/universe Translation-en
Hit http://ports.ubuntu.com vivid-updates/main ppc64el Packages
Hit http://ports.ubuntu.com vivid-updates/restricted ppc64el Packages
Err http://ppa.launchpad.net vivid/main ppc64el Packages
404 Not Found
Hit http://ports.ubuntu.com vivid-updates/universe ppc64el Packages
Ign http://ppa.launchpad.net vivid/main Translation-en_US
Hit http://ports.ubuntu.com vivid-updates/multiverse ppc64el Packages
Ign http://ppa.launchpad.net vivid/main Translation-en
Hit http://ports.ubuntu.com vivid-updates/main Translation-en
Hit http://ports.ubuntu.com vivid-updates/multiverse Translation-en
Hit http://ports.ubuntu.com vivid-updates/restricted Translation-en
Hit http://ports.ubuntu.com vivid-updates/universe Translation-en
Hit http://ports.ubuntu.com vivid-backports/main ppc64el...

OK, I hadn't realised you are using vivid, I will re-build for that release.

A version for vivid is now building. Please run:

sudo apt-get update
sudo apt-get dist-upgrade

that should get the version with the bug fix ready for testing. Thanks.

------- Comment From <email address hidden> 2015-12-15 08:48 EDT-------
I have installed the fixed fwts packages on the Server and executed the test again.

root@lep8d:~# add-apt-repository ppa:colin-king/fwts-fixes

More info: https://launchpad.net/~colin-king/+archive/ubuntu/fwts-fixes
Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keyring `/tmp/tmp6gvue_om/secring.gpg' created
gpg: keyring `/tmp/tmp6gvue_om/pubring.gpg' created
gpg: requesting key B0F01786 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmp6gvue_om/trustdb.gpg: trustdb created
gpg: key B0F01786: public key "Launchpad PPA for Colin King" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK

root@lep8d:~# apt-get update
Hit http://archive.ubuntu.com vivid InRelease
Hit http://ports.ubuntu.com vivid InRelease
Hit http://archive.ubuntu.com vivid-updates InRelease
Hit http://archive.ubuntu.com vivid-backports InRelease
Hit http://ports.ubuntu.com vivid-updates InRelease
Hit http://archive.ubuntu.com vivid/main Sources
Get:1 http://ppa.launchpad.net vivid InRelease [15.4 kB]
Hit http://ports.ubuntu.com vivid-backports InRelease
Hit http://archive.ubuntu.com vivid/restricted Sources
Hit http://ports.ubuntu.com vivid-security InRelease
Hit http://archive.ubuntu.com vivid/universe Sources
Get:2 http://ppa.launchpad.net vivid/main ppc64el Packages [1,503 B]
Hit http://ports.ubuntu.com vivid/main ppc64el Packages
Hit http://archive.ubuntu.com vivid/multiverse Sources
Hit http://archive.ubuntu.com vivid-updates/main Sources
Hit http://ports.ubuntu.com vivid/restricted ppc64el Packages
Get:3 http://ppa.launchpad.net vivid/main Translation-en [1,034 B]
Hit http://archive.ubuntu.com vivid-updates/restricted Sources
Hit http://ports.ubuntu.com vivid/universe ppc64el Packages
Hit http://archive.ubuntu.com vivid-updates/universe Sources
Hit http://archive.ubuntu.com vivid-updates/multiverse Sources
Hit http://ports.ubuntu.com vivid/multiverse ppc64el Packages
Hit http://archive.ubuntu.com vivid-backports/main Sources
Hit http://ports.ubuntu.com vivid/main Translation-en
Hit http://archive.ubuntu.com vivid-backports/restricted Sources
Hit http://archive.ubuntu.com vivid-backports/universe Sources
Hit http://ports.ubuntu.com vivid/multiverse Translation-en
Hit http://archive.ubuntu.com vivid-backports/multiverse Sources
Hit http://ports.ubuntu.com vivid/restricted Translation-en
Hit http://ports.ubuntu.com vivid/universe Translation-en
Hit http://ports.ubuntu.com vivid-updates/main ppc64el Packages
Hit http://ports.ubuntu.com vivid-updates/restricted ppc64el Packages
Hit http://ports.ubuntu.com vivid-updates/universe ppc64el Packages
Hit http://ports.ubuntu.com vivid-updates/multiverse ppc64el Packages
Hit http://ports.ubuntu.com vivid-updates/main Translation-en
Hit http://ports.ubuntu.com vivid-updates/multiverse Translation-en
Hit http://ports.ubuntu.com vivid-updates/restricted Translation-en
Hit http://ports.ubuntu.com vivid-updates/universe Translation-en
Hit http://ports.ubuntu.com vivid-backports/main ppc64el Packages
Hit http://ports.ubuntu.com vivid-backports/restricted ppc64el Packages
Hit http://ports.ubuntu.com vivid-backports/universe pp...

[SRU Justification][Vivid][Wily]

fwts segfaults because the fwts ACPI table loader is failing to load ACPI tables and was trying to load them from file (as a last resort) and unfortunately was accessing a path that was NULL, hence a null pointer deference.

[Fix]
http://kernel.ubuntu.com/git/hwe/fwts.git/commit/?id=9e686bb49b7f447ee4bcf0c3b06013bfaf1e229b

[Testcase]
Run fwts on a system without ACPI tables. Will segfault. With the fix, it runs w/o a segfault

Hello bugproxy, or anyone else affected,

Accepted fwts into wily-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/fwts/15.09.00-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello bugproxy, or anyone else affected,

Accepted fwts into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/fwts/15.03.01-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I've tested this on some x86 VMs with ACPI disabled to mimic the target platform and I can't trigger the segfault with fwts from -proposed for Vivid and Wily.

Would appreciate it if this is tested on ppc64el

OK, I've tested the code path that caused the segmentation fault on the earlier version and I am 100% sure this is now fixed.

------- Comment From <email address hidden> 2016-01-01 11:21 EDT-------
I have installed the proposed fwts package for vivid release and executed the scenario again.

root@powerkvm5:~# uname -a
Linux powerkvm5 3.19.0-42-generic #48-Ubuntu SMP Thu Dec 17 22:53:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
root@powerkvm5:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="15.04 (Vivid Vervet)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 15.04"
VERSION_ID="15.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

root@powerkvm5:~# which fwts
/usr/bin/fwts

root@powerkvm5:~# dpkg -S /usr/bin/fwts
fwts: /usr/bin/fwts
root@powerkvm5:~# dpkg --list | grep fwts
ii fwts 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite
ii libfwts1 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite core support library
ii libfwtsacpica1 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite ACPICA core built for acpiexec functionality
ii libfwtsiasl1 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite ACPICA core built for iasl functionality

root@powerkvm5:~# fwts --batch --interactive --power-states
Running 15 tests, results appended to results.log
Test: Gather kernel system information.
Gather kernel signature. 1 info only
Gather kernel system information. 1 info only
Gather kernel boot command line. 1 info only
Gather ACPI driver version. 1 info only
Test: Scan kernel log for Oopses.
Kernel log oops check. 2 passed
Test: Scan kernel log for errors and warnings.
Kernel log error check. 1 passed
Test: General ACPI information test.
Determine Kernel ACPI version. 1 info only
Determine machine's ACPI version.
Determine AML compiler. 1 info only
Test: Sanity check for UEFI Boot Path Boot####.
Test aborted..
Test: UEFI secure boot test.
Test skipped..
Test: Test firmware has set PCI Express MaxReadReq to a higher value on n..
Test firmware settings MaxReadReq for PCI Express de.. 1 passed
Test: PCIe ASPM test.
PCIe ASPM ACPI test. 1 skipped
PCIe ASPM registers test. 14 passed, 31 warnings
Test: CPU frequency scaling tests.
CPU frequency table consistency 1 passed
CPU frequency table duplicates 1 passed
CPU frequency firmware limits 1 passed
CPU frequency claimed maximum 1 passed
CPU frequency SW_ANY control 1 skipped
CPU frequency SW_ALL control 1 skipped
CPU frequency performance tests. 1 passed
Test: HDA Audio Pin Configuration test.
HDA Audio Pin Configuration test. 1 skipped
Test: XENV Xen Environment Table tests.
Test...

This bug was fixed in the package fwts - 15.03.01-0ubuntu2

---------------
fwts (15.03.01-0ubuntu2) vivid; urgency=medium

  * lib: fwts_acpi_tables: fix segfault when tables fail to load (LP: #1452201)

 -- Colin King <email address hidden> Fri, 18 Dec 2015 10:40:40 +0000

The verification of the Stable Release Update for fwts has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package fwts - 15.09.00-0ubuntu2

---------------
fwts (15.09.00-0ubuntu2) wily; urgency=medium

  * lib: fwts_acpi_tables: fix segfault when tables fail to load (LP: #1452201)

 -- Colin King <email address hidden> Fri, 18 Dec 2015 10:40:40 +0000

------- Comment From <email address hidden> 2016-01-11 04:28 EDT-------
I have installed Ubuntu 15.10 OS on P8 Hardware and executed the fwts tests again.

root@powerkvmpok002:~# uname -a
Linux powerkvmpok002 4.2.0-23-generic #28-Ubuntu SMP Sun Dec 27 17:46:45 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
root@powerkvmpok002:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="15.10 (Wily Werewolf)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 15.10"
VERSION_ID="15.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

root@powerkvmpok002:~# which fwts
/usr/bin/fwts
root@powerkvmpok002:~# dpkg -S /usr/bin/fwts
fwts: /usr/bin/fwts
root@powerkvmpok002:~# dpkg --list | grep fwts
ii fwts 15.09.00-0ubuntu2 ppc64el FirmWare Test Suite
ii libfwts1 15.09.00-0ubuntu2 ppc64el FirmWare Test Suite core support library
ii libfwtsacpica1 15.09.00-0ubuntu2 ppc64el FirmWare Test Suite ACPICA core built for acpiexec functionality
ii libfwtsiasl1 15.09.00-0ubuntu2 ppc64el FirmWare Test Suite ACPICA core built for iasl functionality

root@powerkvmpok002:~# fwts --batch --interactive --power-states
Running 50 tests, results appended to results.log
Test: Gather kernel system information.
Gather kernel signature. 1 info only
Gather kernel system information. 1 info only
Gather kernel boot command line. 1 info only
Gather ACPI driver version. 1 info only
Test: Scan kernel log for Oopses.
Kernel log oops check. 2 passed
Test: Scan kernel log for errors and warnings.
Kernel log error check. 1 passed
Test: General ACPI information test.
Determine Kernel ACPI version. 1 info only
Determine machine's ACPI version.
Determine AML compiler. 1 info only
Test: Sanity check for UEFI Boot Path Boot####.
Test aborted..
Test: UEFI secure boot test.
Test skipped..
Test: Test firmware has set PCI Express MaxReadReq to a higher value on n..
Test firmware settings MaxReadReq for PCI Express de.. 1 passed
Test: PCIe ASPM test.
PCIe ASPM ACPI test. 1 skipped
PCIe ASPM registers test. 21 passed, 61 warnings
Test: CPU frequency scaling tests.
CPU frequency table consistency 1 passed
CPU frequency table duplicates 1 passed
CPU frequency firmware limits 1 passed
CPU frequency claimed maximum 1 passed
CPU frequency SW_ANY control 1 skipped
CPU frequency SW_ALL control 1 skipped
CPU frequency performance tests. 1 passed
Test: HDA Audio Pin Configuration test.
HDA Audio Pin Configuration test. 1 skipped
Test: XENV Xen Environment Table tests.
Test aborted..
Test: XSDT Extende...

------- Comment From <email address hidden> 2016-01-12 06:55 EDT-------
I have installed Ubuntu 15.04 OS on P8 Hardware and executed the fwts tests again.

root@powerkvmpok002:~# uname -a
Linux powerkvmpok002 3.19.0-43-generic #49-Ubuntu SMP Sun Dec 27 17:50:58 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux

root@powerkvmpok002:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="15.04 (Vivid Vervet)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 15.04"
VERSION_ID="15.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

root@powerkvmpok002:~# which fwts
/usr/bin/fwts
root@powerkvmpok002:~# dpkg -S /usr/bin/fwts
fwts: /usr/bin/fwts
root@powerkvmpok002:~# dpkg --list | grep fwts
ii fwts 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite
ii libfwts1 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite core support library
ii libfwtsacpica1 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite ACPICA core built for acpiexec functionality
ii libfwtsiasl1 15.03.01-0ubuntu2 ppc64el FirmWare Test Suite ACPICA core built for iasl functionality

root@powerkvmpok002:~# fwts --batch --interactive --power-states
Running 15 tests, results appended to results.log
Test: Gather kernel system information.
Gather kernel signature. 1 info only
Gather kernel system information. 1 info only
Gather kernel boot command line. 1 info only
Gather ACPI driver version. 1 info only
Test: Scan kernel log for Oopses.
Kernel log oops check. 2 passed
Test: Scan kernel log for errors and warnings.
Kernel log error check. 1 passed
Test: General ACPI information test.
Determine Kernel ACPI version. 1 info only
Determine machine's ACPI version.
Determine AML compiler. 1 info only
Test: Sanity check for UEFI Boot Path Boot####.
Test aborted..
Test: UEFI secure boot test.
Test skipped..
Test: Test firmware has set PCI Express MaxReadReq to a higher value on n..
Test firmware settings MaxReadReq for PCI Express de.. 1 passed
Test: PCIe ASPM test.
PCIe ASPM ACPI test. 1 skipped
PCIe ASPM registers test. 21 passed, 61 warnings
Test: CPU frequency scaling tests.
CPU Frequency tests. : 58.8% /
CPU Frequency tests. 2 failed
Test: HDA Audio Pin Configuration test.
HDA Audio Pin Configuration test. 1 skipped
Test: ACPI Wakealarm tests.
Test existence of RTC with alarm interface. 1 passed
Trigger wakealarm for 1 seconds in the future. 1 failed
Test if wakealarm is fired. 1 failed
Multiple wakealarm firing tests. 1 failed
Test: Re-...