errinjct tool error while running EEH tests (powerpc-utils)

Bug #1597365 reported by bugproxy
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
powerpc-utils (Ubuntu)
Fix Released
Medium
Adam Conrad
Xenial
Fix Released
Medium
Steve Langasek

Bug Description

[SRU Justification]
The errinjct tool cannot run EEH tests on certain systems. This is a relatively low priority hardware enablement SRU (low priority because the tool in question is a test tool only).

[Test case]
1. On a Shiner-T or Grand Canyon system, run the command: sudo errinjct eeh -v -f 6 -s net/enp1s0d0 -a 0x82100000 -m 0x0000FFFFFF800000
2. Verify that this returns an error.
3. Enable xenial-proposed, and upgrade powerpc-utils.
4. Rerun the errinjct command. Verify that it now succeeds.

[Regression potential]
Minimal; this is a test tool only so any regressions will be low impact, and the upstream changes that have been cherry-picked are reasonably limited in scope.

== Comment: #0 - Manvanthara B. Puttashankar - 2016-06-29 01:40:42 ==
---Problem Description---
errinjct tool error while running EEH tests (powerpc-utils)

---uname output---
Linux ltciofvtr-s822l2-lp3 4.4.0-24-generic #43-Ubuntu SMP Wed Jun 8 19:25:36 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = s822l

---Steps to Reproduce---
 errinjct tool needs to pick two patch to address the below error

errinjct eeh -v -f 6 -s net/eth1 -a 0xde000000 -m 0x0000FFFFFF800000
Injecting an ioa-bus-error with the following data:

BUS ADDR: de000000
ADDR MASK: ffffff800000
CONFIG ADDR: 900000
PHB UNIT_ID: 800000020000135
FUNCTION: 6
Store to PCI Memory Address Space - inject an Adress Parity Error
errinjct: Could not open RTAS error injection facility
errinjct: librtas returned an unknown error code (-50331649) for function ioa-bus-error

Parent bug: Bug 121866 : EEH via errinjct tool fails with librtas error in Ubuntu 16.04 (Shiner-T & Grand Canyon)

errinjct issues will be solved with multiple patches 1 librtas + 2 errijnct.

Based on suggestion from developer in Bug 121866 reporting this specific issue to powerpc-utils package , two patch needs to be picked by canonical.

Note: have verified the powerpc-utils for 16.04.01 , the patch hasn't gone into yet.

upstream patch fix link: https://github.com/nfont/powerpc-utils/tree/next/src/errinjct

Userspace tool common name: errinjct

The userspace tool has the following bit modes: ppc64le

Userspace rpm: powerpc-utils

== Comment: #1 - Manvanthara B. Puttashankar <email address hidden> - 2016-06-29 01:48:02 ==
commits to be picked up

commit 386a07c4dc6c09d35116dc6064f7a83d76f4415b
Author: Nathan Fontenot <email address hidden>
Date: Tue May 24 16:32:39 2016 -0400

    errinjct: Correct read_file() error checking

    When reading files from syfs the file length returned from a stat()
    call does not always match the actual length of the file. The read_file()
    routine should be checking for an error return from the read() call
    instead of checking that the amount read matches the file length returned
    from a stat() call.

    Signed-off-by: Nathan Fontenot <email address hidden>

commit 42b83884bfb51c0c8b23bf2949157430f8468caa
Author: Nathan Fontenot <email address hidden>
Date: Tue May 24 16:31:37 2016 -0400

    errinjct: Recognize -5 return from rtas

    Update the errinjct command to recognize the rturn code of -5 from
    rtas indicating that PCI error injection is not enabled.

    Signed-off-by: Nathan Fontenot <email address hidden>

commit ab848a8d5cc4c4b9f5faef1b31be3982fb7d4bf2
Author: Nathan Fontenot <email address hidden>
Date: Tue May 24 16:30:33 2016 -0400

    errinjct: Correct config_addr parsing

    A recent patch (commit id: ccc9c72cec2a4) to move the errinjct function to
    using a common read_file() routine did not correctly update the
    get_config_addr_from_reg() function. This patch corrects this to return the
    proper config address.

    Signed-off-by: Nathan Fontenot <email address hidden>

bugproxy (bugproxy)
tags: added: architecture-ppc64le bugnameltc-143197 severity-critical targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Taco Screen team (taco-screen-team)
affects: ubuntu → powerpc-utils (Ubuntu)
Revision history for this message
Steve Langasek (vorlon) wrote :

Triaging as 'medium', not the critical indicated in the bug tags. This is a test utility whose failure has no runtime impact on the system.

Changed in powerpc-utils (Ubuntu):
assignee: Taco Screen team (taco-screen-team) → Adam Conrad (adconrad)
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-07-19 08:41 EDT-------
Any update on this one?

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-07-19 14:03 EDT-------
Please note that the new release of the powerpc-utils-1.3.2 contains these patches.

There is also a new librtas-2.0.1 release that contains errinjct fixes.

Revision history for this message
Adam Conrad (adconrad) wrote :

Our librtas 2.0.0 contains a cherrypick of the errinjct fixes, and I'm uploading powerpc-utils 1.3.2 right now.

Will cherrypick the ppc-utils fixes to xenial after that.

Revision history for this message
Adam Conrad (adconrad) wrote :

Fixed in yakkety in 1.3.2-1.

Changed in powerpc-utils (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-07-29 08:27 EDT-------
i still see the source as 1.3.1-2 which doesn't have the patches, Am i missing something? or the packages haven't been updated yet?

root@ubuntu:~# dpkg -s powerpc-utils
Package: powerpc-utils
Status: install ok installed
Priority: important
Section: metapackages
Installed-Size: 11
Maintainer: Ubuntu Developers <email address hidden>
Architecture: ppc64el
Version: 1.3.1-2ubuntu0.1
Depends: powerpc-ibm-utils
Description: utilities for maintenance of PowerPC platforms (metapackage)
The powerpc-utils metapackage pulls in the appropriate tools for each
Debian PowerPC port.
Homepage: http://powerpc-utils.ozlabs.org/
Original-Maintainer: Adam Conrad <adconrad@0c3.net>

root@ubuntu:~# errinjct eeh -v -f 6 -s net/enp1s0d0 -a 0x82100000 -m 0x0000FFFFFF800000
errinjct: Could not open file /sys/class/net/enp1s0d0/device/devspec, No such file or directory

Unable to find info for net/enp1s0d0:
ADDR MASK: ffffff800000
CONFIG ADDR: 0
PHB UNIT_ID: 000000000

Please try again or use the -p, -c, -h, and -l flags

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 1597365] Comment bridged from LTC Bugzilla

On Fri, Jul 29, 2016 at 12:29:31PM -0000, bugproxy wrote:
> ------- Comment From <email address hidden> 2016-07-29 08:27 EDT-------
> i still see the source as 1.3.1-2 which doesn't have the patches, Am i
> missing something? or the packages haven't been updated yet?

The bug closure message shows that this bug has been fixed for the upcoming
yakkety release, not for 16.04.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-08-26 17:38 EDT-------
Canonical,

The Xenial portion of the bug is still marked as "New". Can we get a status update on the Xenial side?

Thanks.

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-09-30 05:20 EDT-------
Currently version still at 1.3.1-2 on xenial

tags: added: targetmilestone-inin16042
removed: targetmilestone-inin---
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-09-30 10:38 EDT-------
(In reply to comment #15)
> Currently version still at 1.3.1-2 on xenial

For xenial are there plans to move to 1.3.2? or do we need to provide backports of these patches?

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-10-27 16:34 EDT-------
Hi Canonical.

We are still looking for some sort of response to the Xenial queries.

Thanks.

Steve Langasek (vorlon)
description: updated
Changed in powerpc-utils (Ubuntu Xenial):
importance: Undecided → Medium
status: New → In Progress
assignee: nobody → Steve Langasek (vorlon)
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted powerpc-utils into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.1-2ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in powerpc-utils (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
tags: added: verification-done
removed: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Setting back to verification-needed since no details about the verification were provided.

tags: added: verification-needed
removed: verification-done
Revision history for this message
Steve Langasek (vorlon) wrote :

Breno, please provide details of how you verified this powerpc-utils fix so we have a record of it.

Revision history for this message
Breno Leitão (breno-leitao) wrote :

Hi Steve,

I tested the following package version using the eeh target, and the problem is not solved on the following version. This could migrate to the main archive IMO.

powerpc-utils | 1.3.1-2ubuntu0.2 | http://ports.ubuntu.com/ubuntu-ports xenial-proposed/main ppc64el Packages

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package powerpc-utils - 1.3.1-2ubuntu0.2

---------------
powerpc-utils (1.3.1-2ubuntu0.2) xenial; urgency=medium

  * d/p/0001-errinjct-Correct-config_addr-parsing.patch,
    d/p/0002-errinjct-Recognize-5-return-from-rtas.patch,
    d/p/0003-errinjct-Correct-read_file-error-checking.patch:
    Cherry-pick fixes from upstream to fix failures with errinjct on
    Shiner-T & Grand Canyon systems. LP: #1597365.

 -- Steve Langasek <email address hidden> Sat, 14 Jan 2017 18:57:24 -0800

Changed in powerpc-utils (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.