tempest

Tempest tests fail with skip exception for stestr > 2.5.0 due to teststool #issue 272

Bug #1847749 reported by Ghanshyam Mann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tempest
Fix Released
Undecided
Ghanshyam Mann

Bug Description

stestr which is used by Tempest internally to run the test switch
the customize test runner(which use stdlib unittest) for >=py3.5
else testtools.run.- https://github.com/mtreinish/stestr/pull/265

These two test runner are not compatible due to skip exception
handling(due to unittest2). testtools.run treat unittestt.SkipTest
as error and stdlib unittest treat unittest2.case.SkipTest raised
by testtools.TestCase.skipException.

testtool issue: https://github.com/testing-cabal/testtools/issues/272

testtool is not so active now a days and fix is also not decided,
let's add a workaround in Tempest.

Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

Fix 1 - https://review.opendev.org/#/c/681340/

Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

not fix but workaround ^^.

Woraround for CLI base test class- https://review.opendev.org/#/c/687999/5

OpenStack Infra (hudson-openstack)
Changed in tempest:
assignee: nobody → Ghanshyam Mann (ghanshyammann)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tempest (master)

Reviewed: https://review.opendev.org/687999
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=e64c78dcf720202a0542bb1e1184f5229a11524f
Submitter: Zuul
Branch: master

commit e64c78dcf720202a0542bb1e1184f5229a11524f
Author: Ghanshyam Mann <email address hidden>
Date: Thu Oct 10 22:26:43 2019 +0000

    Add workaround to handle the testtool skip exception in CLI test

    This commit apply the workaround for testtool skip exception
    in CLI test base class.
    - https://review.opendev.org/#/c/681340/6

    I have modified the workaround done in above commit to more generic
    one.

    We are good on skipException from test code. That is handled by
    teststools.TestCase's self.exception_handlers
    - https://github.com/testing-cabal/testtools/blob/f51ce5f934153e80d3e8a95b52e1464daeb30c14/testtools/testcase.py#L275
    Mapped handler _report_skip() will add the respective tests to skip list

    Current CLI test failure
    -https://zuul.opendev.org/t/openstack/build/c1bb61121e1d4d108c52adad200e8991/log/job-output.txt#6354

    I have tested both test.BaseTestCase and lib.base.BaseTestCase with all 8 combinations
    of below:
     - py >= 3.5 and py < 3.5 (on py2.7)
     - stestr >= 2.50 and stestr < 2.5.0
     - skip exception from setUpClass and skip exception from test code

    Closes-Bug: 1847749

    Change-Id: Ib70bdffaf4d38743e7cbbeb88af51bb57ceeedf6

Changed in tempest:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tempest 22.1.0

This issue was fixed in the openstack/tempest 22.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.