puppet-neutron-tripleo-standalone is continuously failing/timing out

timing out in tempest
https://31ecc5aff6d6154e412c-498f1015c47ee06cc2525a14ef8131d8.ssl.cf5.rackcdn.com/716197/4/check/puppet-neutron-tripleo-standalone/fdeccba/job-output.txt

same thing here: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_dce/716661/1/check/tripleo-ci-centos-8-containers-multinode/dceef0c/job-output.txt

http://zuul.openstack.org/builds?job_name=puppet-neutron-tripleo-standalone

hrm..

- job:
    name: puppet-neutron-tripleo-standalone
    parent: tripleo-puppet-ci-centos-8-standalone
    description: |
      TripleO standalone Job running Neutron specific Tempest tests.
    vars:
      featureset_override:
        run_tempest: false
        tempest_test_whitelist:
          - 'network'
        use_os_tempest: true

whitelist: 'network'
From a successful job: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8b1/715890/1/check/puppet-neutron-tripleo-standalone/8b12069/logs/undercloud/var/log/tempest/stestr_results.html

So.. I suspect one the network scenarios is getting hung up.

https://review.opendev.org/716823 - just in case .. move to non-voting

create a job in the component pipeline w/ the exact tempest config
https://review.rdoproject.org/r/26213

add monitoring .. https://review.rdoproject.org/r/26214

I believe that the problem is that current tempest scope 'network' is too wide and cause many long running tests.
I don't see any gate jobs testing the same in tripleo, and it doesn't make sense to have wider jobs in puppet side.
We see the patch in puppet-nova to have more specific jobs.
 https://github.com/openstack/puppet-nova/commit/b784174c3cb618b676badb3719ce84a4e19fb0e6

I submitted a change to puppet-neutron to implement the same.
 https://review.opendev.org/#/c/716952

The issue was fixed in the change in puppet-neutron[1], which makes tempest execute less test cases to avoid timeout.
 [1] https://review.opendev.org/#/c/716197/

@weshayutin

I think there is nothing we can do from tripleo side, and we can close the bug at tripleo side as WON'T FIX or INVALID.
I'll leave the current status at the moment to ask your confirmation about no more action plans regarding this, but please feel free to close this.

Reviewed: https://review.opendev.org/717040
Committed: https://git.openstack.org/cgit/openstack/puppet-neutron/commit/?id=175f063b182c627100404621961371b73b7e6f03
Submitter: Zuul
Branch: stable/train

commit 175f063b182c627100404621961371b73b7e6f03
Author: Takashi Kajinami <email address hidden>
Date: Thu Apr 2 23:53:56 2020 +0900

    Limit scope of puppet-neutron-tripleo-standalone tempest tests

    This is the partial backport of the gate job fix[1] in master, which
    limits scope of tempest tests executed in tripleo job to avoid timeout
    error in gate jobs.

    [1] 6cb1d1af735a0fbd2d811ec40fecbd7d1c987898

    Closes-Bug: #1870257
    Change-Id: I6f72d6be245b5c530de0746fb8b7cec26748bb45

Reviewed: https://review.opendev.org/717041
Committed: https://git.openstack.org/cgit/openstack/puppet-neutron/commit/?id=d6fb669f0d3a91942d9a88b606e0b52bc5ec1773
Submitter: Zuul
Branch: stable/stein

commit d6fb669f0d3a91942d9a88b606e0b52bc5ec1773
Author: Takashi Kajinami <email address hidden>
Date: Thu Apr 2 23:53:56 2020 +0900

    Limit scope of puppet-neutron-tripleo-standalone tempest tests

    This is the partial backport of the gate job fix[1] in master, which
    limits scope of tempest tests executed in tripleo job to avoid timeout
    error in gate jobs.

    [1] 6cb1d1af735a0fbd2d811ec40fecbd7d1c987898

    Conflicts:
     .zuul.yaml

    Closes-Bug: #1870257
    Change-Id: I6f72d6be245b5c530de0746fb8b7cec26748bb45
    (cherry picked from commit 175f063b182c627100404621961371b73b7e6f03)

Change abandoned by Alex Schultz (<email address hidden>) on branch: master
Review: https://review.opendev.org/716823
Reason: we fixed the glitch

This issue was fixed in the openstack/puppet-neutron 16.2.0 release.

This issue was fixed in the openstack/puppet-neutron 16.2.1 release.

IIUC, this is can be closed on tripleo as well.

This issue was fixed in the openstack/puppet-neutron stein-eol release.