fix test_allowed_address_pair test case to fix race condition during network deletion

Bug #1894835 reported by Gayathri Devi Kathiri
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tempest
Fix Released
Undecided
Gayathri Devi Kathiri

Bug Description

"test_allowed_address_pair" tempest test cases failed during cleanup with the below error:

Details: {'type': 'SubnetInUse', 'message': 'Unable to complete operation on subnet e6615d0f-171b-433b-9324-3b067e50ff6b: One or more ports have an IP allocation from this subnet.', 'detail': ''}, ), (, Conflict with state of target resource
Details: {'type': 'NetworkInUse', 'message': 'Unable to complete operation on network 863441e6-0cc8-478c-b5b5-b201c3dbfa71. There are one or more ports still in use on the network.', 'detail': ''}, ))

tempest.api.network.test_allowed_address_pair.AllowedAddressPairIpV6TestJSON.test_create_list_port_with_address_pair-- FAIL

tempest.api.network.test_allowed_address_pair.AllowedAddressPairIpV6TestJSON.test_update_port_with_address_pair -- FAIL

tempest.api.network.test_allowed_address_pair.AllowedAddressPairIpV6TestJSON.test_update_port_with_cidr_address_pair -- FAIL

tempest.api.network.test_allowed_address_pair.AllowedAddressPairIpV6TestJSON.test_update_port_with_multiple_ip_mac_address_pair -- FAIL

Looks like subnet/network delete is triggered while port being deleted.

This rare scenario is observed when whole tempest repo test suite is run.
Since the tempest test case failed at cleanup with error indicating that ports are still in use on the network, logged into the environment where test case failed and found that port delete triggered by the tempest is successful. So, there are no ports associated with the network/subnet after test case failure.
Neutron deleted the port as triggered by the tempest. Network/subnet existed in the environment since it failed at the cleanup of networks/subnet.

Tempest code calls the delete_port method but doesn't wait whether port is deleted which could lead to a rare situation when port is being deleted but tempest is already trying to delete the network.

Changed in tempest:
assignee: nobody → Gayathri Devi Kathiri (gayathridevikathiri)
Changed in tempest:
status: New → In Progress
description: updated
Revision history for this message
Martin Kopec (mkopec) wrote :

There has been the same problem with other network tests recently: https://bugs.launchpad.net/tempest/+bug/1893487

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tempest (master)

Reviewed: https://review.opendev.org/750329
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=a09be7e018c9e3b59cb4707b9146f336552b060d
Submitter: Zuul
Branch: master

commit a09be7e018c9e3b59cb4707b9146f336552b060d
Author: Gayathri Devi Kathiri <email address hidden>
Date: Tue Sep 8 11:54:07 2020 +0000

    Fix test_allowed_address_pair test case cleanup

    Make network cleanup to wait until its
    dependent ports are cleaned up

    Change-Id: I91f5ced7873755081f94117c6a43bffcb85faab6
    Closes-bug: #1894835

Changed in tempest:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tempest 25.0.1

This issue was fixed in the openstack/tempest 25.0.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.