Tempest ip filtering assumes certain ip allocation

Related fix proposed to branch: master
Review: https://review.openstack.org/274907

http://logs.openstack.org/42/273042/3/check/gate-tempest-dsvm-full/5864f86/logs/screen-n-api.txt.gz#_2016-02-01_18_11_29_658 is an example of the filter in action, with a probably unexpected value.

Reviewed: https://review.openstack.org/274907
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=a3c0fb0a6767be71482c4308d6d3d9b713183e0b
Submitter: Jenkins
Branch: master

commit a3c0fb0a6767be71482c4308d6d3d9b713183e0b
Author: Sean Dague <email address hidden>
Date: Mon Feb 1 17:01:45 2016 -0500

    skip test_list_server_filters_by_regex

    The test for the regex only works if we are always allocating linearly
    from the bottom of the network block or on a small network
    block. We're changing the IP allocator in Nova to be more resiliant,
    and this test can be proven to be trivially wrong in that case.

    It should be skipped until it can be fixed.

    Change-Id: I27f73c1edf12218818c4d279efbd9fef5cdef672
    Related-Bug: #1540645

I am not able to confirm and the link seems to be broken. Please provide a working link if possible.

The log just went away because of the log rotation.

This test is still enabled in defcore (2016.8). How are we supposed to pass defcore in the mean time?

[Expired for tempest because there has been no activity for 60 days.]

The test is skipped in tempest citing this bug, so it shouldn't be closed.

To trigger the issue reliably, use some large enough subnet by e.g. setting

[network]
project_network_cidr = 10.0.0.0/8
project_network_mask_bits = 12

in tempest.conf.

You're right Dr. Jens Harbott, thank you. I set the network values as you suggested and I was able reproduce the issue on the second run.

{0} tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filtered_by_ip_regex [0.712201s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/stack/tempest/tempest/api/compute/servers/test_list_server_filters.py", line 312, in test_list_servers_filtered_by_ip_regex
    (self.s2_name, servers, all_servers))

      File "/home/stack/tempest/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 421, in assertIn
    self.assertThat(haystack, Contains(needle), message)

      File "/home/stack/tempest/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 502, in assertThat
    raise mismatch_error

    testtools.matchers._impl.MismatchError: 'tempest-ListServerFiltersTestJSON-instance-1400666750' not in <map object at 0x7f98f7c19cf8>: tempest-ListServerFiltersTestJSON-instance-1400666750 not found in [{'id': '9159497a-50e9-4006-a854-1dd97cd76b1f', 'name': 'tempest-ListServerFiltersTestJSON-instance-348922714', 'links': [{'rel': 'self', 'href': 'http://10.0.0.145:8774/v2.1/servers/9159497a-50e9-4006-a854-1dd97cd76b1f'}, {'rel': 'bookmark', 'href': 'http://10.0.0.145:8774/servers/9159497a-50e9-4006-a854-1dd97cd76b1f'}]}], all servers response: {'date': 'Tue, 23 Jun 2020 08:11:20 GMT', 'server': 'Apache', 'content-length': '4652', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-e04652e6-b382-48a0-a154-801a68d6c6a7', 'x-compute-request-id': 'req-e04652e6-b382-48a0-a154-801a68d6c6a7', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.145:8774/v2.1/servers/detail'}
Body: {'servers': [{'id': '6f7a59cb-2d21-45c1-bc04-c6ff72fbfe5c', 'name': 'tempest-ListServerFiltersTestJSON-instance-1739922658', 'status': 'ACTIVE', 'tenant_id': 'eb9df7d40dd3498badc2f5f41b0b3414', 'user_id': 'e4c927333433407d96feaa401ec8dfef', 'metadata': {}, 'hostId': 'fc3705f3c79aee895d127ecdbeb0e81af6429ad642c16961e6f60a67', 'image': {'id': 'cbc5f817-eea2-43de-b3be-b350b7101038', 'links': [{'rel': 'bookmark', 'href': 'http://10.0.0.145:8774/images/cbc5f817-eea2-43de-b3be-b350b7101038'}]}, 'flavor': {'id': '62825840-ccf1-41b6-9887-8b91602f9f4b', 'links': [{'rel': 'bookmark', 'href': 'http://10.0.0.145:8774/flavors/62825840-ccf1-41b6-9887-8b91602f9f4b'}]}, 'created': '2020-06-23T08:11:10Z', 'updated': '2020-06-23T08:11:19Z', 'addresses': {'tempest-ListServerFiltersTestJSON-742853868-network': [{'version': 4, 'addr': '10.0.2.208', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:76:36:30'}]}, 'accessIPv4': '', 'accessIPv6': '', 'links': [{'rel': 'self', 'href': 'http://10.0.0.145:8774/v2.1/servers/6f7a59cb-2d21-45c1-bc04-c6ff72fbfe5c'}, {'rel': 'bookmark', 'href': 'http://10.0.0.145:8774/servers/6f7a59cb-2d21-45c1-bc04-c6ff72fbfe5c'}], 'OS-DCF:diskConfig': 'MANUAL', 'progress': 0, 'OS-EXT-AZ:availability_zone': 'no...

Fix proposed to branch: master
Review: https://review.opendev.org/737545

Reviewed: https://review.opendev.org/737545
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=0a07a0e51f004f79d1ddbc448371c55cfb87ce40
Submitter: Zuul
Branch: master

commit 0a07a0e51f004f79d1ddbc448371c55cfb87ce40
Author: Martin Kopec <email address hidden>
Date: Tue Jun 23 13:54:49 2020 +0000

    Fix and unskip test_list_servers_filtered_by_ip_regex

    The test assumes that all ip addresses will be allocated in a very
    specific and static way. The patch improves the calculation of the
    ip regex.

    Change-Id: Icb871c2b106ee0b9770b89d15c54bef65297977c
    Closes-Bug: #1540645

This issue was fixed in the openstack/tempest 25.0.1 release.