test_images contains some inefficient and wrong test cases

Bug #1010980 reported by Jay Pipes
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tempest
Fix Released
High
Attila Fazekas

Bug Description

There are a number of issues with the negative tests in tempest/tests/test_images.py that need to be addressed:

1) There are many test methods that create a new server and then test that an image cannot be created in various vm state transitions/statuses. These tests should be combined and use a single server instance instead of creating dozens

2) Some tests just don't make any sense at all. For instance:

    @attr(type='negative')
    def test_delete_image_that_is_not_yet_active(self):
        """Return an error while trying to delete an active that is creating"""

        server = self.create_server()

        snapshot_name = rand_name('test-snap-')
        resp, body = self.client.create_image(server['id'], snapshot_name)
        image_id = data_utils.parse_image_id(resp['location'])
        self.image_ids.append(image_id)

        # Do not wait, attempt to delete the image, ensure it's successful
        resp, body = self.client.delete_image(image_id)
        self.assertEqual('204', resp['status'])
        self.image_ids.remove(image_id)

        self.assertRaises(exceptions.NotFound, self.client.get_image, image_id)

The above test is:

 * Creating an instance
 * Snapshotting the instance
 * Deleting the snapshot (image)
 * Asserting that the snapshot does not then appear

This test, however:

 a) Does not check that the image snapshot is actually not yet active... it just assumes it isn't active
 b) The docstring doesn't make sense...
 c) This set of actions is already tested extensively in other tests in the test case

So, IMHO, it should be removed. It represents something that unit tests should cover, not really functional tests.

3) Some tests check for seemingly incorrect assertions. For example:

    @attr(type='negative')
    def test_create_image_from_deleted_server(self):
        """An image should not be created if the server instance is removed """
        server_name = rand_name('server')
        resp, server = self.servers_client.create_server(server_name,
                                                         self.image_ref,
                                                         self.flavor_ref)
        self.servers_client.wait_for_server_status(server['id'], 'ACTIVE')

        # Delete server before trying to create server
        self.servers_client.delete_server(server['id'])

        try:
            # Create a new image after server is deleted
            name = rand_name('image')
            meta = {'image_type': 'test'}
            resp, body = self.client.create_image(server['id'], name, meta)

        except:
            pass

        else:
            image_id = data_utils.parse_image_id(resp['location'])
            self.client.wait_for_image_resp_code(image_id, 200)
            self.client.wait_for_image_status(image_id, 'ACTIVE')
            self.client.delete_image(image_id)
            self.fail("Should not create snapshot from deleted instance!")

The except: block is bare... what exception is being expected here?

And this:

    @attr(type='negative')
    def test_create_image_when_server_is_building(self):
        """Return error when creating an image of a server that is building"""
        server_name = rand_name('test-vm-')
        resp, server = self.servers_client.create_server(server_name,
                                                       self.image_ref,
                                                       self.flavor_ref)
        self.servers.append(server)
        snapshot_name = rand_name('test-snap-')
        self.assertRaises(exceptions.Duplicate, self.client.create_image,
                          server['id'], snapshot_name)

Why is Duplicate expected here?

Jay Pipes (jaypipes)
Changed in tempest:
importance: Undecided → High
status: New → In Progress
assignee: nobody → Jay Pipes (jaypipes)
Revision history for this message
Jay Pipes (jaypipes) wrote :
Revision history for this message
Rohit Karajgi (rohitk) wrote :

Hi Jay,

For the case "def test_create_image_when_server_is_building" above, I think nova would throw "409: "conflictingRequest"
for the Create Image request. Tempest rest_client raises exceptions.Duplicate when it gets a 409 error code.

So I think that exception should just be called conflictingRequest in Tempest instead of Duplicate.

Changed in tempest:
assignee: Jay Pipes (jaypipes) → Attila Fazekas (afazekas)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tempest (master)

Fix proposed to branch: master
Review: https://review.openstack.org/19454

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tempest (master)

Reviewed: https://review.openstack.org/19454
Committed: http://github.com/openstack/tempest/commit/46a1d9223d53f6c02fe284e9b7d652a554349fc7
Submitter: Jenkins
Branch: master

commit 46a1d9223d53f6c02fe284e9b7d652a554349fc7
Author: Attila Fazekas <email address hidden>
Date: Fri Jan 11 10:19:42 2013 +0100

    Refactor compute image tests

    Several test cases are combined to use a single server instance instead of
    creating dozens of servers.

    Bug: #1010980

    Change-Id: I4af98ccf53c666ba38630fc3e79a2b3811ede595

Changed in tempest:
status: In Progress → Fix Released
Sean Dague (sdague)
Changed in tempest:
milestone: none → havana-3
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.