Test Cases: Negative test cases for Server Metadata

Fix proposed to branch: master
Review: https://review.openstack.org/8280

I am still trying to understand how we are deciding which argument variants for APIs are supposed to have a Tempest test. The nova unit test in nova/nova/tests/api/openstack/compute/test_server_metadata.py has these tests and a zillion more. I thought the point was to test things that unit tests *could not* because the unit tests splice out real vm or db operations in order to be fast, or because multiple components are involved. But all these test cases end up in the same code as the unit tests do, just more slowly.

It seems to me we should have a few cases to make sure the basic code path works, but only be testing lots of multiple argument variants when we expect that they could induce different behavior in parts of the system that the unit tests are stubbing out. What do other people think?

I don't mean to pick on this bug in particular. It applies to all classes of Tempest test cases.

David - I got your point.
1. We would like to have negative tests coverage so that those tests are run through application server in complex deployments to check stability of the integrated environment . We do not run unit tests in those deployments.
2. With annotataor , those negative tests can be chosen to run or skip.
3. I agree. we will balance and limit those negative tests .

Thanks, Ravi. I am just trying to extract a principle for what it means for a Tempest test of some API to be "done". It is important that we document this and I am not saying it is trivial to do so.

We are going to revisit negative test cases separately using a fuzz testing tool.