Elastic-recheck finding patterns in successful tests

Jenkins failed for a review[1]. The only failing testsuite was gate-tempest-dsvm-neutron-large-ops . Elastic-recheck reports that it thinks the failure was [2]. So I looked through the logs for gate-tempest-dsvm-neutron-large-ops and the signature[3] is not present.

I noticed there was a comment in [2] that this shouldn't cause grenade to fail, and sure enough the signature[3] shows up in the check-grenade-dsvm log.

So I think what's happening is that elastic-recheck looking at logs from successful testsuites and not just the failed ones as expected.

[1] https://review.openstack.org/#/c/62068/5
[2] https://bugs.launchpad.net/bugs/1259907
[3] "Failed to upload testbucket/bundle.img.manifest.xml"