Test framework: Skipped runtime errors are still "unexpected" even if compile error is expected

Bug #596734 reported by Matt Giuca
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mars
Fix Released
Medium
Matt Giuca

Bug Description

If the MTC file includes under the "compile" section "expect: compile_error", this indicates that the test is supposed to compile successfully but currently doesn't, and not to be noisy about it (report the compile failure as an expected fail).

However, if the compile fails, all of the runtime tests are reported as "skipped" (which generates an 'x' in the visual output, and does not count as a pass/expected fail). For a compile error which has been expected, all of the skips should also be expected. Therefore, the 'x' should instead be a grey '.', and the skips should count towards the pass/expected fail total, not the skipped total (such that you can still get 100% pass/expected fail).

For example, current output:
mytest: .xxx (one red dot, three grey xs)
1/4 cases passed (or expected fail)
3 cases skipped

Expected output:
mytest: .... (one red dot, three grey dots)
4/4 cases passed (or expected fail)

Tags: testing

Related branches

Revision history for this message
Matt Giuca (mgiuca) wrote :

Fixed in trunk r1103.

Changed in mars:
status: Triaged → Fix Committed
Matt Giuca (mgiuca)
Changed in mars:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.