Happenstance testing is a test smell where you are asserting something more specifically than relevant to what you’re testing. For example, trying to assert the order of items in a set can lead to disappointment when your runtime changes its algorithm for arranging sets.
Similarly, asserting the exact test of an error message, unless it’s the construction of the message that’s the thing under test, can lead to brittle test failures when the message is improved in a way that the test shouldn’t care about.
Here’s a sanitised example of a recent test failure:
1 | StringProcessorTest.subStringExtractionMustBeInRange:407 expected:<[begin -1, end 3, length 5]> but was:<[String index out of range: -1]> |
The original assertion was checking that some sort of error was thrown, but the assertion had been loaded with the specific text of the error from the string library in Java. One Java upgrade later, and this failed. Clearly, the error message wasn’t the point.
In general, make sure only to assert stuff you care about.
Published on Java Code Geeks with permission by Ashley Frieze, partner at our JCG program. See the original article here: Exceptional Happenstances Opinions expressed by Java Code Geeks contributors are their own. |
