A JUnit Rule to Conditionally Ignore Tests

I always believed that using @Ignore to deactivate tests is a bad idea. Except, maybe as one way to put tests that fail intermittently into quarantine to attend to them later (as Martin Fowler describes it here). This bears the danger that the test suite decays as more and more tests keep getting ignored and forgotten. Therefore you should have a policy in place to ensure that tests aren’t quarantined for too long. Well, so I thought until recently when we ran into this:

In a project that Frank and I work on, we ran into an SWT issue described here.
On non-Windows platforms, asserting whether an SWT widget has got the input focus does not work with automated tests.

We decided to ignore focus-related tests on non-Windows platforms for now. Though our build server runs on Linux we found it safe enough as both our development environments run on Windows.

In JUnit, assumptions are the means to skip tests that aren’t meaningful under the given condition. Expressed that way, our test would look like this:

public void testFocus() {
  assumeTrue( isRunningOnWindows() );
  // ...
}

But we didn’t want the test code mingled with conditions if it should be executed at all. The code that decides whether the test is ignored should be separated from the test code itself.

This led us to creating a ConditionalIgnore annotation and a corresponding rule to hook it into the JUnit runtime. The thing is simple and best explained with an example:

public class SomeTest {
  @Rule
  public ConditionalIgnoreRule rule = new ConditionalIgnoreRule();

  @Test
  @ConditionalIgnore( condition = NotRunningOnWindows.class )
  public void testFocus() {
    // ...
  }
}

public class NotRunningOnWindows implements IgnoreCondition {
  public boolean isSatisfied() {
    return !System.getProperty( "os.name" ).startsWith( "Windows" );
  }
}

The ConditionalIgnore annotation requires a ‘condition’ property that points to a class that implements IgnoreContition. At runtime, an instance of the IgnoreCondition implementation is created and its isSatisfied() method decides whether the test is ignored (returns true) or not (returns false). Finally there is an IgnoreConditionRule that hooks the annotations into the JUnit runtime.

If the IgnoreCondition implementation decides to ignore a test case, an AssumptionViolatedException is thrown. Therefore the ConditionalIgnore annotation has the same effect as if an Assume condition would return false. With a slight difference that we consider an advantage: @Before and @After methods are not executed for ignored tests.

The source code of the rule and its related classes can be found here.

The remaining issue with Assume is that it affects the test statistics. If an Assume condition is found to be false, it treats the test as having passed even though it hasn’t run. To overcome that, you’d have to provide your own runner that handles AssumptionViolatedException the way you want.

Even though I just wrote about ignoring tests in length, I am still convinced that tests at best should not be ignored and if so only in exceptional cases.
 

Reference: A JUnit Rule to Conditionally Ignore Tests from our JCG partner Rudiger Herrmann at the Code Affine blog.

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

JPA Mini Book

Learn how to leverage the power of JPA in order to create robust and flexible Java applications. With this Mini Book, you will get introduced to JPA and smoothly transition to more advanced concepts.

JVM Troubleshooting Guide

The Java virtual machine is really the foundation of any Java EE platform. Learn how to master it with this advanced guide!

Given email address is already subscribed, thank you!
Oops. Something went wrong. Please try again later.
Please provide a valid email address.
Thank you, your sign-up request was successful! Please check your e-mail inbox.
Please complete the CAPTCHA.
Please fill in the required fields.

Leave a Reply


two × 8 =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close